Transformer attention.
Follow tokens through a query, key, and value pipeline. Compare attention patterns and inspect a token-to-token matrix.
Your experiment starts here
Ready to explore?
Runs in your browser.
Educational simulation: attention patterns and next-token probabilities are illustrative, not outputs from a trained language model.
Quick start
Get hands-on.
Enter up to 10 tokens or choose an example. Select an attention head and toggle the matrix to compare simulated patterns.
What to look for
- Queries, keys, values, and scaled dot-product attention.
- Row-wise softmax normalization of attention scores.
- How different attention heads can emphasize different relationships.