Fly through GPT-2's residual stream
Each layer has 768 dimensions. Watch how the model's prediction evolves from input to output.
๐ฏ CLICK A BALL
See what feature it represents, why it's firing, and trace its circuit upstream/downstream.
๐ฎ LOGIT LENS
The prediction box shows what GPT-2 would output at each layer. Click it to trace that prediction's circuit.
โก BOS GROUNDING
The central pole is the BOS (beginning-of-sequence) token. All tokens attend to it - it grounds the computation.
WASD/QE fly โข RMB look โข SHIFT boost
ESC deselect โข R reset โข X look at tower