The claim that neural networks represent meaningful features — “this token is a verb,” “this image contains a dog,” “this concept is sarcasm” — as directions in some activation space. To find whether a feature is active for a given input, you take a dot product of the activation vector with that feature’s direction. To compose features, you add their directions. Linearity is the operational claim: features behave like vectors, not like arbitrary regions.
They aren’t, strictly. The nonlinearities (ReLU, GELU, softmax) are doing real work. But the representations between nonlinearities — the residual stream in a transformer, the hidden activations in an MLP layer — appear to be largely linear in their feature content. The Anthropic interpretability team has accumulated a lot of evidence here: probing classifiers work, steering vectors work, activation arithmetic (“king - man + woman ≈ queen”) works in modern models too.
This is where superposition comes in. A vector space of dimension d has d orthogonal directions. If features had to be orthogonal, a 4096-dimensional residual stream could hold at most 4096 features. But networks behave as if they’re storing far more features than that — maybe 10× to 100× more, by various estimates.
Almost-orthogonality is cheap in high dimensions. In a 4096-dimensional space, you can fit exponentially many vectors that have pairwise cosine similarity below, say, 0.1. This is the Johnson-Lindenstrauss regime: random projections preserve approximate distances. So if features are tolerant of a little interference, you can pack many more of them than you have dimensions. The cost is noise: when feature A is active, it leaks a small amount onto feature B’s direction.
Sparsity. Most features are inactive on any given input. If only a small fraction of features fire at a time, the expected interference at any one feature’s direction stays low. Superposition is a bet that the feature distribution is sparse, and the bet usually pays.
When features start being correlated rather than independent, or when too many fire at once, interference becomes a real cost. Anthropic’s toy models work shows that as you crank sparsity down, the network smoothly transitions from “dedicated neurons, one feature each” to “superposition, many features per neuron.” The transition isn’t sharp.
Because individual neurons are polysemantic. A single neuron in a real transformer fires on a mishmash of unrelated features — French text, code comments, math expressions — not because the model is confused but because it’s storing those features along non-axis-aligned directions, and the neuron’s activation is just a projection. Trying to interpret one neuron at a time is reading shadows. The “features” you can recover cleanly are the directions, found via sparse autoencoders or similar techniques.
Yes. Sparse autoencoders trained on activations now routinely recover thousands of monosemantic feature directions from models that look polysemantic at the neuron level. This is the methodological move that made the Scaling Monosemanticity work (Anthropic, 2024) possible.
The interesting open question is whether features remain linear in the deepest models, or whether scale eventually introduces nonlinear feature geometry (curved manifolds, multi-token compositional structure). Templeton et al.’s recent work and the Gemma Scope releases are the right entry points.