Reading Without a Reader:
Large Language Models Collapse the Brain’s Dissociated
Reading and Writing Systems into a Single Entangled Code
Abstract
In the literate human brain, reading and writing doubly dissociate: a ventral decoding route (impaired in pure alexia) and a fronto-parietal encoding route (pure agraphia), sharing a partial orthographic core. A decoder-only large language model (LLM) instead drives both from one autoregressive path optimized on text (a cultural invention, not an evolved instinct). We ask how entangled that mechanism is, comparing an input-side “reading code” with an output-side “writing code” via an index (CKA, Procrustes residual, mutual -NN) calibrated against an independent-init floor and a tied ceiling. On GPT-2, OPT and Pythia (14M–1.4B), untied models hold one coupled but sub-ceiling code (–, far above floor) on a non-monotonic couple-then-differentiate trajectory, drifting 3.2 farther than in every frequency decile. We then report an equally informative negative: the matching behavioural test (that comprehension and production fail together rather than dissociate) cannot be run. For minimal pairs the analogue of pure alexia is empty by theorem, since greedy production implies a vocabulary-wide argmax and so wins the pairwise ranking. Differential-damage indices under ablation are not scale-identified: heavy-tailed damage makes linear standardizations collapse onto their larger term, and the rank transform that fixes this is bounded, so its null saturates. And both scores contain the target’s log-probability, which alone explains most of their variance and manufactures the coupling it seems to reveal. We withdraw the coupling statistic, the cross-level bridge and an encoder/decoder separation measure. In a model reading and writing off one next-token distribution, no pair of output-side scores isolates either ability (entanglement that needs no index to see). By analogy, not homology, this situates LLMs in the space of possible minds.
Keywords: large language models; mechanistic interpretability; input and output embeddings (weight tying); representational similarity (CKA); comprehension/production dissociation; cognitive neuroscience of reading.
1 Introduction
Spoken language is, on the dominant view, an evolved biological instinct: it emerges spontaneously in every typically developing child, in every human culture, without explicit instruction (Pinker, 1994). Reading and writing are not. Written language is a recent cultural invention (the earliest scripts are roughly five millennia old), far too young to have been shaped by natural selection. Literacy is instead grafted onto cortex that evolved for other purposes, a process Dehaene and colleagues call neuronal recycling: the acquisition of reading repurposes a region of the ventral occipitotemporal cortex, the visual word form area (VWFA), that becomes selective for letter strings only after a reader learns to read (Dehaene and Cohen, 2007, 2011). The cultural, acquired character of literacy is visible even in its cognitive byproducts: explicit awareness of speech as a sequence of phonemes does not arise spontaneously but is a consequence of alphabetic literacy (illiterate adults fail the phoneme-manipulation tasks that late-literates pass; Morais et al., 1979), and the cortical reorganization induced by reading acquisition leaves a measurable anatomical signature (Carreiras et al., 2009).
Crucially, in the human brain the cultural skill of literacy is realized not as one faculty but as two. Reading (decoding visual symbols toward meaning) and writing (encoding meaning toward orthographic and motor output) doubly dissociate: a focal lesion can abolish reading while sparing writing (pure alexia without agraphia, the classical disconnection syndrome described by Déjerine (1892)), and another can abolish writing while sparing reading (pure agraphia, implicating a distinct graphemic and fronto-parietal encoding system; Caramazza et al., 1987; Roux et al., 2009). The two systems are partly separable yet share a common orthographic core: the same ventral occipitotemporal and left inferior frontal populations are engaged during both reading and spelling (Purcell et al., 2017), and even decoding alone is itself multi-route (Coltheart et al., 2001). The human reading/writing architecture is thus best described as two dissociable systems over a shared orthographic substrate.
Large language models acquire functional literacy under radically different conditions. A decoder-only transformer has no pre-existing evolved substrate to recycle; it optimizes the cultural artifact of written text directly, using a single general-purpose mechanism (self-attention; Vaswani et al., 2017) whose in-context behavior is carried by identifiable induction circuits (Olsson et al., 2022; Elhage et al., 2021). Most importantly, the architecture collapses reading and writing into one entangled autoregressive pass. There is no anatomical separation between a decoding network and an encoding network; there is one stack of weights that both consumes and produces text. The nearest structural analog to the brain’s read/write split is the pair formed by the input embedding , which maps tokens into the residual stream (the “reading code”), and the output unembedding , which maps the residual stream back to token logits (the “writing code”). In tied models (GPT-2, OPT) these are forced to be identical by construction (Press and Wolf, 2017; Inan et al., 2017; Radford et al., 2019; Zhang et al., 2022); in untied models (the Pythia / GPT-NeoX suite) they are free to be learned separately (Biderman et al., 2023). We stress at the outset that this is an analogy, not a homology: and are input- and output-side token-coding matrices in the same modality and processing stage, not literal reading and writing networks, and the alexia/agraphia dissociation is a motivating contrast rather than a measured correspondence. (We also note that the residual-stream framework of Elhage et al. (2021) uses “write/read” for the opposite direction: components write to and read from the stream; our labels instead track the brain’s input-reading/output-writing convention.) We make no claim about “understanding.”
This framing raises a precise empirical question. If the brain realizes literacy as two dissociable systems over a shared core, and a decoder-only LLM realizes it as one statistical code, then we may read LLMs as occupying a distinct point in the space of possible minds (achieving functional literacy by a route the human brain does not take) rather than as a model of the human reading/writing brain (Sloman, 1984; Shanahan, 2025; Mitchell and Krakauer, 2023). We are deliberate about what is and is not at stake here: because a decoder-only architecture commits to a single forward path, some read/write coupling is expected a priori, so the existence of coupling is not itself a discovery. Our empirical contribution is to quantify how far that coupling sits from a genuine-dissociation floor and a forced-collapse ceiling, and to characterize its training trajectory. The “distinct point in the space of minds” reading is offered as an interpretive lens on those measurements, not as a proven fact. Geometrically, we define an entanglement index that fuses three established representation-similarity measures (linear CKA, Kornblith et al., 2019; orthogonal-Procrustes residual with per-token cosine, Smith et al., 2017; and mutual -NN overlap, Huh et al., 2024) between and , calibrated against an independent-initialization floor, a shuffled-token null, and the tied ceiling.
We also set out to test the matching behavioural prediction, contrasting comprehension (log-probability discrimination) with production (greedy generation) and asking whether the two dissociate as they do in the brain. That test, and two redesigns of it, could not be made valid, and we report why as a result in its own right: comprehension and production, as output-side scores define them, are both functionals of a single next-token distribution, so any two scalar summaries of it share terms, and those shared terms generate the very coupling one is trying to detect. The reading and writing of these models are not merely entangled in the weights; at the output they are not separately addressable. We situate all of this against the wider debate on formal versus functional linguistic competence (Mahowald et al., 2024; Fedorenko et al., 2024) and symbol grounding (Harnad, 1990).
Contributions.
-
•
A consolidating replication. Six of our ten probes reproduce previously established findings (induction heads, Olsson et al., 2022; the literacy data-efficiency gap; form-before-meaning, Mahowald et al., 2024; compression scaling; verbatim memorization, Carlini et al., 2023; and an imitation-vs-innovation gap, Yiu et al., 2024), run uniformly on one Pythia suite on a single GPU. We claim these as confirmation, not discovery; our new contributions are the items below.
-
•
A brain-anchored read/write axis for interpretability. We reframe the input embedding and output unembedding of a decoder-only LLM as a “reading code” and a “writing code” , motivated by the brain’s doubly dissociable reading and writing systems, and introduce a controlled entanglement index with an independent-init floor, a shuffled-token null, and a tied ceiling.
-
•
One coupled, sub-ceiling read/write code in the weights. In untied Pythia models sits far above the independent-init floor but well below the tied ceiling: one coupled-but-sub-ceiling code rather than either two independent codes or one shared code. The trajectory is non-monotonic, a “couple-then-differentiate” program (peak at step 4000), and the differentiation is asymmetric (in pythia-160m moves farther than in every frequency decile), so the writing code, not the reading code, does the moving.
-
•
Three reasons the behavioural test cannot be run, which are themselves a result. We attempted the matching behavioural prediction (that comprehension and production should fail together rather than dissociate) and neither it nor two redesigns could be made valid (Section 4.8). For minimal pairs the model analogue of pure alexia is empty by theorem, because greedy production implies a vocabulary-wide argmax and hence wins the pairwise ranking. Indices of differential damage under ablation are not scale-identified: the damage distributions are heavy-tailed enough that every linear standardization collapses the index onto its larger term, and the rank transform that fixes this exactly is bounded, so its family-wise null saturates. And the two scores share the target’s log-probability, which alone accounts for most of the variance in both and manufactures the coupling it appears to reveal. We report the withdrawn claims explicitly and keep the diagnosis, which is the sharper statement: in a model whose reading and writing are read off one next-token distribution, no pair of output-side scores isolates one ability from the other.
-
•
An honest accounting of nulls. We report prominently that we find no evidence for the “compression decoupled from reasoning” sub-claim (the point estimate runs opposite, Pearson , , , underpowered) and withdraw it, that ablating the induction circuit leaves the orthographic word-form gap intact (fraction remaining ), and that the scale trend in is flat and underpowered. We also withdraw, with their reasons, a behavioural coupling statistic and its sign test, a cross-level geometrybehaviour bridge that was computed against it, and a representational encoder/decoder separation measure.
-
•
A position in the space of possible minds (as a lens). Granting that single-path coupling is architecturally expected, what the quantification adds is its calibration (coupled, but well short of the tied ceiling, and reached by a non-monotonic route) alongside the finding that the distinction resists output-side operationalization at all. Together these invite reading decoder-only LLMs as realizing functional literacy through a single entangled statistical code: a distinct point in the space of minds, offered as an interpretive framing and an analogy, not a homology, to the human reading/writing brain.
Relation to prior work.
Our geometric result is closest to Lopardo et al. (2026), who first reported that untied token embeddings drift toward the output space, with the output-side matrix moving more than the input-side one. We credit that work for the asymmetric-drift trajectory; our Experiment 7 is a controlled, properly-nulled replication of it on the read/write axis (the floor, null, ceiling, and frequency-decile controls are detailed in Section 2.3 and Section 3.2). Our conceptual frame extends Mahowald et al. (2024), who dissociate formal from functional linguistic competence: we add a second, orthogonal read/write axis grounded in the neuropsychology of literacy and carry it down to weight geometry. On the behavioral side, comprehension/production asymmetries in LLMs have been studied directly (Lam et al., 2025; Pickering and Garrod, 2013); we differ by attempting to bridge that behavioral dissociation to the underlying weight geometry, and by reporting that the bridge cannot be built (not because the correlation is weak, but because the behavioural term does not survive scrutiny as a measurement; Section 4.8). Finally, where Shanahan (2025) and Mitchell and Krakauer (2023) argue on conceptual grounds that LLMs are a different kind of system, our contribution is to supply controlled, mechanistic evidence at the level of weights, with its nulls reported plainly, for one specific way in which they differ from the human reading/writing brain.
2 Background and Related Work
Our argument draws together four bodies of work: (i) the cognitive neuroscience of reading and writing, which establishes that human literacy is realized by two partially dissociable cortical systems; (ii) the cultural-artifact view of written language and the resulting human–machine gap in sample efficiency; (iii) the interpretability of transformers and the geometry of their input/output embeddings, which gives a tractable weight-level analog of a “reading code” and a “writing code”; and (iv) the debate over whether large language models (LLMs) are best understood as a distinct kind of mind.
2.1 Reading and writing in the brain
Reading is not an evolved faculty. On the neuronal recycling hypothesis (Dehaene and Cohen, 2007), the cultural invention of writing (too recent for natural selection to have shaped a dedicated organ) colonizes and repurposes phylogenetically older visual and language cortex, inheriting its computational constraints. The empirical anchor is the Visual Word Form Area, a patch of left ventral occipitotemporal cortex that becomes selectively responsive to orthographic strings only in literate readers (Dehaene and Cohen, 2011). Learning to read measurably reorganizes the brain, sharpening visual responses and reshaping functional and even structural connectivity (Carreiras et al., 2009). Reading is thus an acquired overlay on evolved hardware, a point we return to when interpreting the acquisition of orthographic structure in a from-scratch transformer.
Crucially for our thesis, reading (decoding) and writing (encoding) are doubly dissociable. Pure alexia without agraphia (loss of reading with spared writing, attributable to a disconnection of visual input from the word-processing network) dates to Déjerine (1892) and supplies the first half of the dissociation logic. Its mirror image, pure agraphia, isolates a writing deficit with preserved reading; the writing system has its own architecture, including an abstract graphemic buffer that holds letter identity and order during spelling (Caramazza et al., 1987) and a writing-specific frontal locus (Exner’s area) implicated by direct cortical stimulation (Roux et al., 2009). Even decoding alone is not monolithic: dual-route models posit parallel lexical and grapheme–phoneme pathways (Coltheart et al., 2001). At the same time the two are not fully independent. Within-subject imaging and lesion evidence indicate a shared orthographic core, with overlapping left ventral occipitotemporal and inferior frontal representations serving both spelling and reading (Purcell et al., 2017). The human picture is two systems with their own failure modes over a partially shared substrate (precisely the structure we ask whether a decoder-only LLM reproduces or collapses).
2.2 Written language as a cultural artifact, and sample efficiency
Writing is a recent technology: proto-cuneiform accounting marks appear around 3300 BCE (Spar, 2004), making literacy a roughly five-millennium-old invention that each child must be explicitly taught. A striking behavioural consequence is that phonemic awareness does not arise spontaneously but is a consequence of alphabetic literacy (illiterate adults fail phoneme-manipulation tasks that matched late-literate adults pass; Morais et al., 1979). This contrasts sharply with spoken language, acquired universally and without instruction and best characterized as an evolved instinct (Pinker, 1994). Our thesis rests on this asymmetry: an LLM optimizes the cultural artifact, not the evolved faculty.
It does so at a data cost orders of magnitude beyond the human reference. Children are exposed to on the order of – words by adolescence whereas competitive models consume trillions of tokens; the BabyLM challenge frames this gap and proposes developmentally plausible budgets (Warstadt et al., 2023). Chollet (2019) reframes intelligence itself as skill-acquisition efficiency relative to priors and experience, which makes the human–machine data gap a difference in kind rather than degree. We treat any human–machine token comparison as illustrative and order-of-magnitude only: web text is not child-directed speech, and humans receive multimodal, embodied, interactive input.
2.3 Transformers, induction heads, and embedding geometry
The mechanism LLMs bring to this artifact is general-purpose self-attention (Vaswani et al., 2017), a modality-agnostic routing operation rather than a language-specific faculty. Mechanistic interpretability has made parts of it legible: the residual-stream formalism of Elhage et al. (2021) casts each component as reading from and writing to a shared activation stream and identifies how attention heads compose into induction heads, circuits implementing prefix-matching copy that underwrite much of emergent in-context learning (Olsson et al., 2022). Complementary analyses show heads specialize into interpretable roles, with a small number doing most of the work (Voita et al., 2019).
The geometric heart of our contribution concerns the input embedding , which maps tokens into the stream, and the output unembedding , which maps the stream out to next-token logits. Our “reading/writing” labels track the brain’s input-reading, output-writing convention; note this is the reverse of the residual-stream usage of Elhage et al. (2021), in which components write to and read from the stream. We mean input- versus output-side only, and since and are indexed by BPE tokens rather than graphemes, we reserve “orthographic” for the sub-token word-form analysis. Weight tying sets , forcing the two codes to be identical (Press and Wolf, 2017; Inan et al., 2017); it is standard in the GPT-2 family and OPT, and is the architectural condition under which reading and writing are entangled by construction. The Pythia suite leaves the embeddings untied, learning and separately and exposing aligned checkpoints that let us watch the two codes differentiate over training (Biderman et al., 2023).
The closest prior work to the geometric half of our study is Lopardo et al. (2026), which measures alignment between input and output embeddings and reports that output gradients dominate early training, biasing token embeddings toward the output space and producing an asymmetric drift. We credit that work for first reporting this trajectory; our Experiment 7 is a controlled, properly nulled replication on the read/write axis, adding an independent-init floor, a shuffled-token null, a tied ceiling, and a frequency-decile decomposition that addresses the gradient-exposure confound. To compare and as representational spaces we use established similarity measures (linear CKA, Kornblith et al., 2019; orthogonal-Procrustes residual with per-token cosine, Smith et al., 2017; and mutual -nearest-neighbor overlap, Huh et al., 2024), all situated within the representational-similarity tradition (Kriegeskorte et al., 2008). That and should be coupled but not identical in untied models is consistent with evidence that the output embedding is functionally specialized rather than a transpose of the input embedding, encoding token-probability and frequency structure (Cho et al., 2024). Because performance improves predictably with scale and data (Kaplan et al., 2020), we interpret any geometry along the scale axis against that backdrop and report its trend honestly as flat and underpowered.
2.4 Are LLMs a different kind of mind?
A final strand asks what such systems are. Balanced surveys call for an extended science of intelligence that catalogs distinct modes of understanding rather than forcing a binary verdict (Mitchell and Krakauer, 2023). A long-running critique holds that form learned from distribution cannot by itself yield grounded meaning: the symbol-grounding regress (Harnad, 1990) and the “stochastic parrot” framing (Bender et al., 2021). The most directly operationalizable formulation for our purposes is the dissociation of formal from functional linguistic competence: LLMs are strong on formal patterns yet patchy on functional, world-directed use, paralleling a neural dissociation between the brain’s language network and its reasoning systems (Mahowald et al., 2024; Fedorenko et al., 2024). We extend this form/function axis with a second, brain-anchored read/write axis. Finally, psycholinguistics treats comprehension and production as distinct-but-coupled systems linked by forward models (Pickering and Garrod, 2013), and recent work leverages human production–interpretation asymmetries to probe LLM cognitive plausibility (Lam et al., 2025). We attempted the corresponding behavioural test and report in Section 4.8 why it cannot be constructed at the output of a decoder-only model. The positioning is therefore conservative: we ask not whether LLMs understand, but whether the way they realize functional literacy (by collapsing the brain’s dissociated read/write architecture into one entangled statistical code) places them at a distinct point in the space of possible minds.
3 Models and Methods
Our experiments exploit a single architectural fact. A decoder-only transformer maps tokens to vectors through an input embedding matrix and maps the final residual state back to a distribution over tokens through an output unembedding (“readout”) matrix , where is the vocabulary size and the model width (Vaswani et al., 2017; Elhage et al., 2021). We treat as the model’s reading code (the input-side token code that turns a token into an internal vector) and as its writing code (the output-side token code that turns an internal vector into a token). This is an analogy, not a homological claim: and are input- and output-side codings of the same modality at the same processing stage (indexed by BPE tokens, not graphemes, so we do not call them “orthographic”) and not the brain’s spatially distinct reading (ventral occipitotemporal) and writing (fronto-parietal graphemic) networks. (Our input=reading/output=writing labelling is the reverse of the residual-stream “writes-to/reads-from” convention of Elhage et al. (2021); we intend input- vs output-side only.) The brain’s reading and writing systems doubly dissociate (pure alexia spares writing and pure agraphia spares reading; Déjerine, 1892; Caramazza et al., 1987; Roux et al., 2009) while sharing a partial orthographic core (Purcell et al., 2017). We ask, by tight analogy only, whether the LLM’s input/output codes are one entangled system or two dissociable ones, and we never claim the model “reads” or “understands.”
3.1 Models
We use two model families that differ in exactly the property of interest: whether the reading and writing codes are forced to be identical.
Tied (entangled by construction).
The GPT-2 family (distilgpt2, gpt2, gpt2-medium, gpt2-large; Radford et al., 2019) and the two OPT models we use (facebook/opt-125m, facebook/opt-350m; Zhang et al., 2022) tie their embeddings, imposing (Press and Wolf, 2017; Inan et al., 2017). For these models the reading and writing codes are the same matrix by construction; they serve as a ceiling reference (see §3.2), not as evidence about learning.
Untied (codes learned separately).
The Pythia / GPT-NeoX suite (EleutherAI/pythia-{14m, 70m, 160m, 410m, 1b, 1.4b}; Biderman et al., 2023) keeps and as independent parameter blocks, so any geometric similarity between them is learned rather than imposed. Pythia additionally releases checkpoints per model, trained on the Pile in a fixed data order, with tokens per step; this lets us measure how the codes diverge over training, not just at convergence. All models are run locally on a single RTX 4070.
3.2 Reading and writing codes, and the entanglement index
To quantify how similar the reading code and the writing code are, we compare the two matrices with three complementary representational-similarity measures, each invariant to a different nuisance transformation, following standard practice in the similarity-of-representations literature (Kriegeskorte et al., 2008).
It helps to picture each matrix as a map that places every vocabulary token at a point in space, so that and are two maps of the same set of tokens. The three measures then ask three different questions about whether the maps agree. The first asks whether they agree on the overall pattern of which tokens sit near which, ignoring any difference in overall size or orientation. The second asks whether one map can be rotated bodily onto the other, and how much disagreement is left over after the best possible rotation. The third ignores global shape entirely and asks a local question: for each token individually, are its nearest neighbours the same on both maps? A pair of codes could score high on one and low on another (two maps can share a global shape while disagreeing token by token), which is why we report all three rather than any single one.
-
1.
Linear CKA (centered kernel alignment) between and , a scale- and rotation-invariant global similarity of the two row-space geometries (Kornblith et al., 2019); this is our primary metric.
-
2.
Orthogonal Procrustes residual plus mean per-token cosine: we solve for the orthogonal map minimizing and report the residual and the resulting per-token cosine similarity, testing whether the two codes are the same space up to a rotation/reflection (Smith et al., 2017).
-
3.
Mutual -nearest-neighbor overlap, the fraction of each token’s nearest neighbors in that are also neighbors in , a local/neighborhood-structure measure (Huh et al., 2024).
We fuse these into a single composite entanglement index . Writing for linear CKA, for the Procrustes residual, for the mean per-token cosine, and for mean mutual-NN overlap (), and using the tied-model ceiling and the independent-init floor (below) to normalize each component to via (with the residual oriented as ), we define as the unweighted mean of the four normalized components,
so at the tied ceiling and at the independent-init floor. The two anchors are what make the number mean anything, in the way a temperature scale needs two fixed points before a reading is informative: a raw similarity value cannot be called high or low until one knows what fully unrelated codes score (the floor) and what deliberately identical codes score (the ceiling). We always report the raw components alongside so the composite never hides a divergent component, and we verified the trends are not an artifact of the equal-weighting choice. To make interpretable we anchor it against three controls:
-
•
a tied ceiling: for tied models , so exactly (a sanity check on the pipeline, not an empirical finding);
-
•
an independent-initialization floor: between two freshly, independently initialized matrices, the value expected with no shared learning (empirically near zero on every component);
-
•
a shuffled-token null: recomputed after randomly permuting the token-to-row correspondence between and , which destroys any token-aligned structure while preserving each matrix’s marginal geometry.
For the training-trajectory analysis (§3.3, Exp. 7) we additionally decompose and the per-matrix drift by token frequency decile, to separate genuine read/write differentiation from the confound that frequent tokens simply receive more gradient updates on the output side. All headline similarity values are reported with bootstrap confidence intervals (2,000 resamples, percentile method); the independent-init floor is computed per model from two Gaussian matrices matched to each code’s scale under a fixed seed, and mutual-NN uses with query subsampling for tractability on the full vocabulary. We note one interpretive caveat inherited from this design: and are compared in a common token basis, so the metrics are sensitive to the byte-pair tokenization shared by the two codes rather than to any sub-symbolic orthography.
3.3 The ten probes
The ten experiments trace a single arc: the first six (Exps. 1–6) consolidate established results, Exp. 7 introduces the read/write analysis in the weights, and Exps. 8–10 are the three attempts to test it at the output. Self-attention is the general mechanism (Exp. 1) that acquires written language as a cultural overlay (Exp. 2) at a large data cost (Exp. 3), mastering form before meaning (Exp. 4) by fitting and storing text statistics (Exps. 5–6); on top of this we ask whether the model carries a single entangled read/write code (Exp. 7) and whether it behaves coupled rather than doubly dissociated (Exp. 8).
Exp. 1: Induction heads and causal ablation.
We localize prefix-matching induction heads (Olsson et al., 2022; Elhage et al., 2021) in all models via an induction score, then causally ablate the top-scoring head and measure the change in in-context learning (ICL) on a synthetic copy-completion task, benchmarked against a null distribution of random-head ablations (so the smallest attainable permutation is ) and against MLP-block ablations (Voita et al., 2019). The claim is scoped to “induction circuits are causally implicated in ICL,” not to attention dominating MLPs.
Exp. 2: Orthographic word-form acquisition.
We track the orthographic word-form gap (bits-per-byte on illegal minus real letter strings) across Pythia checkpoints, test its temporal alignment with the induction phase change, run a causal test (does ablating the top induction head remove the gap?), and fit a layerwise linear probe for the real/legal/illegal three-way distinction against a permutation null, with BPE-confound controls (token-count and subword-unigram matching). Because a from-scratch transformer has no pre-existing evolved substrate, the analogy to neuronal recycling and the VWFA (Dehaene and Cohen, 2007, 2011; Morais et al., 1979) is to acquisition timing only.
Exp. 3: Tokens-to-competence.
Using the released Pythia checkpoints (tokens step ), we plot loss and BLiMP grammaticality (a benchmark of minimal sentence pairs differing in one grammatical feature, scored by whether the model assigns the grammatical member higher probability) against tokens seen and compare the data budget to the 100M-word “by age 13” human reference (Warstadt et al., 2023; Kaplan et al., 2020). This comparison is illustrative and order-of-magnitude only: humans receive multimodal, embodied, interactive input, and web text is not child-directed speech, so this is not a claim of cognitive equivalence (Chollet, 2019).
Exp. 4: Formal vs. functional competence.
We contrast formal linguistic competence (BLiMP, with a per-phenomenon breakdown) against functional competence (PIQA, a templated situation-tracking probe, and absurdity judgments), with difficulty/length-matched controls and a surface baseline built from positive pointwise mutual information (PPMI, a co-occurrence statistic that captures how often two words appear together relative to chance, and therefore what can be predicted from word co-occurrence alone) under two normalization schemes, operationalizing the form/function dissociation of Mahowald et al. (2024); Fedorenko et al. (2024).
Exp. 5: Compression and the statistical artifact.
We fit the bits-per-byte () scaling law in non-embedding parameters, build a BPE Kneser–Ney -gram comparator over the same tokenization, measure surface-perturbation fragility, and test whether compression decouples from functional competence across the six Pythia sizes. The decoupling sub-claim is reported as a planned test that may fail.
Exp. 6: Repository vs. grounding ceiling.
We measure extractable verbatim memorization vs. scale (Carlini et al., 2023), an imitation-vs-innovation gap on three-choice tool items against a rarity baseline (Yiu et al., 2024), and a bounded symbol-grounding demonstration comparing hidden-state geometry to perceptual norms against a PPMI baseline (Harnad, 1990). The three parts measure different constructs; “repository, not abstraction” is a hypothesis, not proven by any single probe.
Exp. 7: The entanglement index across architecture, scale, and training.
Using the machinery of §3.2, we compare and (i) across tied vs. untied architectures, (ii) across the Pythia scale ladder, and (iii) along the Pythia-160m (and 410m, 14m) training trajectory. The trajectory analysis is a controlled, properly-nulled replication of the asymmetric output-side drift first reported by Lopardo et al. (2026), using the independent-init floor, shuffled-token null, and frequency-decile controls defined above (Press and Wolf, 2017; Inan et al., 2017; Smith et al., 2017). Summary values appear in Table B9.
Exps. 8–10: the behavioural and causal tests, and why they are withdrawn.
We attempted three successive operationalizations of the read/write distinction at the output, and report their designs here because the reasons they fail are results in their own right (Section 4.8).
Exp. 8, a contingency table. Comprehension is log-probability discrimination (does the model prefer the correct written form over a competitor?) and production is greedy generation (does the model write the form?). These are mechanistically distinct measurements, so we assembled the table whose off-diagonal “dissociation mass” would be the analogue of the brain’s alexia/agraphia double dissociation (Coltheart et al., 2001; Pickering and Garrod, 2013; Lam et al., 2025; Mahowald et al., 2024), summarized the coupling by (the Matthews correlation between comprehension- and production-correctness), and tested a geometrybehaviour bridge by correlating with . Production’s success set is a subset of comprehension’s, however, and for single-token pairs that nesting is a theorem; in our stimulus set the two marginals were additionally measured on disjoint item subsets, making the off-diagonal an exact function of the margins. The table, , the sign test over models, and the bridge computed against are all withdrawn.
Exp. 9, an architecture contrast. To ask whether coupling is specific to the decoder-only forward path we ran a matched probe across decoder-only (Pythia), encoder–decoder (T5 v1.1 (Raffel et al., 2020), plus the Pile-trained Pile-T5 as a data-matched control) and masked-encoder (BERT (Devlin et al., 2019); RoBERTa (Liu et al., 2019)) models, and measured the representational separation of the encoder and decoder stacks as between their per-layer hidden states on the same inputs. The behavioural half inherits Exp. 8’s defect. The representational half we withdraw for three independent reasons: the contrast is fixed by architecture class before any data is collected, the measure is unchanged by destroying linguistic structure, and its value is unstable in the stimulus set. We re-measured it on hundreds of distinct corpus sentences with a random-init floor, a shuffled-token null, and a within-stack depth-drift reference; details and numbers are in Section 4.8.
Exp. 10, a lesion contrast. Because baseline nesting constrains the two success sets but not their derivatives, we ablated one attention head, attention sublayer or MLP sublayer at a time and asked whether comprehension and production are damaged differentially, scoring both on graded capabilities (a comprehension margin, and a production capability that is zero exactly when greedy decoding would write the form) rather than thresholded accuracies. Severity is controlled with the mean Kullback–Leibler divergence of the ablated next-token distribution from its unablated value at the target’s positions, which is neutral between reading and writing. Inference is a paired bootstrap over items with a step-down max-statistic permutation test over sites. The per-site index is not scale-identified and the pooled coupling statistic is explained by a term the two scores share; both are withdrawn, with the diagnosis reported in Section 4.8. Throughout, we read these attempts through the “space of possible minds” lens (Sloman, 1984; Shanahan, 2025; Mitchell and Krakauer, 2023).
3.4 Reproducibility and honesty
All experiments run in fp32 by default: we observed that fp16 produces NaNs on the GPT-NeoX/Pythia models, and bf16 requires torch . We use fixed random seeds and frozen stimulus sets (including the author-made situation-tracking and tool-innovation items) so that every reported number is reproducible on a single RTX 4070. All models and benchmarks we use are publicly available (the GPT-2, OPT, and Pythia/GPT-NeoX checkpoints, and the Pile, BLiMP, and PIQA), so every result can be regenerated from public sources following the procedures described above.
Statistical approach.
Our inferences are deliberately modest. With six model sizes and three model families (sharing tokenizers within family), the cross-model and scale analyses rest on small, non-independent samples; we treat all fits as descriptive, apply no multiple-comparison correction, and designate no test as confirmatory at a corrected . Where a sign or permutation test is reported (e.g. the behavioral coupling), we state it explicitly; elsewhere we attach bootstrap confidence intervals and label small- relationships “suggestive.” Model family is a confound for every cross-model comparison, which we flag rather than correct away. Finally, because a decoder-only architecture commits to a single forward path, some read/write coupling is expected a priori (our claims concern its magnitude and trajectory, not its existence).
4 Results
We organize the results along the argument arc. Self-attention is a modality-general routing primitive, and induction circuits within it are causally implicated in in-context learning (Section 4.1); on written language the model acquires an orthographic word-form code as a cultural overlay (Section 4.2) at a data cost orders of magnitude beyond the human budget (Section 4.3); it masters form before meaning (Section 4.4) by compressing and storing text statistics (Sections 4.5–4.6); and it does so with a single, entangled read/write code (Section 4.7). Because a decoder-only model has a single forward path, some read/write coupling is expected a priori; our contribution is therefore not the existence of coupling but its quantification (its distance from an independent-init floor and a tied ceiling, and the shape of its training trajectory).
We then report a second, negative contribution. We attempted the matching behavioural test, that comprehension and production should fail together rather than dissociate, and neither it nor two successive redesigns could be made valid (Section 4.8). The obstacles are structural rather than incidental: each candidate measurement reduced to an algebraic identity between the two scores or to a quantity with no identified scale. We state the three results explicitly, because in a model whose reading and writing are read off a single next-token distribution, the fact that no pair of output-side scores isolates one ability from the other is itself informative about entanglement. All confidence intervals are bootstrap 95% intervals unless noted; the compression/reasoning decoupling and several secondary trends are reported as plain nulls.
4.1 Induction circuits are causally implicated in in-context learning
In-context copying is supported by induction heads in every model we examine. Prefix-matching induction scores are high across all eight small open models, ranging from (pythia-70m) to (pythia-160m), far above the chance rate expected for a random head (Table B1; Appendix Figure A1). To test whether these heads are causally implicated in in-context learning (ICL) rather than merely correlated with it, we zero-ablate the single top induction head and measure the change in a synthetic copy-completion ICL score relative to a 24-random-head null. The effect is significant for opt-125m (), pythia-160m (), and pythia-410m () (all permutation , which is the resolution floor of the 24-head null, not a precise estimate); it is weak or non-significant in the GPT-2 family. Crucially, we also ablate the MLP block at the same layer: its effect on ICL is comparable to or larger than the head’s (e.g. pythia-70m for the MLP vs for the head). We therefore scope the claim narrowly: induction circuits are causally implicated in ICL, not that attention dominates the MLP. Across models the relationship between maximum induction score and in-context gain is positive but does not reach significance (Spearman , , ; Appendix Figure A2). We report this small- cross-model trend as suggestive.
4.2 Written language as an acquired cultural overlay
A from-scratch transformer has no evolved substrate for orthography; any sensitivity to legal letter sequences must be acquired. Using the aligned Pythia checkpoints, we measure an orthographic word-form gap, defined as over the continuation tokens of a word form (carrier masked), ordered real legal-pseudoword illegal at every checkpoint. The raw gap is dominated by a tokenization confound (illegal consonant strings fragment into many more BPE sub-tokens than real words), so for pythia-160m it is already large at random initialization ( bits/byte) and, if anything, shrinks as the model learns to compress all three classes (to at the final checkpoint). The genuine, acquired signal appears only once this confound is removed: the token-count–residualized gap grows over training from to bits/byte, and a token-count–matched subset grows from to (Table B2; Figure 1). Orthographic word-form selectivity is therefore acquired over training, as the neuronal-recycling framing predicts for a cultural skill, though we cannot fully exclude residual subword-frequency structure. We do not claim a clean co-onset with the induction phase change: the raw-gap onset is itself a tokenization artifact (its maximum-slope step is step 1), and, as we show next, the induction circuit is causally unrelated to the gap. We thus report the controlled word-form gap as a gradually acquired code rather than one temporally locked to circuit emergence.
We confirm the same orthographic distinction directly. A linear probe separates real/legal/illegal word forms with three-class accuracy at pythia-160m’s best layer (layer 8), well above both chance () and a permutation null (), and a token-count–matched real-vs-scrambled control () rules out token-count leakage (Table B3). Decodability peaks in mid-to-late layers, the “letterbox” depth profile (Figure 2).
Critically, this orthographic code is not carried by the induction circuit. Ablating the top induction head (L4H6 in pythia-160m) leaves the word-form gap essentially intact: it moves from a clean to under ablation, a fraction remaining of (a matched random-head control shifts the gap by only ; the gap does not collapse). This is an honest negative: the mechanism that supports in-context copying (Section 4.1) is dissociable from the one that encodes orthographic legality. We emphasize that the analogy to the visual word form area and neuronal recycling (Dehaene and Cohen, 2007, 2011; Morais et al., 1979) is to acquisition timing only; a randomly initialized transformer has no pre-existing evolved cortex to recycle.


4.3 The data cost of competence
Both loss and grammaticality improve smoothly as a function of tokens seen (Table LABEL:tab:exp3_data_dynamics). For pythia-160m, perplexity on held-out text falls from at step 1 to at the final checkpoint, while BLiMP grammaticality rises from chance () to ; the larger pythia-410m reaches . An across-size loss-vs- power law at the final checkpoint has exponent (with the within-run fits flagged as autocorrelated, so their intervals understate uncertainty). The machine reaches this competence only after tokens ( word-equivalents assuming a nominal tokens/word), three to four orders of magnitude beyond the 100M-word budget a child has by age 13 (band 50–200M words, i.e. 65–260M tokens; Warstadt et al., 2023). We treat the human comparison as illustrative and order-of-magnitude only: children receive multimodal, embodied, interactive, child-directed input, whereas the Pile is adult written text, so this is not a confirmed cognitive equivalence (Chollet, 2019).
4.4 Form before meaning
Across the Pythia scale ladder, formal competence (BLiMP grammaticality) exceeds and saturates ahead of functional competence (Table B5). At the top of our ladder, pythia-1.4b reaches BLiMP [, ] while its functional composite (the mean of the PIQA physical-commonsense, situation-tracking, and absurdity-detection accuracies) reaches , a formfunction gap of . Form rises early and plateaus; function lags and remains below it. These results hold under a difficulty/length-matched control and a PPMI surface baseline, following the formal/functional distinction of Mahowald et al. (2024); Fedorenko et al. (2024). We are candid that the gap is modest at 1.4b, that the situation-tracking items are author-made and templated (frozen across runs), and that there is no human ceiling; the claim is scoped to these probes, not to meaning in general (Harnad, 1990).
4.5 Optimizing a statistical artifact: compression and its limits
Compression of held-out text improves as a clean power law in non-embedding parameters: bits/byte falls from (pythia-14m) to (pythia-1.4b) with exponent [, ] and (no-offset fit; Table B6), consistent with established scaling laws (Kaplan et al., 2020).
Compared against a Kneser-Ney BPE -gram on the same tokenization, the interpretable signal is the scaling of the neural-gram gap, not the raw overlap. The neural/-gram perplexity ratio falls from (pythia-14m) to (pythia-1.4b): the neural model increasingly outpaces the -gram with scale (Table B7). Top-1 next-token agreement between the -gram and pythia-160m is only , with a symmetric KL of nats, so we frame the result around the scaling of the gap rather than claiming the -gram recovers most of the model.
The neural model is also consistently more fragile to word-order corruption than the -gram: the model-gram excess NLL under a length-5 window shuffle grows with scale, from nats (pythia-14m) to nats (pythia-1.4b) (Table LABEL:tab:exp5_perturbation). Zipf reproduction is demoted to a descriptive observation, since its own shuffled control shows it to be near-trivial.
Null: no evidence that compression decouples from reasoning.
We tested whether better compression and better functional competence dissociate across scale. We find no evidence for decoupling: across the six Pythia sizes the point estimate runs in the opposite direction (bits/byte vs functional composite Pearson , ; Spearman , ; ; Figure 3), i.e. if anything the two improve together. With this is underpowered and cannot establish co-variation either; the only safe conclusion is that the “compression decoupled from reasoning” hypothesis is not supported, and we withdraw it as a positive claim.

4.6 A knowledge repository against a grounding ceiling
Three separate probes characterize the model as a text repository operating against a distributional ceiling; we report them separately rather than merged (Table B8). First, extractable verbatim memorization of Pile text grows log-linearly with scale, with slope [, ] in non-embedding parameters (), against a non-member negative control consistent with prior memorization laws (Carlini et al., 2023). Second, on three-choice tool-use items, the imitationinnovation gap reaches while a PPMI rarity baseline reproduces a gap of , so the innovation deficit is not explained by option rarity (Yiu et al., 2024). Third, on a bounded symbol-grounding demonstration, hidden-state geometry tracks perceptual size norms with up to (rising with scale), beating a PPMI baseline ( max ), whereas for lightness the model does not beat the PPMI baseline (Figure 4; Harnad, 1990). The three probes measure different constructs, and “repository, not abstraction” is a hypothesis, not proven by any single one.

4.7 A single, entangled read/write code
We now turn to the central claim. We treat the input embedding as a “reading code” (the input-side token code) and the output unembedding as a “writing code” (the output-side token code); we use “reading/ writing” for the input/output side only, and reserve “orthographic” for the sub-token word-form analysis of Section 4.2, since and are indexed by BPE tokens, not graphemes. We quantify their entanglement with a composite index fusing linear CKA (Kornblith et al., 2019), an orthogonal-Procrustes residual with per-token cosine (Smith et al., 2017), and mutual -NN overlap (Huh et al., 2024), against an independent-init floor, a shuffled-token null, and a tied ceiling (Kriegeskorte et al., 2008).
The tied models (gpt2, gpt2-medium, opt-125m, opt-350m) return by construction (; Press and Wolf, 2017; Inan et al., 2017); this is a pipeline sanity check, not a finding. The untied Pythia models are the substance: ranges from to , with every component measure far above its independent-init floor (Table B9; Figure 5). In plain terms, the two codes sit roughly a third of the way from “completely unrelated” to “forced to be identical”. The reading and writing codes are thus coupled but sub-ceiling: one entangled code, not the two dissociated systems of the brain, and also not a trivial collapse onto a single shared matrix. The coupling is genuine per-token structure rather than a global-subspace artifact: the observed per-token cosine and mutual-kNN overlap greatly exceed both a shuffled-token null and the independent-init floor in every untied model (Table B10). The sub-ceiling differentiation is moreover consistent with evidence that the output embedding is functionally specialized rather than a mirror of the input embedding (Cho et al., 2024).
The training trajectory is non-monotonic and informative (Figure 6). For pythia-160m, / CKA starts at (initialization, effectively independent), rises to by step 1000, peaks at at step 4000, then differentiates back down to at step 64000 and at the final step 143000: the codes couple early and then partly pull apart. This differentiation is asymmetric: at step 143000 the writing code has drifted as far as the reading code from initialization (drift vs drift ). This asymmetric-drift trajectory was first reported by Lopardo et al. (2026); our contribution here is a controlled replication on the read/write axis, with an independent-init floor, a shuffled-token null, and a frequency-decile decomposition addressing the gradient-exposure confound. That confound (the worry that simply “moves more” because the output matrix receives a softmax gradient on every step while input rows update only for tokens seen) is ruled out by decomposing the drift by token-frequency decile: the writing code drifts roughly as far as the reading code in every decile, from the rarest to the most frequent tokens (Table B11), so the asymmetry is not carried by frequent tokens alone.
Null: no entanglement trend with scale.
Across the six untied sizes, does not vary systematically with model size: Spearman (permutation , ; Appendix Figure A3). We report this as a flat, underpowered null rather than a trend.


4.8 The read/write distinction resists operationalization at the output
Having found one coupled read/write code in the weights, we set out to test the matching behavioural prediction: that comprehension and production should fail together rather than dissociate. We report here that this test, and two successive redesigns of it, could not be made valid. The obstacles are not implementation defects but properties of what the measurements are: each attempt reduced to an algebraic identity between the two scores, or to a quantity with no identified scale. We give the three results because they are, we think, the substantive finding of this half of the paper. In a model whose reading and writing are read off one next-token distribution, no pair of output-side scores isolates one ability from the other, and that is a form of entanglement no index had to be constructed to see.
Result 1: for minimal pairs the LLM-alexia cell cannot fire.
The natural analogue of the alexia/agraphia contrast is a comprehension production contingency table, whose “produces-but-does-not-comprehend” cell would be the model’s pure alexia. That cell is empty by logic. Production, operationalized as greedy generation of the target form , is a -way task; comprehension, operationalized as ranking above one competitor , is -way. If greedy decoding emits , then ’s token was the argmax over the entire vocabulary, so necessarily outranks : production’s success set is a subset of comprehension’s. For a one-token-versus- one-token pair this is a theorem, not a tendency. The cell can fire only on pairs that have equal token counts and at least two tokens, but equal token count is exactly what makes a per-token mean log-probability comparison unbiased, so the two requirements compete. On a purpose-built -item pool, one item satisfied both under the GPT-2 and Pythia tokenizers simultaneously; most pairs were single-token, the remainder length-mismatched. Adding tokenizers makes it worse, and the only remaining route to a non-empty cell is to let beat on the mean through length normalization, which would manufacture the result from a scoring artifact.
In our original stimulus set the same degeneracy took a second and more concrete form. Comprehension saturated at on one item family while production floored at on the other, so the two marginals were measured on disjoint item subsets and the off-diagonal mass reduced exactly to the difference of the two marginals. We verified that identity to numerical precision on every model checkpoint row we had computed. A contingency table whose cells are exact functions of its margins cannot test independence, so the coupling statistic, its sign test, and the geometrybehaviour bridge built on top of it are all withdrawn.
Result 2: differential-damage indices are not scale-identified.
Baseline nesting constrains the two success sets but not their derivatives, so we turned to the model analogue of a lesion study: ablate one component at a time and ask whether comprehension and production are damaged differentially. This requires comparing two damages on a common scale, and no such scale exists here. The damage distributions are extremely heavy-tailed: in GPT-2 the ratio of the largest to the median production damage exceeds two orders of magnitude. Any linear standardization therefore leaves the two variances mismatched (even a robust median-absolute-deviation scaling left a forty-fold ratio) and a difference of two mismatched-variance terms is dominated by the larger. We built three such indices in turn, standardizing by baseline across-item variability, by a robust scale estimated among severity-matched peers, and by the robust across-site spread. Each one correlated between and with the total size of the lesion, and between and with its own production term: all three were measuring how much the ablation broke the model, and each flagged the largest lesions in every architecture as “selective”.
A rank transform removes the confound exactly rather than approximately. Ranking each damage across sites makes both marginals uniform, hence their variances equal, and since , the selectivity contrast becomes exactly uncorrelated with total damage; measured, its correlation with lesion size is and with an independent severity measure , with no nuisance regression at all. But the rank difference is bounded in , and a family-wise permutation null for a bounded statistic saturates at its own bound: over several hundred sites the null maximum reaches against a possible , so no site can exceed it. Relaxing the null does not help, because a per-site bootstrap over items tests whether a site’s damage is non-zero rather than whether the site is unusual among its peers, and it flags ordinary sites. The per-site test is thus either valid without power or powered without validity.
Result 3: the two scores share a term, which manufactures coupling.
A pooled statistic escapes the multiplicity problem, so we asked instead whether comprehension damage and production damage co-vary across sites at matched lesion severity, using the divergence of the next-token distribution from its unablated value as a read/write-neutral severity covariate. They do, in all three models tested, with rank correlations of to against a stratified permutation null. The result is nonetheless void, for a reason internal to the two measures. Writing , and for the mean log-probabilities of the target, the competitor and the argmax token, the comprehension margin is and the production capability is : both contain . A site that moves the target’s own log-probability therefore moves both damages in the same direction with no differential effect whatever. That is not a small correction: the movement in alone accounts for of the variance in production damage and in comprehension damage. Removing the shared term algebraically ( and are recoverable from what is logged, and share nothing) leaves a coupling of , marginal against its permutation null, with two thirds of the association gone; removing it by regression leaves nothing distinguishable from the null at all. The measured coupling is, to a first approximation, a restatement of the fact that both scores are anchored to the same log-probability.
An encoder/decoder proxy does not substitute.
One might avoid output-side scores altogether by asking whether an architecture with physically distinct stacks separates the two pathways representationally, and comparing encoder and decoder hidden states on the same text is the obvious measure. It does not work either, for three reasons we can state briefly. First, the comparison is decided before any data is collected: a model with two disjoint stacks must yield a large difference and a single-stack model must yield approximately none, so the architecture class fixes the answer. Second, the measure is insensitive to language. Shuffling the tokens within each sentence, which destroys every syntactic and semantic relation while preserving the token inventory, changes the encoder–decoder separation by on average across the four encoder–decoders we tested, and in two of them the shuffled text separates more. Whatever the number reflects, it is present for arbitrary token sequences (consistent with its being the difference between a bidirectional stack and a causal stack reading the same tokens shifted by one position). Third, the estimate is unstable in the stimulus set: measured on hundreds of distinct corpus sentences rather than a handful of repeated frames, two of the four models fall by roughly a third, the ordering with model size stops being monotonic, and an apparent robustness across training corpora disappears. We therefore make no representational-separation claim for encoder–decoders.
What the three results amount to.
None of this shows that reading and writing are indistinguishable inside these models; it shows that they are not separable by the output-side operations available to us, and it identifies why. Comprehension and production scores are both functionals of one next-token distribution, so any two scalar summaries of it share terms; the shared terms then generate the very coupling one is trying to detect, and removing them removes the signal along with the artifact. This is a sharper statement than the behavioural coupling we originally reported, and it is the honest one: we can measure the entanglement of the reading and writing codes in the weights, where an independent-init floor and a tied ceiling give the index a scale, but we cannot convert that into a behavioural double dissociation test, and we now understand that as a consequence of the single-forward-path design rather than a limitation of our stimuli. Whether a richer probe (multi-token generation scored against a reference, or a model organism trained with deliberately separated read and write pathways) can recover a genuine dissociation test is open, and we regard it as the natural next experiment rather than a gap in this one.
5 Discussion
5.1 What the measurements support, and what they cannot
The positive result of this paper is a calibrated measurement at one level. In the untied Pythia models the input embedding (the “reading code”) and the output unembedding (the “writing code”) are coupled but sub-ceiling: the entanglement index settles roughly a third of the way up its range, far above the independent-initialization floor yet well below the tied ceiling that GPT-2 and OPT realize by construction. The number is interpretable because it has both a floor and a ceiling; the trajectory adds that the coupling is not monotone but a couple-then-differentiate program, and that the differentiation is carried almost entirely by .
We set out to pair this with a behavioural double dissociation test and could not build one that was valid, across three designs (Section 4.8). It would be easy to present that as a limitation and move on. We think it is more than that, because the three failures share a cause. Comprehension and production, as anything at the output can define them, are both functionals of one next-token distribution; any two scalar summaries of that distribution therefore share terms, and the shared terms generate exactly the coupling one is trying to detect. In the contingency design the shared structure is set-theoretic: greedy production implies a vocabulary-wide argmax, so production’s successes are a subset of comprehension’s, and for single-token pairs the alexia cell is empty by theorem rather than by measurement. In the lesion design it is algebraic: the comprehension margin and the production capability both contain the target’s own log-probability, whose movement alone accounts for most of the variance in each, so their apparent co-variation is largely a restatement of that fact. Between these sits a purely statistical obstacle: with damage distributions as heavy-tailed as these, no linear standardization puts the two on a common scale, and the rank transform that does is bounded, which costs the family-wise test its power.
The upshot is a sharper claim than the behavioural coupling we originally reported, and it does not depend on any index we constructed. Reading and writing in a decoder-only model are not merely coupled in the weights; at the output they are not separately addressable at all. That is what one should expect of a single forward path, and it is worth saying plainly rather than leaving as an unexamined architectural assumption. It is also why we are careful, below, not to read the weight-level coupling as though a behavioural dissociation had been ruled out empirically. It has not been tested; we argue it cannot be, by these means.
5.2 Reading without a reader
This matters because it inverts the organization observed in the literate human brain. There, reading and writing doubly dissociate: pure alexia spares writing while abolishing reading (Déjerine, 1892), and pure agraphia spares reading while abolishing writing (Caramazza et al., 1987; Roux et al., 2009), with only a partial shared orthographic core (Purcell et al., 2017; Coltheart et al., 2001). This / contrast is an analogy, not a homology (see Section 6): alexia and agraphia motivate the question but are not measured here, and we intend no claim of literal reading, writing, or “understanding.” With that disclaimer in force, the contrast is informative precisely because the model and the brain fall on opposite sides of it. The brain builds literacy by recruiting two pre-existing, dissociable circuits onto a culturally invented artifact (Dehaene and Cohen, 2007, 2011; Morais et al., 1979); the decoder-only LLM builds the same functional competence by collapsing decoding and encoding into one autoregressive mechanism. Behaviourally it shows no read/write dissociation (Section 4); in this sense it “reads without a reader and writes without a writer.” We are careful to scope this to behaviour: we have not performed the model analog of a lesion study (a localized intervention that selectively removes comprehension while sparing production, or vice versa), so the stronger claim that no dissociable component exists is a conjecture we leave to future work, not a measured result.
This places decoder-only language models as a distinct point in the space of possible minds (Sloman, 1984; Shanahan, 2025): systems that attain a recent cultural skill (written language, an invention of roughly five millennia, not an evolved faculty; Pinker, 1994) through an architecture whose internal division of labour bears no structural resemblance to the brain that invented the artifact. We read this not as a deficiency but as a characterization. Following Shanahan (2025) and Mitchell and Krakauer (2023), the appropriate move is to catalogue the mode of intelligence on its own terms rather than to score it against a human template, and our two-level convergence supplies one concrete, measurable axis (read/write entanglement) on which the machine and the brain are organized oppositely.
5.3 The development of the entangled code
The training trajectory clarifies how the single code arises. In pythia-160m the / relationship is non-monotonic: from an essentially uncorrelated initialization the two matrices first couple sharply, peak early in training, and then partially differentiate for the remainder of it. The differentiation is asymmetric, the writing code drifting several times farther from initialization than the reading code, and by the same factor in every frequency decile, which rules out the frequent-token gradient-exposure explanation (Section 4.7). As noted above, we credit Lopardo et al. (2026) for first reporting this asymmetric output-side drift; our controlled, read/write-axis replication is detailed in Section 4.7. The picture is a “couple-then-differentiate” developmental program that nonetheless terminates in coupling, not dissociation.
5.4 Taking the nulls seriously
We regard the negative results as load-bearing, not embarrassing, and report them plainly. Three are central.
The geometrybehaviour bridge is not merely null; it is unavailable.
We had intended to ask whether the degree of weight entanglement predicts the degree of behavioural coupling, and we reported that correlation as a null. We now withdraw the test rather than the finding, because its behavioural term was the degenerate coupling statistic of Section 4.8: correlating against a quantity that is an exact function of its own margins tests nothing. Two obstacles would remain even with a sound behavioural measure. The tied models sit degenerately at with no variance and cannot enter a cross-model test at all. And within training, production is undefined at the early checkpoints where is low and changing, because the model cannot yet generate the target forms; behaviour becomes measurable only after the geometry has plateaued, so the two quantities have little overlapping dynamic range. A cross-level bridge is therefore future work contingent on a valid behavioural measure, and we claim no mechanistic chain from geometry to behaviour.
Compression is not decoupled from functional competence.
We had hypothesized that improvements in raw text compression might come apart from improvements in functional, meaning-sensitive competence. We find no evidence for that decoupling. Across the six Pythia sizes the point estimate runs in the opposite direction (bits-per-byte vs. functional accuracy Pearson , ; Spearman , ; ): if anything better compression accompanies better functional performance. With this is underpowered and cannot establish co-variation either; the defensible conclusion is simply that the “compression decoupled from reasoning” sub-claim is not supported, and we withdraw it as a positive claim. This does not overturn the broader form-before-meaning pattern: formal competence (BLiMP) leads functional competence across the ladder, with a gap of a clear margin even at the largest model we test. The specific decoupling mechanism we proposed, however, is absent here.
Mechanistic and scale links are weaker than hoped.
Two further negatives constrain our claims. Ablating the top induction head (L4H6) leaves the orthographic word-form gap intact (the gap after ablation is indistinguishable from the clean gap, and from a matched random-head control) so orthographic decoding is not carried by the induction circuit, even though both emerge in temporally aligned phases. The cross-model link between maximal induction and in-context gain is positive but non-significant, and the -versus-scale trend is flat and underpowered (Spearman , permutation , ). We therefore scope the induction claim to “induction circuits are causally implicated in in-context learning” rather than to any architecture-wide or scale-wide generalization.
6 Limitations
Analogy, not homology.
As emphasized throughout, and are not reading and writing networks. They are matrices that map tokens into and out of the residual stream (Elhage et al., 2021), operating in a single modality (discrete text) at a single architectural stage. The brain’s alexia/agraphia dissociation involves distinct anatomical systems across distinct stages of perception and production (Purcell et al., 2017). Our comparison is a structured analogy that we believe is illuminating, but every quantitative claim concerns the model alone; the neuroscience is the motivating frame, not a co-measured variable.
The architecture nearly forces the result.
A decoder-only transformer is architecturally committed to a single forward path, so some degree of read/write coupling is expected a priori. The independent-init floor shows that the matrices can be fully dissociated at initialization, so the learned coupling is not a trivial consequence of dimensionality; but the finding should be read as “learning drives the codes toward a shared geometry that the architecture permits and rewards,” not as a discovery that two free systems chose to merge.
Tied-versus-untied is confounded across model families.
The tied condition is realized only by the GPT-2 and OPT families and the untied condition only by Pythia (Press and Wolf, 2017; Inan et al., 2017). Tying therefore co-varies with training data, tokenizer, and optimization recipe. The tied ceiling is true by construction and is reported as a sanity check, not as a finding; the substantive numbers come from the untied Pythia suite (Biderman et al., 2023), where the aligned checkpoints let us hold the recipe fixed across scale and training time. A clean within-family tied-vs-untied manipulation would strengthen the architectural claim and is left to future work.
Base models only; and no architecture contrast.
All models studied are base pretrained LMs; instruction tuning and RLHF most directly reshape production behaviour, so claims about output-side behaviour are scoped to base models. We had also intended to test whether read/write coupling is specific to the decoder-only forward path, by contrasting decoder-only, encoder–decoder and masked-encoder models behaviourally and by measuring how differently an encoder–decoder’s two stacks represent the same text. Both halves are withdrawn (Section 4.8): the behavioural half inherits the contingency defect, and the representational half is fixed by architecture class before any data is collected, is unchanged when linguistic structure is destroyed, and is unstable in the stimulus set. We therefore make no cross-architecture claim in either direction. This is a real gap: the weight-level entanglement is measured only on decoder-only suites, so whether an architecture with physically separated pathways would show a different read/write geometry is open. A matched-data, matched-tokenizer manipulation of architecture alone, with a behavioural measure that survives the objections in Section 4.8, is the experiment we would run next.
Small samples and few-point fits.
Several headline relationships rest on small : the scale trend and the compression–functional correlation use , and the bridge tests use across models and within training. We report bootstrap confidence intervals where available and label few-point fits as suggestive; the corresponding nulls are stated as underpowered rather than as positive evidence of no effect.
Stimuli, tokenization, and the human comparison.
The situation-tracking and absurdity items in the functional battery are author-constructed and templated (frozen across runs for reproducibility), with no human ceiling, so functional accuracy should be read as performance on these probes rather than on meaning in general. All measures inherit the BPE tokenizer, which we control for with token-count and subword-unigram matching but cannot fully neutralize. Finally, the comparison of machine token budgets to the 100M-word human-by-age-13 reference (Warstadt et al., 2023) is order-of-magnitude and illustrative: children receive multimodal, embodied, interactive input, whereas these models see web text, so the contrast indexes a gross efficiency gap, not a controlled equivalence (Chollet, 2019).
7 Conclusion
Written language is a recent cultural invention, not an evolved instinct (Pinker, 1994; Dehaene and Cohen, 2007), and in the literate brain it is carried by two doubly-dissociable systems for decoding and encoding that share only a partial orthographic core. A decoder-only language model attains the same functional skill by a structurally opposite route: it collapses reading and writing into one entangled autoregressive code. We operationalized this contrast through the reading code and the writing code and a composite entanglement index , and we found in the weights that the code is single rather than dissociated: untied codes coupled well above the independent-init floor yet well below the tied ceiling, on a couple-then-differentiate training trajectory with an asymmetric output-side drift that replicates Lopardo et al. (2026). We were equally explicit about the limits. The matching behavioural test cannot be run: comprehension and production, as output-side scores define them, are functionals of one next-token distribution and share terms that manufacture the coupling they appear to reveal, so a coupling statistic, a cross-level bridge and an encoder/decoder separation measure are withdrawn. And the “compression decoupled from reasoning” hypothesis is not supported, because compression and functional competence co-vary with scale.
The broader lesson speaks to the debate over the path to general intelligence. These models demonstrate that a single general mechanism (self-attention; Vaswani et al., 2017; Olsson et al., 2022) optimized on a cultural artifact can reach impressive functional literacy without reconstructing any of the brain’s architecture for that skill, mastering form before meaning (Mahowald et al., 2024; Fedorenko et al., 2024) and acting in part as a repository of ingested text rather than a grounded abstraction (Carlini et al., 2023; Harnad, 1990; Yiu et al., 2024; Bender et al., 2021). Rather than asking whether such a system is on the road to human-like understanding, our results recommend treating it as what it measurably is: a distinct point in the space of possible minds (Sloman, 1984; Shanahan, 2025; Mitchell and Krakauer, 2023), one that reads and writes through a single statistical code where the brain keeps two. Charting that space, rather than collapsing it onto a human yardstick, is the lens we offer for the next round of the question.
References
- On the dangers of stochastic parrots: can language models be too big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), pp. 610-623, Cited by: §2.4, §7.
- Pythia: a suite for analyzing large language models across training and scaling. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202; arXiv:2304.01373, Cited by: §1, §2.3, §3.1, §6.
- The role of the graphemic buffer in spelling: evidence from a case of acquired dysgraphia. Cognition, 26(1), 59–85. Cited by: §1, §2.1, §3, §5.2.
- Quantifying memorization across neural language models. In International Conference on Learning Representations (ICLR 2023); arXiv:2202.07646, Cited by: 1st item, §3.3, §4.6, §7.
- An anatomical signature for literacy. Nature, 461(7266), 983-986. Cited by: §1, §2.1.
- Understanding token probability encoding in output embeddings. In Proceedings of COLING 2025, pp. 10618-10633; arXiv:2406.01468, Note: arXiv:2406.01468 Cited by: §2.3, §4.7.
- On the measure of intelligence. Note: arXiv:1911.01547 Cited by: §2.2, §3.3, §4.3, §6.
- DRC: a dual route cascaded model of visual word recognition and reading aloud. Psychological Review, 108(1), 204-256. Cited by: §1, §2.1, §3.3, §5.2.
- The unique role of the visual word form area in reading. Trends in Cognitive Sciences, 15(6), 254–262. Cited by: §1, §2.1, §3.3, §4.2, §5.2.
- Cultural recycling of cortical maps. Neuron, 56(2):384-398. Cited by: §1, §2.1, §3.3, §4.2, §5.2, §7.
- Contribution à l’étude anatomo-pathologique et clinique des différentes variétés de cécité verbale. Mémoires de la Société de Biologie, 4, 61–90. Cited by: §1, §2.1, §3, §5.2.
- BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT 2019, pp. 4171-4186; arXiv:1810.04805, Cited by: §3.3.
- A mathematical framework for transformer circuits. Note: Transformer Circuits Thread (transformer-circuits.pub) Cited by: §1, §2.3, §2.3, §3.3, §3, §4.1, §6.
- Language is primarily a tool for communication rather than thought. Nature, 630:575-586. Cited by: §1, §2.4, §3.3, §4.4, §7.
- The symbol grounding problem. Note: Physica D: Nonlinear Phenomena, 42(1-3):335-346 Cited by: §1, §2.4, §3.3, §4.4, §4.6, §7.
- Position: the platonic representation hypothesis. In Proceedings of ICML 2024, PMLR 235:20617-20642; arXiv:2405.07987, Cited by: §1, §2.3, item 3, §4.7.
- Tying word vectors and word classifiers: a loss framework for language modeling. In ICLR 2017; arXiv:1611.01462, Cited by: §1, §2.3, §3.1, §3.3, §4.7, §6.
- Scaling laws for neural language models. Note: arXiv:2001.08361 Cited by: §2.3, §3.3, §4.5.
- Similarity of neural network representations revisited. In ICML 2019 (PMLR 97); arXiv:1905.00414, Cited by: §1, §2.3, item 1, §4.7.
- Representational similarity analysis – connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, article 4. Cited by: §2.3, §3.2, §4.7.
- Leveraging human production-interpretation asymmetries to test llm cognitive plausibility. In Proceedings of ACL 2025 (Short Papers); arXiv:2503.17579, Note: arXiv:2503.17579 Cited by: §1, §2.4, §3.3.
- RoBERTa: a robustly optimized bert pretraining approach. Note: arXiv:1907.11692 Cited by: §3.3.
- Weight tying biases token embeddings towards the output space. Note: arXiv:2603.26663 External Links: 2603.26663 Cited by: Table B11, Table B11, §1, §2.3, §3.3, Figure 6, Figure 6, §4.7, §5.3, §7.
- Dissociating language and thought in large language models. Trends in Cognitive Sciences 28(6): 517-540. Cited by: 1st item, §1, §1, §2.4, §3.3, §3.3, §4.4, §7.
- The debate over understanding in ai’s large language models. Proceedings of the National Academy of Sciences 120(13): e2215907120. Cited by: §1, §1, §2.4, §3.3, §5.2, §7.
- Does awareness of speech as a sequence of phones arise spontaneously?. Cognition, 7(4), 323-331. Cited by: §1, §2.2, §3.3, §4.2, §5.2.
- In-context learning and induction heads. Note: Transformer Circuits Thread (Anthropic) Cited by: Figure A1, Figure A1, 1st item, §1, §2.3, §3.3, §7.
- An integrated theory of language production and comprehension. Behavioral and Brain Sciences 36(4):329-347. Cited by: §1, §2.4, §3.3.
- The language instinct: how the mind creates language. William Morrow / Penguin (book). Cited by: §1, §2.2, §5.2, §7.
- Using the output embedding to improve language models. In Proceedings of EACL 2017, pp. 157-163 (arXiv:1608.05859), Cited by: §1, §2.3, §3.1, §3.3, §4.7, §6.
- Shared orthographic neuronal representations for spelling and reading. NeuroImage, 147, 554–567. Cited by: §1, §2.1, §3, §5.2, §6.
- Language models are unsupervised multitask learners. Note: OpenAI Technical Report Cited by: §1, §3.1.
- Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67; arXiv:1910.10683. Cited by: §3.3.
- The graphemic/motor frontal area (exner’s area) revisited. Annals of Neurology, 66(4), 537-545. Note: Annals of Neurology, 66(4), 537–545 External Links: Document Cited by: §1, §2.1, §3, §5.2.
- Palatable conceptions of disembodied being: terra incognita in the space of possible minds. Note: arXiv preprint arXiv:2503.16348 (also PhilPapers SHAPCO-6) External Links: 2503.16348 Cited by: §1, §1, §3.3, §5.2, §7.
- The structure of the space of possible minds. Note: in S. Torrance (ed.), The Mind and the Machine: Philosophical Aspects of Artificial Intelligence, Ellis Horwood Cited by: §1, §3.3, §5.2, §7.
- Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In ICLR 2017; arXiv:1702.03859, Cited by: §1, §2.3, item 2, §3.3, §4.7.
- The origins of writing. Note: The Met — Heilbrunn Timeline of Art History (essay) Cited by: §2.2.
- Attention is all you need. In Advances in Neural Information Processing Systems 30 (NeurIPS 2017), pp. 5998-6008, Cited by: §1, §2.3, §3, §4.1, §7.
- Analyzing multi-head self-attention: specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the ACL, pp. 5797-5808, Cited by: Figure A1, Figure A1, §2.3, §3.3.
- Findings of the babylm challenge: sample-efficient pretraining on developmentally plausible corpora. In Proceedings of the BabyLM Challenge / CoNLL (also arXiv:2504.08165 reissue), Cited by: §2.2, §3.3, §4.3, §6.
- Transmission versus truth, imitation versus innovation: what children can do that large language and language-and-vision models cannot (yet). Perspectives on Psychological Science, 19(5):874-883 (arXiv:2305.07666, 2023). Cited by: 1st item, §3.3, §4.6, §7.
- OPT: open pre-trained transformer language models. Note: arXiv:2205.01068 Cited by: §1, §3.1.
Appendix A Supplementary figures
The four figures collected here support claims established in the main text but do not carry an independent argument: two are single-model illustrations (the per-head induction map and the cross-model induction-vs-ICL scatter), and two visualize underpowered null relationships (the entanglement index against scale, and the geometrybehaviour bridge). We place them here to keep the main narrative focused while preserving the full evidentiary record.



Appendix B Supplementary tables
The tables below collect every numerical result referenced in the main text, in order of first mention and renumbered B1–B11. Table LABEL:tab:exp3_data_dynamics gives the full tokens-to-competence trajectory across all released checkpoints.
Model L H Max induction [95% CI] ICL (syn.) ICL ablate head ICL ablate MLP head vs null distilgpt2 6 12 0.946 [0.944, 0.947] -12.656 0.097 0.258 -0.091 gpt2 12 12 0.935 [0.933, 0.937] -12.570 -0.056 0.123 -1.502 gpt2-medium 24 16 0.964 [0.963, 0.965] -12.627 0.009 0.109 0.077 facebook/opt-125m 12 12 0.937 [0.937, 0.938] -12.254 0.466 0.144 9.572 facebook/opt-350m 24 16 0.949 [0.947, 0.951] -12.440 0.039 0.096 0.477 EleutherAI/pythia-70m 6 8 0.869 [0.866, 0.873] -11.404 1.413 2.158 0.885 EleutherAI/pythia-160m 12 12 0.978 [0.977, 0.979] -17.680 1.652 0.650 4.249 EleutherAI/pythia-410m 24 16 0.971 [0.970, 0.972] -12.356 0.094 -0.270 2.069
Model Word-form gap @init Word-form gap @final Token-ctrl gap @final Tok-matched gap @final Legality gap @final Legality tok-ctrl @final pythia-160m 5.883 [5.839, 5.922] 4.375 [4.325, 4.436] 1.854 [1.739, 1.952] 3.795 [3.583, 4.007] 1.652 [1.617, 1.706] 0.594 [0.585, 0.608] pythia-410m 5.891 [5.825, 5.954] 4.535 [4.493, 4.565] 2.034 [1.880, 2.221] 4.366 [4.204, 4.518] 1.816 [1.762, 1.857] 0.767 [0.738, 0.817]
Model Best layer 3-class acc Chance Perm null Real-vs-scrambled (best layer) EleutherAI/pythia-160m@step143000 8 0.933 0.333 0.352 0.869 gpt2 9 0.935 0.333 0.331 0.828
| Model | Ckpt | Tokens (word-eq.) | Loss (nats) | Perplexity (doc-level range) | BLiMP acc [95% CI] |
|---|---|---|---|---|---|
| EleutherAI/pythia-160m | step1 | 2.10e+06 (1.61e+06) | 11.044 | 62554.8 [61964.9, 63169.2] | 0.484 [0.463, 0.507] |
| EleutherAI/pythia-160m | step2 | 4.19e+06 (3.23e+06) | 11.043 | 62499.8 [61911.3, 63114.6] | 0.484 [0.463, 0.507] |
| EleutherAI/pythia-160m | step4 | 8.39e+06 (6.45e+06) | 11.005 | 60180.1 [59673.1, 60812.4] | 0.483 [0.461, 0.505] |
| EleutherAI/pythia-160m | step8 | 1.68e+07 (1.29e+07) | 10.844 | 51211.4 [50889.6, 52043.3] | 0.486 [0.465, 0.509] |
| EleutherAI/pythia-160m | step16 | 3.36e+07 (2.58e+07) | 10.473 | 35356.4 [34893.7, 36390.6] | 0.500 [0.478, 0.521] |
| EleutherAI/pythia-160m | step32 | 6.71e+07 (5.16e+07) | 9.992 | 21851.0 [21254.5, 22754.5] | 0.520 [0.498, 0.542] |
| EleutherAI/pythia-160m | step64 | 1.34e+08 (1.03e+08) | 9.321 | 11171.6 [10741.7, 11673.3] | 0.442 [0.420, 0.463] |
| EleutherAI/pythia-160m | step128 | 2.68e+08 (2.06e+08) | 8.186 | 3592.1 [3416.2, 3789.4] | 0.508 [0.487, 0.530] |
| EleutherAI/pythia-160m | step256 | 5.37e+08 (4.13e+08) | 7.430 | 1686.5 [1589.3, 1791.8] | 0.599 [0.577, 0.620] |
| EleutherAI/pythia-160m | step512 | 1.07e+09 (8.26e+08) | 6.565 | 709.9 [671.7, 764.6] | 0.587 [0.565, 0.608] |
| EleutherAI/pythia-160m | step1000 | 2.10e+09 (1.61e+09) | 5.432 | 228.5 [231.5, 270.9] | 0.647 [0.626, 0.668] |
| EleutherAI/pythia-160m | step2000 | 4.19e+09 (3.23e+09) | 4.707 | 110.7 [111.9, 130.0] | 0.716 [0.696, 0.736] |
| EleutherAI/pythia-160m | step4000 | 8.39e+09 (6.45e+09) | 4.323 | 75.4 [76.0, 87.7] | 0.756 [0.737, 0.774] |
| EleutherAI/pythia-160m | step8000 | 1.68e+10 (1.29e+10) | 4.121 | 61.6 [62.5, 71.2] | 0.753 [0.733, 0.772] |
| EleutherAI/pythia-160m | step16000 | 3.36e+10 (2.58e+10) | 3.984 | 53.7 [54.5, 62.6] | 0.772 [0.752, 0.790] |
| EleutherAI/pythia-160m | step32000 | 6.71e+10 (5.16e+10) | 3.905 | 49.7 [50.3, 57.5] | 0.757 [0.738, 0.776] |
| EleutherAI/pythia-160m | step64000 | 1.34e+11 (1.03e+11) | 3.861 | 47.5 [48.0, 54.7] | 0.762 [0.742, 0.780] |
| EleutherAI/pythia-160m | step143000 | 3.00e+11 (2.31e+11) | 3.904 | 49.6 [49.7, 56.5] | 0.744 [0.725, 0.763] |
| EleutherAI/pythia-410m | step1 | 2.10e+06 (1.61e+06) | 11.058 | 63420.1 [63017.6, 64207.2] | 0.551 [0.528, 0.572] |
| EleutherAI/pythia-410m | step2 | 4.19e+06 (3.23e+06) | 11.055 | 63272.2 [62870.7, 64067.1] | 0.551 [0.529, 0.573] |
| EleutherAI/pythia-410m | step4 | 8.39e+06 (6.45e+06) | 10.965 | 57816.1 [57508.4, 58802.2] | 0.564 [0.542, 0.585] |
| EleutherAI/pythia-410m | step8 | 1.68e+07 (1.29e+07) | 10.686 | 43722.2 [43343.7, 45066.2] | 0.583 [0.561, 0.605] |
| EleutherAI/pythia-410m | step16 | 3.36e+07 (2.58e+07) | 10.242 | 28063.6 [27459.6, 29307.7] | 0.538 [0.516, 0.560] |
| EleutherAI/pythia-410m | step32 | 6.71e+07 (5.16e+07) | 9.930 | 20529.2 [19859.2, 21515.3] | 0.512 [0.490, 0.534] |
| EleutherAI/pythia-410m | step64 | 1.34e+08 (1.03e+08) | 9.487 | 13189.4 [12623.7, 13878.4] | 0.475 [0.453, 0.497] |
| EleutherAI/pythia-410m | step128 | 2.68e+08 (2.06e+08) | 8.615 | 5515.5 [5283.7, 5847.2] | 0.497 [0.476, 0.519] |
| EleutherAI/pythia-410m | step256 | 5.37e+08 (4.13e+08) | 7.660 | 2122.0 [2010.7, 2263.1] | 0.558 [0.536, 0.579] |
| EleutherAI/pythia-410m | step512 | 1.07e+09 (8.26e+08) | 6.897 | 989.3 [928.9, 1055.5] | 0.583 [0.561, 0.605] |
| EleutherAI/pythia-410m | step1000 | 2.10e+09 (1.61e+09) | 5.497 | 244.1 [247.8, 287.2] | 0.660 [0.639, 0.680] |
| EleutherAI/pythia-410m | step2000 | 4.19e+09 (3.23e+09) | 4.513 | 91.2 [91.6, 105.8] | 0.696 [0.675, 0.716] |
| EleutherAI/pythia-410m | step4000 | 8.39e+09 (6.45e+09) | 4.078 | 59.0 [59.9, 68.8] | 0.758 [0.738, 0.777] |
| EleutherAI/pythia-410m | step8000 | 1.68e+10 (1.29e+10) | 3.842 | 46.6 [47.6, 54.3] | 0.782 [0.764, 0.801] |
| EleutherAI/pythia-410m | step16000 | 3.36e+10 (2.58e+10) | 3.645 | 38.3 [38.8, 44.3] | 0.806 [0.788, 0.824] |
| EleutherAI/pythia-410m | step32000 | 6.71e+10 (5.16e+10) | 3.557 | 35.1 [35.6, 40.5] | 0.781 [0.763, 0.800] |
| EleutherAI/pythia-410m | step64000 | 1.34e+11 (1.03e+11) | 3.475 | 32.3 [32.8, 37.5] | 0.777 [0.758, 0.795] |
| EleutherAI/pythia-410m | step143000 | 3.00e+11 (2.31e+11) | 3.381 | 29.4 [29.8, 34.2] | 0.805 [0.787, 0.823] |
| EleutherAI/pythia-70m | step1 | 2.10e+06 (1.61e+06) | 11.002 | 59996.7 [59729.3, 60941.4] | 0.491 [0.469, 0.513] |
| EleutherAI/pythia-70m | step2 | 4.19e+06 (3.23e+06) | 11.002 | 59982.0 [59714.8, 60926.2] | 0.491 [0.470, 0.513] |
| EleutherAI/pythia-70m | step4 | 8.39e+06 (6.45e+06) | 10.997 | 59710.0 [59444.3, 60651.9] | 0.493 [0.472, 0.515] |
| EleutherAI/pythia-70m | step8 | 1.68e+07 (1.29e+07) | 10.937 | 56202.6 [55982.7, 57129.1] | 0.495 [0.474, 0.517] |
| EleutherAI/pythia-70m | step16 | 3.36e+07 (2.58e+07) | 10.745 | 46409.4 [46217.0, 47296.7] | 0.501 [0.479, 0.523] |
| EleutherAI/pythia-70m | step32 | 6.71e+07 (5.16e+07) | 10.325 | 30483.4 [30025.6, 31435.9] | 0.525 [0.503, 0.546] |
| EleutherAI/pythia-70m | step64 | 1.34e+08 (1.03e+08) | 9.587 | 14572.1 [14210.9, 15168.6] | 0.584 [0.563, 0.606] |
| EleutherAI/pythia-70m | step128 | 2.68e+08 (2.06e+08) | 8.172 | 3539.5 [3392.0, 3735.9] | 0.514 [0.492, 0.537] |
| EleutherAI/pythia-70m | step256 | 5.37e+08 (4.13e+08) | 7.408 | 1649.8 [1543.5, 1747.6] | 0.587 [0.566, 0.609] |
| EleutherAI/pythia-70m | step512 | 1.07e+09 (8.26e+08) | 6.596 | 732.5 [682.5, 777.7] | 0.572 [0.551, 0.594] |
| EleutherAI/pythia-70m | step1000 | 2.10e+09 (1.61e+09) | 5.531 | 252.3 [254.2, 298.3] | 0.609 [0.587, 0.631] |
| EleutherAI/pythia-70m | step2000 | 4.19e+09 (3.23e+09) | 4.969 | 143.9 [144.2, 168.2] | 0.682 [0.661, 0.702] |
| EleutherAI/pythia-70m | step4000 | 8.39e+09 (6.45e+09) | 4.728 | 113.1 [114.2, 132.5] | 0.720 [0.699, 0.739] |
| EleutherAI/pythia-70m | step8000 | 1.68e+10 (1.29e+10) | 4.595 | 99.0 [100.2, 116.5] | 0.739 [0.720, 0.758] |
| EleutherAI/pythia-70m | step16000 | 3.36e+10 (2.58e+10) | 4.497 | 89.7 [90.3, 104.5] | 0.713 [0.692, 0.732] |
| EleutherAI/pythia-70m | step32000 | 6.71e+10 (5.16e+10) | 4.444 | 85.1 [85.3, 98.2] | 0.718 [0.698, 0.737] |
| EleutherAI/pythia-70m | step64000 | 1.34e+11 (1.03e+11) | 4.433 | 84.2 [85.2, 98.1] | 0.720 [0.699, 0.739] |
| EleutherAI/pythia-70m | step143000 | 3.00e+11 (2.31e+11) | 4.464 | 86.8 [87.1, 100.1] | 0.718 [0.698, 0.737] |
Model N (non-emb.) BLiMP (formal) PIQA Situation Absurdity Grounding gap pythia-14m 1.2M 0.695 [0.674, 0.715] 0.596 [0.552, 0.636] 0.525 [0.433, 0.617] 1.000 [1.000, 1.000] -0.050 pythia-70m 18.9M 0.718 [0.698, 0.737] 0.576 [0.532, 0.618] 0.333 [0.250, 0.417] 1.000 [1.000, 1.000] 0.500 pythia-160m 85.1M 0.744 [0.725, 0.763] 0.616 [0.570, 0.656] 0.425 [0.333, 0.508] 1.000 [1.000, 1.000] 0.308 pythia-410m 302.3M 0.805 [0.787, 0.823] 0.670 [0.626, 0.708] 0.717 [0.633, 0.800] 1.000 [1.000, 1.000] -0.225 pythia-1b 805.7M 0.806 [0.789, 0.824] 0.692 [0.650, 0.730] 0.725 [0.642, 0.800] 1.000 [1.000, 1.000] -0.433 pythia-1.4b 1208.6M 0.824 [0.807, 0.841] 0.688 [0.646, 0.726] 0.583 [0.492, 0.675] 1.000 [1.000, 1.000] 0.008
Model CE (nats) PPL bits/byte EleutherAI/pythia-14m 1189888 4.728 113.028 1.489 EleutherAI/pythia-70m 18915328 3.959 52.383 1.247 EleutherAI/pythia-160m 85056000 3.395 29.821 1.069 EleutherAI/pythia-410m 302311424 2.896 18.107 0.912 EleutherAI/pythia-1b 805736448 2.705 14.949 0.852 EleutherAI/pythia-1.4b 1208602624 2.593 13.364 0.816
Model neural PPL n-gram PPL ratio log-PPL gap EleutherAI/pythia-14m 1189888 104.467 473.479 0.221 1.511 EleutherAI/pythia-70m 18915328 48.577 473.479 0.103 2.277 EleutherAI/pythia-160m 85056000 27.922 473.479 0.059 2.831 EleutherAI/pythia-410m 302311424 16.234 473.479 0.034 3.373 EleutherAI/pythia-1b 805736448 13.383 473.479 0.028 3.566 EleutherAI/pythia-1.4b 1208602624 11.843 473.479 0.025 3.688
Finding Headline Baseline/control P1 memorization vs log(params) 0.011 [0.007, 0.017] non-member extraction (generic fluency) P1 memorization vs log(prefix len) 0.017 [-0.012, 0.045] non-member extraction (generic fluency) P2a imitationinnovation gap (max) 0.090 PPMI rarity gap 0.000 P2b geometry vs perceptual |rho| (best) 0.826 PPMI |rho| max 0.512
Model Tying E [95% CI] CKA Procrustes resid. mean cos. kNN@10 CKA floor pythia-14m untied 0.226 [0.223, 0.228] 0.182 1.142 0.401 0.159 0.003 pythia-70m untied 0.349 [0.338, 0.360] 0.353 1.015 0.475 0.396 0.010 pythia-160m untied 0.308 [0.301, 0.315] 0.210 0.966 0.454 0.451 0.015 pythia-410m untied 0.327 [0.324, 0.330] 0.295 1.124 0.404 0.534 0.020 pythia-1b untied 0.348 [0.345, 0.351] 0.339 1.105 0.439 0.585 0.039 pythia-1.4b untied 0.335 [0.332, 0.338] 0.317 1.109 0.421 0.588 0.039 gpt2 tied 1.000 [1.000, 1.000] 1.000 0.000 1.000 1.000 0.015 gpt2-medium tied 1.000 [1.000, 1.000] 1.000 0.000 1.000 1.000 0.020 opt-125m tied 1.000 [1.000, 1.000] 1.000 0.000 1.000 1.000 0.015 opt-350m tied 1.000 [1.000, 1.000] 1.000 0.000 1.000 1.000 0.010
Model cos (obs.) cos (shuffle) cos (floor) kNN (obs.) kNN (shuffle) kNN (floor) pythia-14m 0.401 0.038 0.043 0.159 0.000 0.000 pythia-70m 0.475 0.076 0.085 0.396 0.000 0.000 pythia-160m 0.454 0.094 0.104 0.451 0.000 0.000 pythia-410m 0.404 0.114 0.121 0.534 0.000 0.000 pythia-1b 0.439 0.159 0.171 0.585 0.000 0.000 pythia-1.4b 0.421 0.158 0.171 0.588 0.000 0.000
Frequency decile drift drift ratio 0 (rarest) 1.551 5.048 3.254 1 () 1.581 5.066 3.204 2 () 1.583 5.069 3.203 3 () 1.583 5.070 3.204 4 () 1.584 5.098 3.218 5 () 1.591 5.145 3.235 6 () 1.593 5.156 3.236 7 () 1.592 5.172 3.249 8 () 1.584 5.179 3.270 9 (most freq.) 1.533 5.180 3.379