{"id":"90c5f2e3-cb27-4bf2-ad42-b734506ffe79","arxiv_id":"2505.22308","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Different procedural pretraining tasks create complementary, transferable structures in a transformer's attention and MLP weights, and structures from different tasks can be combined into one initialization.","lead":"This paper shows that when tiny transformers are pretrained on simple synthetic rule-based data, the useful skills they learn land in different parts of the network, sometimes in attention layers and sometimes in MLP layers, and these parts can be mixed across models. The finding is a step toward understanding how reasoning skills can be taught separately from factual knowledge in language models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Selective-transfer results conflate stored structure with optimization-landscape effects; composition tables lack random-component controls.","rationale":"The reader's weakest assumption correctly identifies the central soft spot: the selective-transfer protocol cannot distinguish 'this component stores the skill' from 'this component provides a favorable optimization landscape.' The paper's own statement that the entire model is fine-tuned after partial initialization makes this ambiguity direct, and the full-transfer-versus-attention-only gap on HAYSTACK is exactly the kind of result that would change if the untransferred components were varied. The composition result inherits the same problem and additionally lacks a random-component control, so the modular-composition conclusion is not uniquely supported. I agree with the reader's assessment and do not see a reason to move the verdict: CONDITIONAL remains appropriate until the control experiments are run. I am not raising a separate objection to the use of procedural data or to the perturbation analysis, which does provide some evidence that precise weight structure matters. The remaining concerns, such as high variance in several cells of Appendix C and the absence of released code or data, are secondary to the localization and composition issue.","tokens_in":13738,"tokens_out":3960,"duration_ms":49776,"concrete_test":"Run the Table 1 composition with four arms, all sharing seeds and fine-tuning budget: (a) SET attention + ECA MLP, (b) SET attention + random MLP, (c) random attention + ECA MLP, (d) all-random baseline. If (a) does not beat both (b) and (c) by more than seed noise on all four tasks, complementarity is not established. In addition, for the localization claim, after fine-tuning the attention-only transfer on HAYSTACK, freeze all other weights, reset the attention weights to a fresh random initialization, and measure test accuracy: if accuracy remains near the unreset level, the skill was absorbed by the fine-tuned MLP/embeddings rather than stored in attention; if it collapses, the attention weights carry the structure.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that procedural pretraining creates modular structures in specific architectural components rests on selective-transfer experiments in Section 4.1 in which only the transferred component is pretrained and then 'the entire model is fine-tuned.' Final accuracy therefore measures the optimization trajectory induced by the initialization, not whether the transferred component stores the downstream skill at convergence. The strongest observation, that attention-only transfer beats full transfer on HAYSTACK (IDENTITY: 18.8% to 99.0%), is equally consistent with the pretrained MLPs being a poor starting point while random MLPs adapt more easily. The perturbation experiments in Section 4.2 show that precise weights matter, but they do not localize the effect to attention versus MLP. The composition experiment in Section 5 and Table 1 lacks controls in which one component is replaced by random weights, so the apparent complementarity of SET attention and ECA MLPs may simply reflect SET attention alone with a neutral MLP initialization. Without these controls, the paper's strongest claim about independent, composable modular structures is not uniquely supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper pretrains small GPT-2-style transformers on procedurally generated data (k-DYCK, k-DYCK SHUFFLE, STACK, IDENTITY, SET, and ECA Rule 110) and then fine-tunes them on algorithmic diagnostic tasks (needle-in-a-haystack, addition, reversed addition, multiplication, sorting, and language modeling). It reports that different procedural pretraining tasks improve different downstream skills, that selective transfer of only attention or only MLP weights can outperform full-model transfer, that perturbing pretrained weights degrades performance in task-dependent ways, and that combining the attention weights of a SET-pretrained model with the MLP weights of an ECA-pretrained model can yield a strong multi-task initialization. The central claim is that procedural pretraining creates modular, composable inductive structures that reside in specific architectural components.","tokens_in":13915,"tokens_out":5447,"duration_ms":54103,"significance":"If the localization and composition claims hold, this would be a useful step toward understanding when and why synthetic procedural data helps downstream reasoning, and toward composing skills from separately pretrained models. The paper has real strengths: it includes random-initialization baselines, noise and shuffling perturbations, full-versus-selective transfer comparisons, and a clearly specified experimental pipeline. However, the main evidence for 'where the structure resides' and for 'composable modules' is inferential and, in its current form, does not uniquely exclude simpler explanations based on optimization-landscape effects of the initialization. The result is therefore promising but not yet established.","major_comments":[{"comment":"The claim that useful structures 'reside in specific architectural components' is not uniquely supported, because in the selective-transfer setup described in §4.1 'the entire model is fine-tuned' after initialization. A comparison such as STACK attention-only 98.9 ± 0.8 versus STACK full-transfer 55.2 ± 39.3 on HAYSTACK (Table 4) is equally consistent with the hypothesis that the pretrained MLP weights are a worse optimization starting point than random MLP weights, and that the attention weights themselves carry little task-specific structure. The same confound affects Table 7, where IDENTITY attention-only reaches 91.3 ± 10.1 on SORTING while IDENTITY full transfer reaches only 19.9 ± 0.5. To localize the effect, the authors should add controls where the non-transferred components are frozen rather than fine-tuned, or where each component is replaced individually by random weights while the others are held fixed, and they should report performance at initialization and during early fine-tuning.","section":"§4.1, Tables 4–7"},{"comment":"The composition experiment lacks random-component controls. The table compares SET (attention only), ECA (MLPs only), and SET-attention + ECA-MLPs, but there is no row such as SET-attention + random-MLPs or random-attention + ECA-MLPs. Without these controls, the strong performance of the combined model on REVERSED ADDITION (82.9 ± 16.9) relative to SET attention alone (54.4 ± 10.4) does not uniquely establish that the ECA MLP weights contribute transferable structure; it could result from a more favorable random-like MLP initialization. Conversely, on ADDITION the combined model (80.3 ± 13.9) is not better than SET attention alone (81.1 ± 12.2). The authors should add random-component controls and also test at least one additional pair of pretraining tasks before claiming that structures 'can be composed' generally.","section":"§5, Table 1"},{"comment":"The text states that 'procedurally pretrained models can largely outperform randomly initialized ones for every downstream task,' but Table 3 contains many configurations where procedural pretraining hurts relative to random initialization, e.g., 4-DYCK on ADDITION (52.7 ± 0.3 vs. 59.1 ± 7.0), STACK on SORTING (21.3 ± 0.6 vs. 82.7 ± 11.6), and 4-DYCK on REVERSED ADDITION (35.7 ± 2.5 vs. 76.4 ± 23.2). The paper should explicitly acknowledge that the beneficial effects are selective and task-dependent, and should avoid presenting 'best per task' cells as if they were representative of all procedural pretraining. Because the diagnostic tasks were chosen from the same benchmark family on which the positive results are demonstrated, the selection of favorable pretraining tasks per diagnostic should be justified as hypothesis-driven rather than post hoc.","section":"§3, Figure 2 and Table 3"},{"comment":"The perturbation analysis applies noise or shuffling to the 'best transfer configurations,' which differ across tasks: attention-only for HAYSTACK, full transfer for ADDITION and SORTING, and so on. Differences in sensitivity across tasks may therefore reflect which components were transferred rather than the nature of the learned skill. For example, SORTING's robustness to shuffling (98.7 → 94.2) is measured on a full-transfer model, whereas HAYSTACK's sensitivity (98.9 → 17.2) is measured on an attention-only transfer. The authors should either apply the perturbations to the same transfer configuration across all tasks or explicitly discuss this confound and report both full- and selective-transfer perturbation results.","section":"§4.2, Table 8"}],"minor_comments":[{"comment":"There are several typographical errors: 'Switzerlad' in the author affiliation, 'autoregressivley' in the SORTING description, 'occurance' in the STACK description, 'two-thrids' in the STACK data generation, and 'afforementioned' in the downstream training section.","section":"Appendix B.3"},{"comment":"The main-text figures omit error bars, and the caption says full statistics are in Appendix C; for a paper whose claims often rest on differences that are small relative to the reported standard deviations (e.g., REVERSED ADDITION in Table 6), the main figures should show at least some measure of variability.","section":"Figure 2"},{"comment":"The claim that 'attention layers often carry the most transferable information' is nuanced by Table 6: on REVERSED ADDITION, MLP-only transfer frequently outperforms attention-only transfer (e.g., STACK MLP-only 74.4 ± 24.7 vs. attention-only 42.1 ± 8.1). The text should acknowledge this more explicitly to avoid overgeneralizing the attention-centric conclusion.","section":"§4.1, Tables 4–7"},{"comment":"The caption of Table 8 states that pretrained models were 'selected based on best individual performance per task,' but the choice for REVERSED ADDITION is 8-DYCK SHUFFLE (90.1 ± 5.9) even though ECA achieves 91.1 ± 16.1 in Table 3; the selection rule should be stated precisely, including how ties or overlapping confidence intervals were resolved.","section":"Appendix C, Table 8"},{"comment":"The description of embedding transfer is incomplete for ECA: Appendix B.1 explains that ECA replaces token embeddings with a linear projection, and that 'averaged embeddings' are used for transfer, but the main text's description of resetting to 'average pretrained vector' should reference this special case for clarity.","section":"§2, weight transfer paragraph"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely question and its experimental setup is transparent, but the central localization and composition claims currently rest on an inference that is confounded by fine-tuning dynamics. The missing controls (frozen components, random-component substitutions, and consistent perturbation conditions) are straightforward to run within the paper's scope, so I see this as a major-revision situation rather than a rejection. I would recommend that the revision be evaluated by the same referee if possible, since the requested controls are specific to the current experimental design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a short empirical paper with a clear, interesting thesis: different procedural pretraining tasks impart distinct, partly complementary inductive biases to small transformers, and those biases can be recombined by mixing attention and MLP weights from separately pretrained models. The broad claim holds up. The paper runs more controls than most in this area—random baselines, noise perturbations, weight shuffling, full versus selective transfer—and the headline effects are real. The cleanest result is that attention-only transfer from IDENTITY/STACK/SET gives near-perfect HAYSTACK performance while full transfer fails, a large and reproducible effect that points to attention carrying the relevant structure. The perturbation results also show that shuffling destroys most of the benefit on HAYSTACK and ADDITION, so this is not just a weight-magnitude artifact. The composition experiment (SET attention + ECA MLPs) improves on REVERSED ADDITION over either component alone, which is the most interesting observation in the paper.\n\nThe soft spots are real but not fatal. The localization claim—that the skill 'resides' in specific components—is inferred from selective transfer followed by fine-tuning of the entire model. Attention initialization could be providing a favorable optimization landscape rather than storing a ready-made mechanism. The perturbation experiments do not disentangle these interpretations. The composition experiment also lacks a control where one component is replaced by random weights, so we cannot fully attribute the gain to complementarity. Some reported effects have large standard deviations (e.g., SET attention-only at 88.9 ± 27.1 on HAYSTACK), and no code or data is released. These are legitimate concerns, but the paper is careful not to overclaim in the abstract, and the discussion explicitly lists open questions.\n\nOn novelty and citation pattern: I believe the cross-task recombination of components is genuinely new relative to the cited work. Prior papers did procedural pretraining or subnetwork transfer, but not mixing components from two different procedural tasks to reinforce multiple capabilities. The citations are appropriate and not inflated for effect.\n\nFor this paper, the right venue is a workshop or short-paper track, but I would send it to serious peer review rather than desk-reject it. The core empirical findings are worth reporting, but the modular-residence interpretation needs to be softened or supported by the missing controls. A good referee could push the authors to add a random-component baseline and to reframe 'reside' as 'selectively transferable at initialization.'","headline":"Plausible, well-controlled study showing procedural pretraining creates transferable and partly composable inductive biases; the modular-localization claim is more tentative than the paper suggests.","tokens_in":14434,"tokens_out":3042,"would_cite":true,"duration_ms":32453,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pretraining on procedural data instils distinct, complementary reasoning skills in small transformers, located in specific weight components and composable across models.","keywords":["procedural pretraining","inductive bias","selective weight transfer","modularity","algorithmic reasoning","transformers","attention layers","MLP blocks"],"falsifier":"Run the composition experiment of Section 5 with the ECA MLP block replaced by randomly initialized MLP weights of the same scale, keeping the SET attention block; if this control matches the composed model's accuracy on all four diagnostic tasks, the claimed complementarity of the two pretraining rules collapses.","tokens_in":13556,"feed_emoji":"🧩","tokens_out":8818,"duration_ms":70647,"temperature":0.7,"pith_summary":"The paper claims that pretraining small transformers on simple procedurally generated sequences—nested brackets, stack operations, identity copying, set deduplication, and cellular automata—does not just warm up the network in a generic way. Different procedural rules instil distinct, complementary inductive structures, and those structures settle in different architectural components: attention layers usually carry the most transferable information, while some rules put useful structure into MLP blocks instead. The paper supports this with selective-transfer experiments that initialize only attention or only MLP weights from a pretrained model, and with perturbation experiments showing the benefits depend on precise weight structure rather than mere weight magnitudes. The central constructive result is that modules from separately pretrained models can be recombined, so a model initialized with SET-pretrained attention and ECA-pretrained MLPs fine-tunes well on all four tested reasoning tasks. If right, this points toward training reasoning skills independently of semantic knowledge and assembling them into a generalist model.","feed_headline":"Procedural pretraining builds modular reasoning skills in transformers","feed_subtitle":"Attention and MLP layers carry different skills, so separately pretrained models can be combined.","key_machinery":"The machinery is the decomposition of a transformer's weights into three blocks, $T = (E, A, F)$, with $E$ the token and position embeddings, $A$ the attention layers, and $F$ the MLP layers. The paper's protocol is selective transfer: initialize a fine-tuning run with only one block from a pretrained model and random weights elsewhere, then fine-tune everything. A block is credited with carrying the pretraining structure when this beats both random initialization and full transfer. The perturbation controls, per-layer weight shuffling (which preserves magnitudes but destroys structure) and additive Gaussian noise of increasing variance, separate genuine learned mechanisms from trivial initialization effects. The composition step builds $T_{\\mathrm{comb}} = (E_{\\mathrm{rand}}, A_{\\mathrm{pre}}^1, F_{\\mathrm{pre}}^2)$ from two separately pretrained models, showing that the localized structures are mutually compatible.","core_discovery":"On the paper's terms: procedurally pretrained transformers contain modular inductive structures that are specific to the pretraining rule, localized in particular weight subsets, and transferable across models. Full-model transfer of weights from k-DYCK pretraining to a needle-in-a-haystack task reaches 98.3 percent accuracy versus 11.3 percent from random initialization, while attention-only transfer from STACK or IDENTITY pretraining reaches 98–99 percent, far above full transfer. Reversed addition, in contrast, benefits more from MLP-only or full transfer, showing that the residence of the structure is task-dependent. The perturbation analysis shows that adding Gaussian noise or shuffling weights steadily destroys the downstream benefit, with haystack and addition collapsing under shuffling, which rules out an explanation purely in terms of beneficial weight magnitudes. The composition experiment then shows that taking attention weights from a SET-pretrained model and MLP weights from an ECA-pretrained model gives a single initialization that performs well on haystack, addition, reversed addition, and sorting, where either source alone fails on at least one task.","pith_inferences":["The same selective-transfer logic could serve as a diagnostic on much larger language models, locating where code pretraining or instruction tuning places its skills by fine-tuning only attention or only MLP blocks from a checkpoint.","A direct test of modular composition would replace one donated block with random weights of matched scale; if that control matches the composed model, the complementarity claim would need revision, and the paper does not report this control.","If procedural pretraining installs reasoning mechanisms independently of semantics, pre-pretraining curricula on such data could precede natural-language pretraining, potentially improving the data efficiency of general-purpose models.","The observed task-dependence of where structure resides suggests that capacity could be routed to attention or MLP blocks depending on which downstream skills are prioritized during initialization."],"forward_implications":["Pretraining data can be chosen by the skill it installs, such as k-DYCK for memory recall, ECA for carry-based arithmetic, and SET for sorting, rather than by general language-modeling perplexity.","Partial weight transfer is a practical initialization strategy: copying only attention or only MLP weights can beat transferring the whole pretrained model.","Composed initializations from several small procedural pretrainings can support several downstream capabilities at once, suggesting a path to multi-skill models without a single massive pretraining corpus.","Weight-magnitude effects are not the explanation: the downstream gains require the precise internal arrangement of pretrained weights, so procedural pretraining learns mechanisms, not just calibration."],"supporting_citations":[{"why":"Provides the GPT-2-style architecture whose weights are decomposed into embeddings, attention, and MLP blocks for transfer.","marker":"Radford et al. (2019)"},{"why":"Shows formal-language pretraining is token-efficient and supplies the k-DYCK SHUFFLE generation implementation used as a pretraining rule.","marker":"Hu et al. (2025)"},{"why":"Supplies the ECA Rule 110 pretraining setup and codebase, and the evidence that cellular-automata data aids abstract reasoning.","marker":"Zhang et al. (2024)"},{"why":"Provides the needle-in-a-haystack and reversed-addition task implementations used as downstream diagnostics.","marker":"Zhong & Andreas (2024)"},{"why":"Establishes that weight-magnitude initialization alone changes transformer optimization, the baseline the perturbation controls must beat.","marker":"Huang et al. (2020)"},{"why":"Provides the TinyStories dataset used for the language-modeling diagnostic task.","marker":"Eldan & Li (2023)"},{"why":"Supplies the average-embedding initialization used when pretraining and fine-tuning vocabularies do not align.","marker":"Hewitt (2021)"},{"why":"Provides the stack-based k-DYCK sequence generation procedure with its opening-bracket probability.","marker":"Papadimitriou & Jurafsky (2023)"}],"fun_headline_variants":["Procedural pretraining installs modular reasoning in transformers","Modular reasoning skills transfer across transformer models","Different pretraining rules build distinct transformer skill modules","Attention and MLP hold separate skills from procedural pretraining","Composable transformer skills from procedural pretraining"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a component's benefit after fine-tuning tells you where the pretrained structure actually lives, since the whole model is fine-tuned after partial transfer and a component could help merely by making optimization easier rather than by storing the skill.","fun_headline_variants_meta":{"raw":{"variants":["Procedural pretraining installs modular reasoning in transformers","Modular reasoning skills transfer across transformer models","Different pretraining rules build distinct transformer skill modules","Attention and MLP hold separate skills from procedural pretraining","Composable transformer skills from procedural pretraining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1374,"prompt_tokens":953,"completion_tokens":421,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":569,"tokens_out":421,"duration_ms":4898,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:10:16.048204+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the composition experiment of Section 5 with the ECA MLP block replaced by randomly initialized MLP weights of the same scale, keeping the SET attention block; if this control matches the composed model's accuracy on all four diagnostic tasks, the claimed complementarity of the two pretraining rules collapses.","supporting_citations":[],"review_version":1}