{"id":"73a0ac2a-4253-4e7a-a362-854a9da85837","arxiv_id":"2411.12580","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Influence traces show LLM reasoning on simple math is driven by documents that demonstrate procedures, not by documents containing the answers, suggesting a generalization strategy unlike retrieval.","lead":"This paper traces which pretraining documents actually drive language model outputs on simple math problems, and finds that the answers themselves are rarely influential while documents showing similar procedures are. If right, it suggests LLMs reason by synthesizing procedural knowledge rather than by retrieving memorized answers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Influence is computed over the full chain-of-thought completion, so answer-token influence can be diluted by generic procedural tokens; the finding that answer documents are rarely top-ranked may be an artifact of full-sequence aggregation rather than evidence against retrieval.","rationale":"The paper is a serious empirical effort: the counterfactual re-training experiments in Appendix A.1 validate the EK-FAC pipeline on downstream accuracy, and the correlation with the public implementation is a good reproducibility check. The central claim, however, requires that the influence rankings over pretraining documents for reasoning queries can distinguish retrieval of answers from use of procedural knowledge. The scoring function used (Equation 1 on the full completion) makes this distinction unreliable. Because a chain-of-thought completion is long and mostly generic, the aggregate influence score is dominated by documents that match the generic step-by-step phrasing; a document that specifically supplies the final answer contributes only to one or two tokens and is unlikely to rank in the top 0.01% regardless of whether the model retrieved it. The paper's Finding 3—the key evidence against retrieval—is therefore confounded by completion length and lexical structure. Finding 1 is similarly affected because same-task completions share a template. The control queries are a useful check but do not resolve this, since they do not isolate token-level influence. A token-level influence analysis (restricting f to the answer span) would settle whether the observed absence of answer documents in the top rankings is real or an artifact. If the answer documents become influential at the token level, the paper's conclusion that reasoning is 'unlike retrieval' would need to be substantially revised. Until this test is run, the evidence is insufficient to support the strong version of the central claim, so conditional acceptance is appropriate.","tokens_in":41272,"tokens_out":9794,"duration_ms":105370,"concrete_test":"For each reasoning query, compute the EK-FAC influence score (same pipeline) on the log-likelihood of only the final answer token (or the span containing the numeric answer) instead of the full completion, and rank the 5M documents by that token-level score. Then check whether documents containing the exact answer appear in the top 500 for a substantial fraction of reasoning queries. Also run the same token-level ranking on factual queries as a sanity check, where the answer span is already known to be retrieved. If answer documents become top-ranked for reasoning under token-level influence, Finding 3 is an artifact of full-sequence aggregation and the paper's central claim is not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that reasoning is 'unlike retrieval' rests heavily on Finding 3: answers to reasoning questions almost never appear in the top 0.01% of influential pretraining documents, while factual answers often do (Section 5.2). But the influence score defined in Equation 1 is computed on f = log p(yc|yp) for the full completion, and Section 4 states 'We calculate influence over the full completion, which includes the chain-of-thought.' This makes the score a sum over per-token log-likelihoods. For a reasoning completion, which is long (often 50+ tokens) and dominated by generic step-by-step phrasing, a document that mildly boosts many generic tokens (e.g., a similar worked example) accumulates a large total influence, while a document that strongly boosts only the final answer token (e.g., the exact answer) may appear far down the ranking. The per-nat normalization in Section 3 is a scalar and does not reweight token contributions. Thus Finding 3 does not establish that the model is not retrieving the answer; it only shows the answer document is not the dominant influence on the whole sequence. The comparison to factual queries is confounded because factual completions are short and answer-heavy. The same aggregation issue partly affects Finding 1: same-task reasoning completions share a template, so correlated document influences could reflect shared phrasing rather than shared procedures. Without token-level influence rankings, the evidence does not distinguish retrieval of answers from procedural synthesis.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how two Cohere Command R models (7B and 35B) use pretraining data when producing chain-of-thought reasoning traces for three simple mathematical tasks, compared with factual question answering. The authors use EK-FAC influence functions to rank 5 million sampled pretraining documents (2.5B tokens) by their estimated influence on the log-likelihood of each query completion. They report four main findings: (1) influence scores for queries sharing a reasoning task are positively correlated, suggesting shared procedural knowledge; (2) reasoning queries rely on less individual per-nat influence than factual queries; (3) answers to reasoning questions rarely appear in the top 0.01% of influential documents, unlike factual answers; and (4) procedural documents, especially code, are overrepresented among influential documents for reasoning. The paper concludes that reasoning is 'unlike retrieval' and instead synthesizes procedural knowledge from documents demonstrating similar reasoning. The appendix contains counterfactual retraining experiments validating influence functions on accuracy, detailed approximation justifications, control-query analyses, and extensive qualitative results.","tokens_in":41572,"tokens_out":2950,"duration_ms":34173,"significance":"If the central claim holds, this is a valuable contribution to the interpretability and data-selection literatures: it moves beyond contamination-based explanations and suggests that pretraining data demonstrating procedures, rather than containing exact answers, drives reasoning generalization. The paper is methodologically ambitious, applying influence functions at 7B/35B scale, and it ships unusually strong supporting evidence: counterfactual retraining experiments (Appendix A.1) show that influence-based document removal degrades accuracy more than random or TracIn baselines; the authors verify their internal influence implementation against the public kronfluence code with Pearson R > 0.99; and they provide queries, completions, keywords, and prompts as supplementary material. These strengths make the empirical observations credible. However, the central interpretation rests on several load-bearing approximations and on inference from full-sequence influence rankings, so the evidence as presented is not yet conclusive enough for the broad 'unlike retrieval' conclusion.","major_comments":[{"comment":"The claim that reasoning is 'unlike retrieval' rests heavily on Finding 3, but the influence score in Eq. (1) is computed on the full completion, which includes the chain-of-thought. As stated in Section 4, 'we calculate influence over the full completion.' For a long reasoning trace dominated by generic step-by-step phrasing, a document that mildly boosts many generic tokens can accumulate high total influence, while a document that strongly boosts only the final answer token may rank far lower. The per-nat normalization described in Section 3 is a scalar and does not reweight individual token contributions. Thus Finding 3 does not establish that the model is not retrieving the answer; it only shows that answer documents are not dominant influences on the whole sequence. The comparison to factual queries is confounded because factual completions are short and answer-heavy. I recommend token-level influence rankings, or at least a supplementary analysis restricting influence to the final answer token, to test whether the answer document rises in the ranking.","section":"Section 4 and Eq. (1)"},{"comment":"The experimental setup computes document gradients and EK-FAC Hessians with the base models, while query gradients and completions come from the supervised fine-tuned models, 'assuming the EK-FAC for the fine-tuning phase is the identity' (Section 4). This is a strong assumption: if SFT substantially changes the relevant parameters, the influence scores do not correspond to a well-defined counterfactual for the model that actually produced the completions. Appendix A.7 lists the assumption but does not validate it for the 7B and 35B models used here. Given that the central claim is about how the final models reason, this gap needs to be addressed, for example by comparing rankings computed from base-model query gradients with those from SFT-model query gradients on a small subset, or by citing direct validation evidence that SFT shifts are negligible for these specific models.","section":"Section 4 (Models) and Appendix A.7"},{"comment":"The correlation analysis is presented as evidence for procedural knowledge, but the appendix shows that formatting and superficial similarity can explain a substantial portion of the effect. For example, the reasoning control queries, which require no reasoning but share formatting, sometimes achieve within-group correlations up to 0.38 (Appendix A.9.1), and for the 35B linear equations only 34 of 190 within-task correlations exceed the maximum correlation with control queries. The paper acknowledges part of the correlation is driven by formatting, but the quantitative decomposition is not provided. Without controlling for completion template and document-gradient overlap, Finding 1 is weaker evidence for shared procedures than the abstract claims suggest. I recommend reporting correlations on token-level or template-normalized influences, or explicitly quantifying how much of the same-task correlation survives when generic reasoning tokens are excluded.","section":"Finding 1 and Appendix A.9.1"},{"comment":"The influence calculations only cover MLP parameters because EK-FAC is not properly defined for attention layers. The paper acknowledges this in Appendix A.7 and in the discussion, but the central claim is phrased broadly as 'the approach to reasoning the models use.' If procedural reasoning is substantially implemented in attention layers, the rankings could miss the true influential documents. This is a load-bearing scope limitation for the 'unlike retrieval' conclusion. I suggest either restricting the claims to knowledge stored in MLP parameters or adding evidence, such as attention-head attribution or ablation studies, that attention-layer contributions do not change the qualitative findings.","section":"Appendix A.7"}],"minor_comments":[{"comment":"There is a typo: 'more accuractely' should be 'more accurately.'","section":"Section 3"},{"comment":"The phrase 'distributed similary as the training distribution' should read 'distributed similarly to the training distribution.'","section":"Section 4"},{"comment":"In the PEMDAS example, 'the perform the operations' should be 'they perform the operations.'","section":"Appendix A.8.1"},{"comment":"The power-law slope comparison in Table 28 is reported with significance stars, but the discussion in Appendix A.9.3 notes that the steepest 35B slope may be noise and that the top document is unrelated. Please clarify whether the reported slope differences are robust to excluding this query, since the current presentation is potentially misleading.","section":"Section 5.1 and Appendix A.9.3"},{"comment":"The figure is dense and the redacted document snippets are hard to read; consider enlarging the document text or splitting the figure into two panels with clearer annotations.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on Grosse et al. (2023) and Bae et al. (2024), both with overlapping authors, but the public kronfluence implementation and the reported R > 0.99 correlation with their internal code provide reasonable independence on the computational side. My main concern is scope: the central 'unlike retrieval' claim is stated more strongly than the full-sequence aggregation and MLP-only influence analyses can support. With targeted token-level or answer-token analyses, the paper could either confirm or appropriately qualify the claim. I do not see grounds for rejection; the empirical work is careful and reproducible in the ways that matter."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should read this one, but keep the stress-test concern in mind while you do. The paper applies EK-FAC influence functions to trace which pretraining documents drive reasoning completions in two proprietary models (7B and 35B), and it finds something genuinely new: for simple math tasks, documents that demonstrate the procedure are top-ranked, while documents containing the exact answer or intermediate steps rarely are. The correlation across same-task queries is a solid result, backed by control queries that superficially match the reasoning prompts but don't require reasoning. The counterfactual retraining experiments in Appendix A.1 also give me real confidence that the influence scores track causal effects, not just gradient similarity.\n\nThe soft spot is the one the stress-test flags. Influence is computed on the full completion, including the chain-of-thought, so the score is a sum over per-token log-likelihoods. A long reasoning trace is dominated by generic procedural phrasing, so a document that mildly boosts many generic tokens can outrank a document that strongly boosts only the final answer. That means Finding 3 (answers almost never show up as top-ranked) is not strong evidence against retrieval. The factual comparison is confounded because factual completions are short and answer-heavy. The same aggregation issue could inflate Finding 1: correlations across same-task queries might reflect shared template wording rather than shared procedure, though the control queries and the qualitative look at top documents partly mitigate this.\n\nOther limitations are what you'd expect from this kind of work: proprietary data, MLP-only influence, identity SFT assumption, hand-selected tasks, and a 2.5B-token sample. The authors acknowledge these clearly, and the public kronfluence code correlation (Pearson R > 0.99) is a genuine reproducibility effort.\n\nWho is this for? Anyone working on data attribution, contamination, or whether LLMs reason vs. retrieve. It's not the last word, but it is a serious empirical contribution that deserves referee time. The central claim—that reasoning synthesizes procedural knowledge rather than retrieving memorized answers—is plausible but not proven, and the paper would benefit from token-level influence analysis or a direct test that removes the aggregation confound. I'd send it out.","headline":"Careful influence-function study with a real finding, but the 'not retrieval' claim is weaker than it looks because influence is aggregated over the whole chain-of-thought.","tokens_in":42123,"tokens_out":1064,"would_cite":true,"duration_ms":13892,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On math reasoning tasks, large language models rely on pretraining documents that demonstrate the procedure, not documents that contain the answer.","keywords":["influence functions","procedural knowledge","language model reasoning","pretraining data attribution","chain-of-thought","mathematical reasoning","code in pretraining","data selection"],"falsifier":"Retrain a 7B model on a pretraining corpus from which the top-ranked procedural documents, such as code that computes slope, are removed while answer-containing documents are retained; if accuracy on unseen reasoning queries does not drop, the claimed causal role of procedural documents fails.","tokens_in":41051,"feed_emoji":"🧮","tokens_out":5869,"duration_ms":55177,"temperature":0.7,"pith_summary":"The paper asks whether large language models solve reasoning tasks by retrieving memorised answers from pretraining or by applying a generalisable procedure. To answer it, the authors compute influence-function rankings over 5 million pretraining documents for 80 factual and reasoning queries answered by two models of different sizes (7B and 35B). They find that a document's influence on one reasoning query strongly predicts its influence on another query of the same mathematical task, which is not true for factual queries. The exact answers to reasoning questions almost never appear among the most influential documents, while answers to factual questions often do. The authors conclude that the models synthesise procedural knowledge from documents that demonstrate the same kind of reasoning, rather than retrieving answers.","feed_headline":"Math answers rarely drive LLM reasoning; procedures do","feed_subtitle":"Ranking 5M pretraining documents shows code and worked examples drive reasoning; exact answers rarely do.","key_machinery":"The central machinery is the EK-FAC influence function, an approximate counterfactual attribution that estimates how much removing a pretraining document would change the log-probability of a completion, per nat of completion information. The paper applies it to rank 5 million documents for each query, using a Hessian estimated from a 100,000-document sample and query gradients compressed by approximate SVD; only MLP parameters are used, with the instruction-tuning stage treated as identity. The rankings are then compared across queries with Pearson correlations, searched for literal answers, inspected qualitatively for procedural content, and analysed by source dataset.","core_discovery":"On the paper's own terms, the discovery is that LLM reasoning on simple mathematical tasks is driven by procedural knowledge in pretraining documents: documents that show how to perform the relevant calculation, in formula or code, have similar influence across different queries requiring the same procedure, while documents containing the particular answer or intermediate steps are not highly influential. The paper contrasts this with factual question answering, where the answer itself regularly surfaces at the top of the influence ranking. Together with lower and less volatile per-document influence for reasoning, the authors interpret this as evidence for a generalisation strategy that combines procedural knowledge from many documents, unlike retrieval. They also report that code data is overrepresented among the most influential documents for reasoning, with StackExchange and mathematics sources appearing far more often than their pretraining share would predict.","pith_inferences":["If this pattern generalises, reasoning benchmark contamination may be subtler than exact-answer overlap: a model could score well after seeing worked procedures or code patterns during pretraining even when the test question's answer never appears in training data.","A direct testable extension would be to pretrain two models on corpora that differ only in the presence of procedure-demonstrating documents, then measure transfer to unseen numbers; the paper's account predicts a clear accuracy gap.","Because the influence scores come only from MLP parameters, the actual arithmetic may be performed in attention layers while the feed-forward influence rankings surface the documents that supply the procedure; the paper's correlation patterns are consistent with that division of labour.","The near-zero correlation between the influence rankings of the 7B and 35B models suggests that procedural knowledge is instantiated in different documents for different model sizes, so data-selection conclusions may not transfer across scales without re-measuring."],"forward_implications":["Pretraining data selection for reasoning does not need to cover every numerical case; documents that demonstrate procedures across diverse inputs should carry more weight.","Code and mathematics-heavy sources are disproportionately influential for mathematical reasoning, suggesting code data is a particularly valuable procedural training signal.","Factual question answering remains more retrieval-like: answers show up in top influential documents in 30 to 55 percent of cases, so contamination concerns are more acute for fact recall than for the simple reasoning tasks studied.","Larger models show an even stronger separation: the 35B model never had a reasoning answer among its top influential documents and relied on each individual document even less.","The same procedural documents can explain many different queries, with correlations in influence scores reaching 0.9 for queries that share a procedure, indicating a shared source of generalisable knowledge."],"supporting_citations":[{"why":"Supplies the EK-FAC influence-function method and the scaling approximations that make document attribution tractable on large transformers.","marker":"Grosse et al. (2023)"},{"why":"Introduces influence functions for estimating the counterfactual effect of training examples on model predictions, the basis of the paper's attribution.","marker":"Koh & Liang, 2017"},{"why":"Justifies treating the supervised fine-tuning Hessian as identity, a load-bearing assumption in attributing influence to pretraining documents.","marker":"Bae et al., 2024"},{"why":"Provides the approximate SVD algorithm used to compress query gradients so that scores for 5 million documents can be computed in one pass.","marker":"Halko et al., 2011"},{"why":"Motivates dividing influence scores by the document gradient norm to filter noise in the top-ranked qualitative analyses.","marker":"Choe et al., 2024"},{"why":"Provides the chain-of-thought prompting method used to elicit the reasoning traces whose influence the paper studies.","marker":"Wei et al., 2022"},{"why":"Supplies prior evidence on the role of code in pretraining, which the paper's code-overrepresentation finding connects to.","marker":"Aryabumi et al., 2024"}],"fun_headline_variants":["LLM reasoning runs on procedural know-how, not answer retrieval","Why LLMs solve problems: they learn procedures, not answers","Code and worked examples, not answers, drive LLM math reasoning","LLM math skills stem from procedural data, not from memorized answers","LLM reasoning uses procedural knowledge, not answer lookup"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis depends on the assumption that influence scores computed from MLP parameters alone, with the instruction-tuning stage treated as identity, identify the documents that actually drive the reasoning; if attention layers or fine-tuning carry the reasoning, the rankings could miss the true drivers.","fun_headline_variants_meta":{"raw":{"variants":["LLM reasoning runs on procedural know-how, not answer retrieval","Why LLMs solve problems: they learn procedures, not answers","Code and worked examples, not answers, drive LLM math reasoning","LLM math skills stem from procedural data, not from memorized answers","LLM reasoning uses procedural knowledge, not answer lookup"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000365,"raw_usage":{"total_tokens":1999,"prompt_tokens":1012,"completion_tokens":987,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":628,"completion_tokens_details":{"reasoning_tokens":900}},"tokens_in":628,"tokens_out":987,"duration_ms":8632,"temperature":1.0,"reasoning_tokens":900,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:21:25.420012+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain a 7B model on a pretraining corpus from which the top-ranked procedural documents, such as code that computes slope, are removed while answer-containing documents are retained; if accuracy on unseen reasoning queries does not drop, the claimed causal role of procedural documents fails.","supporting_citations":[],"review_version":1}