{"id":"88309aab-7a10-4e07-a9a9-65f4b1bf20da","arxiv_id":"2506.22977","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Ortu et al.'s main claims reproduce, but the attention-head ablation fails on underrepresented domains and varies with model, prompt, and task.","lead":"This reproduction study confirms the original 'competition of mechanisms' findings on GPT-2 and Pythia, but shows the proposed attention-head ablation works poorly for prompt domains underrepresented in the original data. The results suggest interpretability claims about which attention heads matter may depend heavily on dataset composition, prompt wording, and model architecture.","discovery_kind":"replication","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The architecture-dependence component of the central claim is unsupported because the paper itself leaves open (Section 4.2) that the reduced head specialization observed in Llama 3.1 8B is an artifact of the logit lens method, yet the abstract presents it as an established finding.","rationale":"The reader's weakest assumption was the faithfulness of logit lens and attention attribution. I agree, and I narrow it to the architecture-dependence component, which is the part of the central claim most directly poisoned by this assumption. The prompt-structure and domain results are behavioral outcome measures (next-token prediction) and do not require accurate internal attribution for their headline numbers, even though the head selection depends on attribution. The Llama result, by contrast, is fundamentally an attribution claim: 'reduced attention head specialization' is a statement about the internal distribution of contributions. The paper itself cannot distinguish this from method failure, and the abstract overstates the finding. The fix is a methodologically independent attribution check. This is not a disagreement with consensus; it is an internal inconsistency between the hedged results and the categorical conclusion. The paper is otherwise honest and reproducible, with transparent limitations, so the appropriate verdict remains conditional pending this test.","tokens_in":19245,"tokens_out":9312,"duration_ms":84726,"concrete_test":"Use activation patching (path patching, Conmy et al., 2023) on Llama 3.1 8B over the filtered 'Redefine' prompt set to identify attention heads whose intervention changes the factual/counterfactual prediction, without relying on logit lens. If patching finds a small set of decisive heads, the 'greatly reduced specialization' conclusion is an artifact of logit lens and the architecture-dependence part of the central claim is unsupported; if patching shows distributed contributions across many heads, the claim is supported. A secondary check: apply the same patching to GPT-2 small as a control to confirm the method recovers L10H7/L11H10.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim asserts that the effectiveness of the Ortu et al. (2024) attention-head ablation varies with model architecture. The only evidence for the architecture dimension is the Llama 3.1 8B result, where the paper reports 'greatly reduced attention head specialization.' Yet Section 4.2 explicitly states the result is ambiguous: 'Either the logit lens method fails to discover the highly specialized attention heads, or they are simply not as concentrated for Llama 3.1 8B.' The paper also notes that the logit lens is documented as unreliable for larger models (Section 4.2, citing Belrose et al., 2023). The ablation experiments do not resolve this ambiguity: the heads selected for ablation (L27H20, L28H15, L31H14) were chosen using the same logit-lens attribution. If the logit lens misattributes contributions for Llama, the ablation could appear ineffective even if a small set of decisive heads exists. The body is appropriately hedged, but the abstract and the paper's stated strongest claim drop the hedge, presenting the reduced specialization as an established finding. This is a load-bearing gap because the architecture-dependent-effectiveness conclusion depends on disambiguating a methodological artifact from a real architectural difference, and the paper does not do so.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a reproduction and extension study of Ortu et al. (2024), which analyzed competition between factual recall and counterfactual copy mechanisms in language models. The authors re-filter the original COUNTERFACT-derived datasets, reproduce the original positional-encoding, attention-block-dominance, and attention-head-specialization findings on GPT-2 small and Pythia 6.9B, and extend the analysis to Llama 3.1 8B, to question-answer prompt reformulations, to alternative premise words, and to domain-stratified subsets. The paper concludes that the proposed attention-head ablation is less effective for underrepresented domains and that its effectiveness varies with architecture, prompt structure, domain, and task.","tokens_in":19575,"tokens_out":6058,"duration_ms":63456,"significance":"If the results are taken at face value, the paper makes a useful contribution to the reproducibility literature in mechanistic interpretability and to the practical question of how widely head-level circuit claims transfer. Its strengths include a careful re-filtering of the datasets, a public code repository, the reproduction of the core GPT-2 and Pythia results, and the domain-leakage observation that many COUNTERFACT prompts contain the factual token inside the subject, which can spuriously explain subject-position importance. The main weakness is that the central architecture-dependence claim rests on Llama 3.1 8B results that the body itself explicitly labels as ambiguous, and the abstract overstates this result and the domain result.","major_comments":[{"comment":"The abstract states that Llama 3.1 8B shows 'greatly reduced attention head specialization' as an established finding, but the body of Section 4.2 explicitly leaves open two possibilities: 'Either the logit lens method fails to discover the highly specialized attention heads, or they are simply not as concentrated for Llama 3.1 8B.' The ablation experiments do not resolve this ambiguity because the ablated heads were selected using the same logit-lens attribution that the paper acknowledges may be unreliable for larger models (citing Belrose et al., 2023), and the resulting increase in factual predictions was only from 15% to 22%. This is load-bearing for the architecture-dependence component of the central claim; the abstract should be reworded to present the finding as provisional, or the authors should provide disambiguating evidence (e.g., tuned-lens attribution, a random-head ablation baseline, or a broader head search).","section":"Section 4.2 / Abstract"},{"comment":"The abstract and the introduction claim successful reproduction of the original paper's 'localization of factual and counterfactual information,' but Section 4.1 says the opposite for a key part of that claim: 'the claim that factual information is stored in the subject position does not seem to be backed up by our results or those of the original paper.' Since Section 2 lists 'Positional Information Encoding: factual attributes appearing in subject positions' as one of the core reproduced claims, the paper is internally inconsistent about what was reproduced. Please re-scope the reproduction claim in the abstract and Section 2, and clearly separate the confirmed parts (counterfactual in attribute position, monotonic increase with layers) from the unconfirmed part (factual in subject position).","section":"Section 4.1 vs. Abstract and Section 2"},{"comment":"The abstract's claim that the Ortu et al. (2024) ablation is 'ineffective for domains that are underrepresented in their dataset' is stronger than the evidence supports. Figure 4 reports only post-ablation counts per domain, with no per-domain baseline, so 'ineffective' (no effect) cannot be distinguished from 'effective but insufficient to flip the majority prediction.' The body itself uses the weaker phrase 'making ablation less effective.' Additionally, the MQuAKE results (Figure 5) rest on only 228 filtered prompts, and the paper acknowledges they are tentative. Please either add per-domain baseline-vs-ablation comparisons and confidence intervals, or revise the abstract and Section 5.2 to state the finding more cautiously, e.g., 'the ablation does not transfer to underrepresented domains in these datasets.'","section":"Section 4.5 / Abstract"}],"minor_comments":[{"comment":"The phrase 'somewhat arbitrarily decided' is informal for a journal article; consider describing more precisely how the original study selected the heads and scaling factor, and note that this reproduction follows that selection by design.","section":"Section 3.3.2"},{"comment":"The domain-level win counts are difficult to read because the category names are crowded and the baseline counts are not shown; a table with per-domain baseline and post-ablation factual/counterfactual counts would make the domain claim easier to verify.","section":"Figures 4 and 5"},{"comment":"The statement that the QnA prompt structure is 'significantly more important' than the premise word is based only on GPT-2 small experiments; please state this scope explicitly in the same paragraph and in the conclusion to avoid overgeneralization.","section":"Section 4.3"},{"comment":"The differences across premise words are reported as counts and percentages without any uncertainty quantification; although the counts are deterministic on the fixed dataset, reporting bootstrap intervals or a simple chi-square test would strengthen the comparison.","section":"Section 4.4 / Table 1"},{"comment":"When reporting that only 5,179 of 10,000 GPT-2 small prompts and 9,137 of 10,000 Pythia prompts have the factual token as the top prediction, please specify the decoding criterion used (e.g., greedy top-1 vs. highest logit) and how the original dataset was obtained, so that this verification is fully reproducible.","section":"Section 3.2.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest in its limitations section and the body is generally well hedged, but the abstract and the title-level claim ('effectiveness varies based on model architecture') are not supported by the body without additional disambiguation. The central reproduction results for GPT-2 and Pythia appear solid, and the domain-leakage finding is a genuine contribution. I would encourage the authors to revise the abstract and either add a small number of targeted experiments (e.g., tuned-lens on Llama, random-head ablation baseline, or per-domain baseline counts) or explicitly reframe the central claim as a set of hypotheses for future work. The current mismatch between the abstract and the body is the main reason for major revision rather than minor revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: worth reading for the domain analysis and the careful reproduction; the architecture-generalization claim in the abstract goes beyond what the evidence supports.\n\nWhat's new: the token-leakage observation — that in Autos and Computers prompts the subject often contains the factual answer (e.g. \"Nintendo DS Lite\") — is a real contribution. It explains the subject-position finding in the original paper as a dataset artifact, and it predicts where the ablation fails. That is testable and useful. The QnA and premise-word results are also solid as prompt-sensitivity evidence, even if not deep.\n\nThe reproduction itself is careful: they re-filtered the datasets instead of trusting the original claims, found that thousands of original prompts did not actually have the factual token as top prediction, and verified the headline results on the clean data. Code is public. That is how reproduction should be done.\n\nSoft spots, in order. The Llama conclusion is the weakest link. The body says, plainly, that the reduced head specialization could be a logit-lens artifact for larger models, and that the ablation only moved factual share from 15% to 22%. The abstract, though, presents \"greatly reduced attention head specialization\" as an established fact. That is a load-bearing mismatch: the architecture-dependence component of the central claim rests entirely on this ambiguous result. The stress-test note is right.\n\nThen: MQuAKE is 228 prompts, the authors say so themselves, and they still include it. That is acceptable as a tentative extension but not as evidence for domain generalizations. The domain and premise analyses have no error bars or significance tests; the premise table differences look real (6.3% vs 1.9% factual), but for the ablation percentages, with counts in the hundreds, we need uncertainty. Also, most extensions are GPT-2 small only, which limits how far the \"effectiveness varies by task and domain\" claim can travel.\n\nThe citation pattern is fine; using the same heads and alpha as the original is a consistency choice, not circularity. The paper also correctly self-identifies many limitations in Section 5.4, which I trust.\n\nWho it is for: anyone doing mechanistic interpretability on factual recall or attention head ablation. The domain leakage result alone is worth a cite.\n\nMy recommendation: send it to review. It deserves referee time, and the referee should push for an abstract rewrite and, if possible, a tuned-lens check on Llama before publication. If the authors fix the overreach, this is a useful paper.","headline":"Reproduction is solid and the domain-leakage finding is important, but the abstract overclaims the Llama result that the body itself leaves unresolved.","tokens_in":20036,"tokens_out":2141,"would_cite":true,"duration_ms":21271,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This reproduction confirms the core 'competition of mechanisms' findings for GPT-2 and Pythia, but shows the recommended attention-head ablation does not generalize across architectures, prompts, or domains.","keywords":["mechanistic interpretability","attention head ablation","counterfactual reasoning","logit lens","competition of mechanisms","reproduction study","dataset domain bias","language model generalization"],"falsifier":"Take the original and domain-split datasets, remove every prompt whose subject token contains the factual token, and re-run the L10H7/L11H10 ablation on the remaining underrepresented-domain prompts: if factual predictions still rise to roughly 50%, the paper's domain-dependence conclusion is wrong. For the Llama result, replace the logit-lens attribution with activation patching or a tuned lens: if a small set of decisive heads appears, the paper's claim of reduced specialization is an artifact of the measurement tool.","tokens_in":19055,"feed_emoji":"🤖","tokens_out":8162,"duration_ms":74163,"temperature":0.7,"pith_summary":"This paper asks how far the 'competition of mechanisms' explanation of how language models handle facts versus counterfactuals actually extends. It independently reproduces the original study's three core findings—that factual and counterfactual information occupy different token positions, that attention blocks dominate the competition, and that a few specialized attention heads decide the outcome—on GPT-2 small and Pythia 6.9B. It then extends the setup to Llama 3.1 8B, question-format prompts, different premise words, and a domain-classified version of the datasets, and finds each change matters: Llama shows much weaker attention-head specialization, the question format brings GPT-2 close to a fifty-fifty split, and the recommended head ablation is effective mainly in the automotive and technology categories that dominate the original data. The paper's central claim is that the ablation's effectiveness is conditional on model architecture, prompt structure, domain, and task, so the original mechanism-competition picture does not generalize as stated.","feed_headline":"Attention-head ablation fails on underrepresented LLM domains","feed_subtitle":"The recommended head-level fix only works on the dominant categories; newer models show less specialization.","key_machinery":"The paper's working machinery is the logit-lens inspection of residual-stream position-by-position logits, combined with attention-head ablation that scales the attention of the last token to the attribute position (heads L10H7 and L11H10 in GPT-2 small, L17H28/L20H18/L21H8 in Pythia 6.9B) by a factor α. That combination exposes which head writes the counterfactual token into the final position; the paper runs it across models, prompt structures, premise words, and domains to test whether the same heads remain decisive. The central measured quantity is the factual-over-counterfactual margin Δ_cofa, which the ablation is meant to flip.","core_discovery":"On its own terms, the paper's discovery is that the mechanism-competition story from the original study reproduces cleanly only under the original conditions. On GPT-2 small and Pythia 6.9B with the 'Redefine:' prompt, the paper confirms the positional encoding of factual and counterfactual tokens, the stronger role of attention blocks than MLP blocks, and the existence of a few high-impact heads that promote the counterfactual by suppressing it rather than by boosting the factual token. On Llama 3.1 8B, competition between the two mechanisms still appears, but the head-level specialization largely disappears: only one strong counterfactual head was found, factual contributions are spread widely, and even the best ablation moves factual predictions only from 15% to 22%. Recasting the prompt as a question reduces the copy tendency so much that GPT-2 predicts factual and counterfactual tokens almost equally often, and the choice of premise word (for example 'Review' versus 'Validate') shifts factual-prediction rates from 1.9% to 6.4%, with the gap widening under ablation. Finally, the paper shows that the apparent importance of the subject token for factual recall is inflated by the original dataset's concentration on product domains whose names contain the factual token; in underrepresented domains, the proposed head ablation no longer flips predictions, and on the smaller cross-domain dataset it can even increase counterfactual predictions.","pith_inferences":["Editorial inference: if the subject-token leakage explanation is right, re-running the same ablation on a leakage-filtered COUNTERFACT subset should restore or abolish the domain effect, giving a direct causal test of the paper's explanation.","Editorial inference: the near-equivalence of prompt restructuring and head ablation suggests that for practical safety interventions, sanitizing or reformulating prompts may be cheaper and more reliable than circuit-level edits, a direction the paper's practical discussion implies but does not state as a recommendation.","Editorial inference: for Llama-like models, the failure to find specialized heads could be an artifact of the logit-lens method rather than absent specialization; applying a tuned lens or activation patching would settle which reading is correct, and the paper's negative result on Llama remains its least settled finding."],"forward_implications":["In small models, the question-format prompt alone brings GPT-2's factual and counterfactual predictions near a fifty-fifty split, making prompt restructuring as effective as the original head ablation; combining the two is far stronger.","The recommended GPT-2 heads L10H7 and L11H10 do not transfer across domains: under the same ablation, underrepresented categories mostly still predict the counterfactual token, and the MQuAKE-based test shows weakened or even reversed effects.","Larger and newer models like Llama 3.1 8B exhibit the factual and counterfactual competition but without concentrated head specialization, so the original method does not identify the right levers for attention-head control in those models.","Because the original dataset's dominant domains contain prompts where the subject itself contains the factual token, the subject-position encoding that the original study emphasized is partly an artifact of dataset bias."],"supporting_citations":[{"why":"The study being reproduced; supplies the mechanism-competition claims, the 'Redefine' prompt setup, and the specific attention heads the paper tests.","marker":"Ortu et al., 2024"},{"why":"Supplies the COUNTERFACT dataset of factual and counterfactual prompts from which the original study's datasets derive.","marker":"Meng et al., 2022"},{"why":"Defines GPT-2 small, one of the two main reproduction models.","marker":"Radford et al., 2019"},{"why":"Defines Pythia 6.9B, the other main reproduction model.","marker":"Biderman et al., 2023"},{"why":"Defines Llama 3.1 8B, the extension target where attention-head specialization is found greatly reduced.","marker":"Grattafiori et al., 2024"},{"why":"Provides the TransformerLens library that implements the logit-lens inspection and attention-modification methods used throughout.","marker":"Nanda & Bloom, 2022"},{"why":"Introduces the logit-lens method for reading predictions from intermediate residual-stream states.","marker":"nostalgebraist, 2020"},{"why":"Supplies the MQuAKE dataset used for the cross-domain robustness extension.","marker":"Zhong et al., 2023"},{"why":"Cited as the source of the tuned lens, the alternative interpretability method that may be more reliable for larger models and that bears on how to read the Llama results.","marker":"Belrose et al., 2023"},{"why":"Provides the T5 question-generation model used to build the QnA prompt variants.","marker":"Romero, 2021"}],"fun_headline_variants":["Head ablation flops on rare LLM domains","LLM fact fix only works on dominant data","Counterfactual head trick breaks on new domains","Head ablation unreliable across models and prompts","Larger LLMs lose head specialization for facts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the logit-lens and attention-attribution tools used in the original pipeline actually reveal the internal causes of the final prediction; if they misattribute contributions, especially for Llama 3.1 8B, both the reproduced findings and the new negative results are compromised.","fun_headline_variants_meta":{"raw":{"variants":["Head ablation flops on rare LLM domains","LLM fact fix only works on dominant data","Counterfactual head trick breaks on new domains","Head ablation unreliable across models and prompts","Larger LLMs lose head specialization for facts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1557,"prompt_tokens":1133,"completion_tokens":424,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":749,"completion_tokens_details":{"reasoning_tokens":355}},"tokens_in":749,"tokens_out":424,"duration_ms":5201,"temperature":1.0,"reasoning_tokens":355,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:52:58.210172+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the original and domain-split datasets, remove every prompt whose subject token contains the factual token, and re-run the L10H7/L11H10 ablation on the remaining underrepresented-domain prompts: if factual predictions still rise to roughly 50%, the paper's domain-dependence conclusion is wrong. For the Llama result, replace the logit-lens attribution with activation patching or a tuned lens: if a small set of decisive heads appears, the paper's claim of reduced specialization is an artifact of the measurement tool.","supporting_citations":[],"review_version":1}