{"id":"4abb846c-69ff-4ad4-bf07-c25ebd0854da","arxiv_id":"2412.15628","paper_version":5,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Using synthetic inductive reasoning tasks with a single 'aha' example, the paper shows simple gradient-norm attribution often beats integrated gradients for identifying the crucial example, while interpretability worsens in larger models.","lead":"This paper tests whether standard input attribution methods can identify the one example that disambiguates a rule in few-shot in-context learning. It finds that simple gradient norm beats more complex gradient methods, and that larger models are generally harder to interpret with gradients.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core benchmark validity depends on the assumption that models perform rule induction by comparing the aha example with another example; the paper's own control results show many correct answers survive removal of the aha example, so a large fraction of the reported attribution scores may be…","rationale":"The reader's weakest_assumption identifies the general risk that the aha example may not be the unique locus of reasoning. I agree, and I believe the paper's own Table 2 gives concrete evidence that this risk is realized: on several tasks/models, task accuracy without the aha example is near or at ceiling, so a majority of correct-answer instances may not depend on e* at all. That makes the top-1/top-2 metric partially ill-posed for those items. However, this does not invalidate the paper's exploratory contribution: the benchmark design, the honest discussion of inductive bias, and the qualitative observations about scaling are useful. The central quantitative ranking of gradient-based methods is what needs re-analysis on a causal subset. I do not see a reason to move to REJECT, because the concern is testable with existing machinery (the IE erasure already implements the necessary intervention) and the paper's qualitative findings are likely to survive in modified form. Hence CONDITIONAL, matching the reader's verdict.","tokens_in":20997,"tokens_out":1591,"duration_ms":14748,"concrete_test":"Re-run the main 10-shot evaluation (Figure 2) restricted to prompts where removing the aha example changes the model's prediction from correct to incorrect, using the same IE-elimination setup as the paper's IE method. If GN's top-2 accuracy on this causal subset does not remain clearly above chance and above IG, then the main claim that simple GN is the best gradient-based method is an artifact of including aha-independent prompts.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim is that simple gradient norm (GN) is the best gradient-based attribution method for identifying the disambiguating 'aha' example. But the benchmark's validity rests on the assumption that the model's correct prediction actually depends on the aha example. The paper's own Table 2 shows this assumption fails for a large fraction of items: on Gemma-2-2B, task accuracy without the aha example is already 98.0% on LD (Rule A), 100.0% on VO (Rule A), and 98.0% on TA (Rule A). When the model answers correctly without the aha example, its inductive bias, not the aha example, determines the answer; there is no well-defined 'reasoning locus' at e*. The paper excludes incorrect-answer instances (Appendix E) but does not condition on instances where removing e* actually changes the answer. Consequently, top-1/top-2 accuracy against e* is computed on a mixture of (i) genuine aha-dependent items and (ii) aha-independent items where attribution to e* is meaningless. Any method that assigns moderately high scores to the aha example for spurious reasons can be inflated or deflated by this mixture. The GN-versus-IG comparison and the scaling claims in Section 5.2 therefore inherit a confound: they compare methods on items where the target variable is not causally relevant. Section 6 and the Limitations acknowledge this issue qualitatively ('models might interpret these tasks differently than intended'), but the main results are not re-analyzed on the causal subset. This is a load-bearing concern because the headline ranking of IA methods is exactly the quantity affected.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether conventional input attribution (IA) methods can identify the single disambiguating 'aha' example in in-context learning (ICL). To make this question tractable, the authors design synthetic tasks (LINEAR-OR-DISTINCT, ADD-OR-MULTIPLY, VERB-OBJECT, TENSE-ARTICLE, POS-TITLE, and ASSOCIATIVE-RECALL) in which most demonstrations are ambiguous between two rules and one example resolves the ambiguity. They evaluate four IA methods (input erasure, gradient norm, input×gradient, integrated gradients) plus baselines (edit distance, attention, self-answer, chance) on six open-weight LLMs under 10-, 50-, and 100-shot settings, measuring top-1 and top-2 accuracy for locating the aha example. The main reported findings are that input erasure performs best, that simple gradient norm (GN) is the best gradient-based method, that integrated gradients is consistently worst among gradient methods, and that gradient-based interpretability tends to worsen with model scale while remaining relatively robust to the number of demonstrations.","tokens_in":21348,"tokens_out":3190,"duration_ms":30057,"significance":"If the evaluation is valid, the paper provides practically useful guidance for ICL interpretability: the simplest gradient method can outperform more sophisticated ones, and scaling behavior differs across explanation families. The study is broad in scope, covering six models, six tasks, three context lengths, multiple token-aggregation schemes, base-model comparisons, and a chain-of-thought extension; the appendices report assets and compute in detail, and the code is linked. The central caveat is that the benchmark's validity depends on the aha example being the causal locus of the model's decision. The paper's own control results (Table 2) show that for many items the model answers correctly without the aha example, so the reported attribution accuracies are computed on a mixture of aha-dependent and aha-independent items. This makes the headline quantitative comparisons hard to interpret without a re-analysis on the causal subset, although the authors qualitatively acknowledge the issue in Section 6 and the Limitations section.","major_comments":[{"comment":"The top-1/top-2 evaluation against the aha example assumes that the model's correct prediction is causally dependent on that example. Table 2 shows this fails for a large fraction of Gemma-2-2B items: task accuracy without the aha example is 98.0% for LD (Rule A), 100.0% for VO (Rule A), and 98.0% for TA (Rule A). For those items the correct answer is determined by the model's inductive bias or by other demonstrations, so an attribution score for the aha example has no well-defined ground truth. Appendix E excludes only incorrect-answer instances, not aha-independent correct instances. The main results in Section 5 and the scaling claims in Section 5.2 therefore compare methods on a mixture of items where the target variable is causally relevant and items where it is not. The discussion in Section 6 and the Limitations acknowledges this qualitatively, but the paper does not re-analyze the data on the subset where removing the aha example changes the answer. I request this conditioning as a required revision, because the GN-versus-IG comparison and the model-size conclusions could differ substantially on the causal subset.","section":"§4.2 and Table 2"},{"comment":"All reported attribution accuracies are point estimates with no confidence intervals or significance tests. For example, in Figure 2, differences between GN and I×G top-2 accuracy are often in the single digits over 360 items per task, and the claim that 'IG consistently yielded the lowest attribution accuracy' (Section 5.1) is based on rankings that may be within sampling noise. The paper would be much stronger if each bar carried a bootstrap confidence interval or if pairwise differences were tested (e.g., McNemar's test for paired items). Without such statistics, it is difficult to judge whether the headline finding that GN outperforms I×G and IG is robust or an artifact of a particular seed or item set.","section":"§5 and Figures 2–3"},{"comment":"The scaling claim that gradient-based methods 'maintain accuracy or rather improve against the longer context' is made on a single model (Gemma-2-2B) without error bars. The 10-shot chance rate is 20% for top-2, but the 100-shot chance rate is 2%, so raw accuracy comparisons across context lengths conflate method quality with the decreasing chance baseline. A normalized metric (e.g., accuracy minus chance, or a lift measure) or at least a statistical test against chance would make the robustness claim more convincing. This is load-bearing for the paper's second practical finding.","section":"§5.2 and Figure 3"}],"minor_comments":[{"comment":"The ADD-OR-MULTIPLY row contains a typo: 'numder' should be 'number'.","section":"Table 1"},{"comment":"The caption says 'can be obsered'; this should be 'observed'.","section":"Figure 6 caption"},{"comment":"The phrase 'attribution accurices' should be 'attribution accuracies.'","section":"Appendix E"},{"comment":"The notation x_i is used both for the token index and as a vector in R^d; the distinction between the token at position i and its embedding should be made explicit in the text.","section":"§2.1, Eq. (5)"},{"comment":"The justification for top-2 accuracy states that the model 'should at least consider the e* plus any other example,' but this presumes the model performs pairwise comparison. The authors later acknowledge in Section 6 that models may instead rely on inductive biases, so the top-2 rationale should be softened or explicitly conditioned on the model actually using the disambiguating example.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid, well-documented empirical study, and the authors have been transparent about several of its limitations. The main outstanding issue is that the central evaluation metric is not conditioned on the causal relevance of the aha example, and the paper's own Table 2 provides direct evidence that this matters. Since the authors already have the data, re-analyzing on the aha-dependent subset is feasible and would turn a conditional acceptance into a clean one. The absence of any statistical significance testing is also unusual for a claim of this strength and should be addressed during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this paper is a genuine empirical contribution with a real confound at the center of its main claim. The authors build synthetic ICL tasks in which most in-context examples are ambiguous and a single 'aha' example resolves the rule, then ask whether input attribution (IA) methods can identify that example. That is a fresh, useful evaluation lens. The experiments are broad: six models, six tasks, three context lengths, baselines including attention, edit distance, and self-answer, with code and data released. The headline finding—simple gradient norm (GN) defeats integrated gradients (IG)—is non-obvious and practically relevant for anyone choosing an attribution method in ICL. I believe it likely holds on the subset of prompts where the aha example actually matters.\n\nThe soft spot is the benchmark's validity on prompts where the aha example does not matter. Table 2 shows that on several tasks, Gemma-2-2B answers correctly without the aha example at 98–100% when its inductive bias already points to the right rule. For those items there is no well-defined 'reasoning locus' at e*, yet the top-1/top-2 metrics still score methods against it. The authors exclude wrong-answer instances but never condition on instances where removing e* changes the answer. That means the GN-vs-IG ranking and the scaling claims are computed on a mixture of causal and non-causal items. The authors acknowledge this qualitatively—Section 6 says models may rely on inductive biases—but they do not re-run the analysis on the causal subset. This is fixable: they already have the IE signal to define that subset. But it is load-bearing.\n\nMinor issues: there are no confidence intervals or significance tests anywhere, and with 360 questions per task that is a cheap omission. The top-2 metric is a bit generous, though the chance rate is reported.\n\nRelated work is properly cited; the paper builds on the McCoy/Mueller poverty-of-stimulus and mixed-signal tasks with clear credit.\n\nWho is this for? People working on ICL interpretability or on evaluating attribution methods. The benchmark is reusable, and the paper is a good reading-group discussion piece about what we mean by 'the' informative example in an ICL prompt.\n\nMy recommendation: yes, send it to peer review. The benchmark and experimental breadth justify referee time. The authors should be asked to redo the main analyses conditioning on instances where removing e* changes the predicted answer, and ideally add bootstrap intervals. If the GN-over-IG ordering survives that filter, this becomes a solid result; if not, the discussion of why attribution methods fail in ICL is still worth publishing.","headline":"A useful and honest benchmark for input attribution in ICL, but the headline GN-over-IG ranking is confounded by items where the aha example is not causally needed.","tokens_in":21835,"tokens_out":6345,"would_cite":true,"duration_ms":52499,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Conventional input attributions can locate the single disambiguating 'aha' example in in-context learning, but the simplest gradient-based method—gradient norm—does this best, and larger models are generally harder to interpret with…","keywords":["input attribution","in-context learning","interpretability","large language models","gradient norm","integrated gradients","inductive reasoning","aha example"],"falsifier":"Run the same six tasks in a modified version where the disambiguating information is split across two complementary examples so that no single example is necessary; if gradient-norm attribution still assigns one of them a top score as often as it does in the original benchmark, then the method is keying on position or surface similarity rather than on the disambiguating information itself.","tokens_in":20847,"feed_emoji":"🔍","tokens_out":9673,"duration_ms":77475,"temperature":0.7,"pith_summary":"This paper asks whether standard input-attribution (IA) methods—tools that score how much each input token contributed to a prediction—can explain how a language model performs inductive reasoning from a few in-context examples. To make the question answerable, the authors build synthetic tasks in which most demonstrations are ambiguous between two rules and exactly one 'aha' demonstration resolves the ambiguity. They find that a simple gradient-norm score is the most reliable gradient-based method for pointing to that deciding example, while the more elaborate integrated gradients method consistently does worst among the gradient-based methods. They also find that larger models are generally harder to interpret with gradient-based attributions, although asking the model to name the helpful example works better at larger scale. If this is right, the practical choice of interpretability tool for language models depends on model size and context length, not just on methodological sophistication.","feed_headline":"Gradient norm beats fancier attributions at finding the 'aha' example","feed_subtitle":"And larger models are generally harder to interpret with gradient-based methods, the study finds.","key_machinery":"The load-bearing object is the 'aha example' benchmark: a set of synthetic inductive-reasoning tasks in which every demonstration except one is consistent with two competing rules, and the single remaining demonstration disambiguates which rule is correct. Attribution methods are scored by whether that example receives the highest (top-1) or one of the two highest (top-2) attribution scores in the prompt, with the top-2 metric reflecting that the model must combine the aha example with some other example to infer the rule. The attribution scores themselves are computed at token level and summed within each demonstration; for gradient norm, input×gradient, and input erasure the paper uses a contrastive formulation that targets the correct answer against a foil answer corresponding to the losing rule. This machinery converts the vague question 'which example mattered?' into a measurable localization task, and it is what makes the ranking of methods comparable across tasks and models.","core_discovery":"The paper's central discovery, on its own terms, is that conventional input attribution methods can track at least part of the inductive reasoning process in in-context learning, but the ordering of methods defies the usual assumption that more sophisticated gradient-based methods are better. Across six synthetic tasks, the simplest gradient-based score—gradient norm computed on the contrastive objective of raising the correct answer's probability while lowering a foil's—outperforms input×gradient and integrated gradients, and input erasure outperforms all of them because the tasks are deliberately unsolvable without the disambiguating example. The paper further establishes that interpretability does not scale uniformly with model size: gradient-based attribution accuracy generally falls for larger models, while a self-answer baseline that asks the model to name the helpful example improves with size but degrades sharply when the number of demonstrations grows. Some tasks resembling linguistic generalization (verb-object, tense-article, part-of-speech/title) remain hard for every tested approach, which the authors attribute to the models' own inductive biases rather than to a single flaw in any one method.","pith_inferences":["If the gradient norm's edge over input×gradient really comes from ignoring embedding norms, then tasks with high-variance token frequencies should widen that gap; that is a testable prediction the paper does not run.","The single-aha-example design may understate attribution quality in realistic few-shot prompts where the decisive information is spread across several demonstrations; a distributed-cue variant of the benchmark would separate 'identifying a critical example' from 'explaining the whole inference.'","Because the paper's own control shows models adopt default rules when the aha example is absent, attribution accuracy may partly measure how well the model's preferred rule aligns with the intended rule; conditioning the benchmark on cases where the model has no default preference would sharpen the comparison.","A hybrid explanation pipeline—using gradient norm to prune demonstrations and then asking the model to verbalize its reasoning on the shortlist—could combine the scale trend of self-answer with the long-context robustness of gradient methods; the paper's results suggest such a combination is worth testing."],"forward_implications":["Gradient norm is a cheap, implementation-light diagnostic for finding which demonstration in a few-shot prompt is steering the model's answer, and it remains usable when the prompt is lengthened to 50 or 100 examples.","The added computation of integrated gradients does not pay for itself in this setting; researchers wanting to localize influential examples should try the simplest gradient-based score first.","Interpreting larger models with gradient-based attributions becomes harder, so interpretability results obtained on small models should not be assumed to transfer to larger ones.","Asking the model itself to say which example it used behaves oppositely: it improves with model size but collapses with longer contexts, so the two families of explanation cover different regimes.","Current attribution methods are not yet general-purpose in-context-learning explainers, since several linguistically flavored tasks resisted all tested approaches."],"supporting_citations":[{"why":"defines the in-context learning setting whose few-shot demonstrations are the objects of attribution.","marker":"Brown et al., 2020"},{"why":"the ambiguous inductive-learning scenario that the benchmark extends into aha-example tasks.","marker":"Mueller et al. 2024"},{"why":"supplies the cognitively motivated ambiguous-rule patterns the synthetic tasks are modeled on.","marker":"McCoy et al. 2020"},{"why":"introduces the gradient-norm attribution score that turns out to be the best gradient-based method.","marker":"Simonyan et al., 2014"},{"why":"the other stated source for gradient-norm input attribution in NLP.","marker":"Li et al., 2016a"},{"why":"introduces input erasure, the strongest attribution method in the paper's ranking.","marker":"Li et al., 2016c"},{"why":"shows contrastive explanations improve IA methods; the paper adopts this contrastive formulation for IE, GN, and I×G.","marker":"Yin and Neubig, 2022"},{"why":"defines integrated gradients, the gradient-based method that performs worst in these tasks.","marker":"Sundararajan et al., 2017"},{"why":"defines input×gradient, the method whose embedding-norm weighting fails to beat plain gradient norm here.","marker":"Shrikumar et al., 2017"}],"fun_headline_variants":["Simple gradient norm beats fancier attributions in ICL","Input erasure finds the key example better than gradients in ICL","Larger models resist gradient-based interpretation in ICL","Why simple attributions beat complex ones in ICL reasoning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark assumes that the single 'aha' example is the unique ground-truth locus of the model's inductive reasoning, so top-1 and top-2 attribution accuracy against that example measures whether an explanation captures the reasoning; if the model actually leans on distributed demonstrations, surface cues, or its own inductive biases, the accuracy figures no longer measure explanatory faithfulness.","fun_headline_variants_meta":{"raw":{"variants":["Simple gradient norm beats fancier attributions in ICL","Input erasure finds the key example better than gradients in ICL","Larger models resist gradient-based interpretation in ICL","Why simple attributions beat complex ones in ICL reasoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000848,"raw_usage":{"total_tokens":3679,"prompt_tokens":922,"completion_tokens":2757,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":2689}},"tokens_in":538,"tokens_out":2757,"duration_ms":16059,"temperature":1.0,"reasoning_tokens":2689,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:13:46.978140+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same six tasks in a modified version where the disambiguating information is split across two complementary examples so that no single example is necessary; if gradient-norm attribution still assigns one of them a top score as often as it does in the original benchmark, then the method is keying on position or surface similarity rather than on the disambiguating information itself.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"the ambiguous inductive-learning scenario that the benchmark extends into aha-example tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines integrated gradients, the gradient-based method that performs worst in these tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines input×gradient, the method whose embedding-norm weighting fails to beat plain gradient norm here."}],"review_version":1}