{"id":"6983a54f-f0d4-4de2-8fae-cb74fd112b72","arxiv_id":"2607.23804","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Context attribution methods cannot disentangle in-context from in-weight knowledge and assign unfaithful scores under overlap; new metrics and WMDP-Cyber++ quantify the failure.","lead":"Current LLM context-attribution tools give unreliable scores when the same fact is both in the prompt and already stored in the model weights. The authors give a controlled test suite and four metrics that expose this failure across four popular methods.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"SSP's near-chance result may be baked into the evaluation design: R_bench selection guarantees no context segment is counterfactually needed, yet provenance-based ground truth labels half of them \"ICL.\"","rationale":"The reader flagged the post-hoc disentanglement interface as possibly the wrong readout — I agree with the direction but locate the problem one level deeper: not just the thresholding rule, but the construction of R_bench and the provenance-as-usage ground truth, which together make near-chance SSP close to predetermined on the evaluated subset. This is a sharper, checkable mechanism than \"the interface may be unfair,\" and it also extends to CAC via the C1-vs-counterfactual-semantics tension. I do not think this sinks the paper: the CAC ranking-shift result and the score-distribution shifts (Fig. 3) still demonstrate a real, useful phenomenon — contributive scores are knowledge-state-dependent — and the benchmark plus protocol are genuine contributions. But the strongest claim (\"contributive scores alone cannot disentangle IW from ICL\") is currently supported by a test that cannot distinguish \"methods lack the signal\" from \"the evaluation subset contains no signal to detect.\" That keeps the verdict at CONDITIONAL rather than ACCEPT: the condition should specifically include the usage-verified SSP recomputation (and ideally a corroborative-oracle ceiling on the interface), alongside the reader's artifact-release condition. Not REJECT, because the core diagnostic value and the cross-method consistency of the pattern survive even under my critique, and the fix is a reanalysis of existing outputs rather than new training runs.","tokens_in":18258,"tokens_out":2530,"duration_ms":76029,"concrete_test":"Add a usage-verification probe to R_bench before scoring SSP: for each sample, replace each ICL-labeled segment with a minimally edited version that contradicts the gold answer (keeping style/length), and check whether the fine-tuned model's answer flips. Partition R_bench into samples where ≥1 ICL segment passes this counterfactual-usage probe vs. none do, and recompute SSP (Eq. 10) on each partition. Also report the SSP of a trivial always-IW predictor on R_bench as a ceiling diagnostic. If SSP stays ≈0.5 on the usage-verified partition while always-IW exceeds chance on the full set, the disentanglement claim survives; if SSP rises substantially on the verified partition, the headline \"cannot disentangle\" is an artifact of evaluating on samples where no segment is actually used.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline \"methods cannot disentangle IW from ICL (SSP ≈ 0.5)\" rests on how SSP's ground truth and sample set are constructed, and both choices tilt the metric toward chance regardless of method quality. (1) Sample selection: R_bench (Eq. 9) keeps only questions the fine-tuned model answers correctly with NO context. By construction, then, the answer is fully recoverable from weights, so no context segment — IW or ICL — is counterfactually necessary for the response. Removal/Shapley scores on these samples should be near zero and near-noise for every segment, which is exactly what Figure 3 shows (score mass shifts toward zero after fine-tuning). The disentanglement interface then labels low-score segments IW, so the decision reduces to noise — near-chance SSP is close to a tautology on this subset, not evidence about the methods' discriminative capacity. (2) Ground truth: a segment's label is corpus provenance (D_train vs D_forget), not verified usage. An ICL-labeled segment from D_forget that the model never reads is still counted as \"ICL\"; a method that correctly infers \"the model used its weights, not this segment\" is scored as wrong. Under the paper's own contributive definition (Discussion: contributive vs. corroborative), \"all segments IW\" is arguably the *correct* attribution on R_bench, yet the scoring penalizes it. A related definitional issue underlies CAC: criterion C1 demands a high score for segments whose knowledge is redundant with weights, which contradicts the counterfactual semantics the four methods were built to measure — low CAC after fine-tuning is what a correct counterfactual method should produce. The negative result is therefore partly definitional rather than purely empirical. The paper gestures at this in Discussion but still frames the outcome as \"unfaithful attribution,\" which is the load-bearing overstatement.","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper studies how context-attribution methods (ContextCite, TokenShapley, AttriBoT, TracLLM) behave when the knowledge carried by a context segment is also present in the model's weights. The authors control in-weight (IW) knowledge by LoRA fine-tuning three open models (LLaMA3-8B, Qwen3-8B, Qwen3-32B) on the WMDP cyber-retain corpus, build a provenance-labeled benchmark (WMDP-Cyber++, 1,987 questions with 3 IW + 3 ICL segments each), and propose four metrics: BCS (task-level top-k utility × removal impact on the base model), CAC (RBO ranking consistency between base and fine-tuned models), APS (harmonic mean of the two), and SSP (source-separation precision under a \"disentanglement interface\" that threshold contributive scores into IW/ICL labels). Headline findings: attribution rankings shift after knowledge exposure (CAC 0.34–0.56), task-level attribution quality is moderate and trades off against consistency, and all methods sit at chance (~0.5) on SSP, from which the authors conclude that contributive scores cannot disentangle IW from ICL sources.","tokens_in":18754,"tokens_out":5304,"duration_ms":310299,"significance":"The phenomenon the paper targets is real and practically important: removal-based attribution is blind to knowledge redundancy, and low contributive scores are genuinely ambiguous between \"irrelevant\" and \"redundant with weights\" — a problem for citation-reward mechanisms (e.g., SelfCite) and RAG evidence verification. The paper ships several concrete assets: a provenance-labeled benchmark with documented construction prompts, a controlled-exposure design with verified knowledge acquisition (no-context accuracy 42–54% → 77–97%), a task-level evaluation (BCS) that usefully complements likelihood-based Drop@k/LDS, and robustness checks over RBO persistence. The empirical core — score-mass shifting toward zero after fine-tuning (Figure 3) and rankings changing across knowledge states (Table 1) — is well supported across four methods and three model sizes. However, the most quotable claim (SSP ≈ 0.5 ⟹ \"methods cannot disentangle\") is substantially entangled with the evaluation's own construction, as detailed in the major comments; if reframed as \"contributive scores carry no provenance signal under redundancy,\" the contribution stands but is more modest.","major_comments":[{"comment":"The near-chance SSP result is close to baked into the evaluation design. R_bench (Eq. 9) retains only questions the fine-tuned model answers correctly with NO context; by construction, then, no context segment — IW- or ICL-labeled — is counterfactually necessary for the response on this subset, so removal/Shapley/surrogate scores for every segment should be near zero (Figure 3 confirms the mass shift toward zero). The disentanglement interface labels low-score segments IW, so on R_bench the interface plausibly collapses to 'predict all IW'; with balanced provenance labels this yields SSP = 0.5 exactly, regardless of any discriminative signal in the scores. The paper never reports the predicted-label distribution, per-class precision, or a confusion matrix, so the reader cannot distinguish 'scores are noise' from 'the interface degenerated on a subset where the decision is undefined.' At","section":"§Proposed evaluation protocol, Eqs. (9)–(10); Figure 2"},{"comment":"There is an internal tension between the paper's adopted definition and its faithfulness criteria. The paper explicitly adopts the contributive (counterfactual) notion of attribution, yet C1 requires a high score for a segment 'whether it is explicitly included ICL or implicitly IW' — i.e., a high score even when the segment's knowledge is fully redundant with the weights and its counterfactual contribution is genuinely zero. Under the contributive definition, a method that lowers the score of a redundant segment is behaving correctly, and CAC's premise ('a segment that contributes still contributes whether or not the knowledge is IW, so its attribution should not change') is false by construction. The interesting and defensible claim is that contributive scores are knowledge-state-dependent and therefore uninterpretable as evidence of textual support; the claim that methods are 'unfaith","section":"§Problem statement, criterion C1; §Discussion (contributive vs. corroborative)"},{"comment":"SSP ground truth is corpus provenance (D_train vs. D_forget), not verified usage. An ICL-labeled segment retrieved from D_forget that the model never relies on is still labeled ICL, and a method that correctly infers 'the answer came from the weights, not this segment' is scored wrong. The labels are thus corroborative in the paper's own taxonomy, while the methods and the interface are contributive — a category mismatch that independently pushes SSP toward chance on R_bench (where usage of any segment is nil by selection). A usage-verified variant is feasible within the authors' setup: e.g., test whether supplying only the ICL segments changes the fine-tuned model's answer, or whether only the IW segments suffice, and restrict provenance labels to segments passing such a dependence check. Without this, SSP measures agreement with retrieval provenance, not source separation.","section":"§WMDP-Cyber++ dataset (construction); Eq. (10)"},{"comment":"No uncertainty quantification is provided for any of the four metrics. BCS is computed from aggregate accuracies (Eqs. 3–5), yielding a single point per method/model/k with no resampling-based interval; CAC and SSP are per-sample averages and could easily carry confidence intervals, but none are reported. The text draws comparative conclusions from differences of 0.02–0.05 (e.g., TokenShapley vs. TracLLM APS 0.47 vs. 0.45 on Qwen3-8B at k=3) that are unlikely to survive bootstrap variability over 1,987 questions, and the Drop@k/LDS comparisons in Table 2 have the same issue. Bootstrap CIs over questions (clustered appropriately) should accompany Tables 1–2 and Figure 2.","section":"§Evaluation / Table 1; Figure 2"},{"comment":"The external validity of the central claim rests on LoRA fine-tuning (r=16, 3 epochs) on the WMDP cyber-retain corpus being a faithful proxy for pretraining-era IW knowledge. Two observations weaken this: (i) base models already answer 42–54% of questions correctly with no context (4-choice MC; chance 25%), so a substantial fraction of 'IW' content is pre-existing and heterogeneous; (ii) the fine-tuned Qwen3-8B reaches 96.8% no-context accuracy, consistent with shallow answer-key memorization rather than integrated knowledge, and shallow knowledge may interact with removal-based attribution differently than pretraining knowledge (e.g., higher no-context log-probability inflates p_empty and the IW proxy in Eqs. 13–14). The authors acknowledge this as future work, but since every headline number is conditioned on this one fine-tuning recipe, at least one robustness run (different rank/epoc","section":"§Discussion (Finetuning as a proxy); appendix fine-tuning details"}],"minor_comments":[{"comment":"BCS is a geometric mean of Util and RmI, both of which can be negative (e.g., rm-topk accuracy exceeding full-ctx accuracy by noise) or exceed 1; the paper does not state whether values are clipped or how undefined cases are handled.","section":"Eq. (6)"},{"comment":"Dangling cross-reference: 'we describe the detailed process in Section .' — the section number is missing.","section":"§Proposed evaluation protocol (SSP)"},{"comment":"RBO with persistence ρ=0.5 over only k=6 segments is extremely top-heavy (effective depth ≈ 2); the choice deserves justification beyond the appendix robustness check, and the RBO reference (Webber et al., 2010) is not cited.","section":"§Evaluation (Metrics)"},{"comment":"Axes are unlabeled and units differ across panels (some scores appear to be log-probability differences, others normalized); as the key evidence for the score-shift claim, the figure needs labeled axes and a caption stating the x- and y-variables.","section":"Figure 3"},{"comment":"GPT-4o smoothing with transitional phrases could leak provenance signal (e.g., systematically different connective styles around D_train vs. D_forget segments); a check that a text-only classifier cannot recover provenance from the smoothed context above chance would strengthen the 'ordering carries no provenance signal' claim.","section":"§WMDP-Cyber++ (Step 3)"},{"comment":"The ensemble weight w_loo = 2.0 is introduced without justification or sensitivity analysis.","section":"§Evaluation (TracLLM); appendix"},{"comment":"Typographical issues: 'suprooting' (Introduction), inconsistent capitalization 'Attribot'/'AttriBoT' (Tables 2–3), 'cannot do the disentanglement' (abstract), 'Further details are provided followed by each method in the appendix' (§Evaluation), and hyphenation artifacts in the abstract ('con- tributive').","section":"Throughout"},{"comment":"The y-axis spans only 0.40–0.55; the truncation visually amplifies small differences — consider starting at 0 or noting the chance line explicitly.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The benchmark and the controlled-exposure methodology are genuinely useful contributions and I would like to see this work published in some form. However, the SSP headline — likely the sentence that will be quoted from the abstract — currently rests on a sample-selection construction (Eq. 9) under which near-chance performance is close to a tautology, and the authors do not report the diagnostics (predicted-label distribution, per-class accuracy) that would let a reader see this. The CAC/BCS results and Figure 3 stand independently and are, in my view, the paper's real contribution. Revision is feasible within the manuscript's scope: the needed analyses use existing machinery. If the authors can show the SSP result survives on samples where context is actually load-bearing, the claim is strong; if not, the paper should be reframed around knowledge-state dependence of contributive scores, which is still a worthwhile result."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core here is a controlled stress test of four popular context-attribution methods when the same facts sit in both context and weights. ContextCite already waved at low scores for supporting sources under parametric knowledge; this paper turns that into a protocol (BCS/CAC/APS/SSP), a provenance-labeled set (WMDP-Cyber++ from WMDP retain/forget), and a multi-model multi-method study. That kit is the real product.\n\nWhat they do well: base vs LoRA-finetuned comparisons on three scales, task-level sufficiency/necessity (Util × RmI → BCS) instead of only likelihood drop, ranking stability via RBO, and a clear contrast with Drop@k/LDS that favor LOO-style methods by construction. Table 1 and Figure 3 are the load-bearing evidence: CAC stays low (roughly 0.34–0.56), score mass shifts after fine-tuning for the log-prob methods, and APS rankings diverge from likelihood metrics. That is enough to show “contributive score under fixed model” is not a stable reading of support once IW overlap exists. For RAG faithfulness and citation-reward design, that warning is worth having on the shelf.\n\nSoft spots, in proportion. The LoRA-as-IW proxy is stated and reasonable given closed pretraining data; still a proxy. GPT-4o re-ranking/smoothing in dataset build is a mild contamination risk for labels. The sharper issue is SSP. R_bench keeps only items the finetuned model already answers with no context, so no segment is counterfactually necessary; scores collapse toward noise (Figure 3), and a simple threshold interface then mostly emits IW. With balanced provenance labels, SSP ≈ 0.5 is close to what you get if the method correctly says “weights carried it.” Provenance is not verified usage, and criterion C1 (high score for knowledge that is also in weights) fights the counterfactual semantics these methods were built for—so low CAC is partly what a correct removal method should do. The paper nods at this in Discussion but still sells “unfaithful” and “cannot disentangle” a bit harder than the SSP design supports. That does not sink BCS/CAC/APS; it means SSP is a weak headline metric.\n\nWho it’s for: people building or evaluating context attribution, RAG citations, or faithfulness rewards. Not a new theory of attribution—honest negative empirics plus reusable eval. I’d bring it to reading group, cite the protocol and the failure mode, and send it to peer review. Ask referees to pressure-test LoRA depth, release artifacts, and either redesign SSP around verified usage or demote it. Worth engaging; not a desk reject.","headline":"Solid diagnostic kit showing removal-based context attribution breaks under IW overlap; SSP’s near-chance result is partly baked into R_bench, but CAC/BCS still carry the main claim.","tokens_in":19587,"tokens_out":666,"would_cite":true,"duration_ms":23020,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Context attribution methods give unreliable scores when the model already knows the answer from its weights, and cannot tell in-context evidence from memorized knowledge.","keywords":["context attribution","in-weight knowledge","in-context learning","source separation","LLM evaluation","WMDP-Cyber++","leave-one-out","Shapley values"],"falsifier":"Re-run the same four methods on a model whose pretraining corpus is fully known and public, with true overlap segments taken from that corpus rather than from LoRA, and check whether CAC rises substantially above ~0.5 and SSP rises well above chance; if both stay low, the claim holds under real in-weight knowledge.","tokens_in":19229,"feed_emoji":"🔍","tokens_out":931,"duration_ms":28824,"temperature":0.7,"pith_summary":"Context attribution tools are meant to show which parts of a prompt actually drive an LLM's answer. This paper shows that when the same facts already live in the model's weights, those tools systematically mis-score the evidence: a truly supporting passage can look unimportant simply because removing it does not change the output. The authors build a controlled test—four new metrics plus a cybersecurity benchmark with known in-weight versus in-context labels—and run four leading methods through it. Rankings shift after the model is taught the material, task-level necessity and sufficiency stay only moderate, and attempts to read source labels off the scores stay near chance. The practical upshot is that today's attribution numbers are hard to trust whenever retrieval and training data overlap, which is the common case in real deployments.","feed_headline":"Attribution scores break when the model already knows","feed_subtitle":"Four leading methods cannot separate context from weights; new metrics and a labeled benchmark show why","key_machinery":"An evaluation protocol built on four metrics—BCS (task-level sufficiency and necessity of top-k on a base model), CAC (rank consistency between base and fine-tuned models via RBO), APS (their harmonic mean), and SSP (precision of IW/ICL labels read from scores)—together with WMDP-Cyber++, a benchmark whose context segments carry ground-truth provenance by construction.","core_discovery":"Across four representative context-attribution methods, contributive scores become unfaithful under in-weight overlap: after controlled knowledge exposure, ranking consistency falls (CAC typically 0.34–0.56), top-k segments are only moderately necessary and sufficient on a base model (BCS), and adapted source-separation precision stays near random (~0.5). Contributive scores alone therefore cannot disentangle in-context learning from in-weight knowledge.","pith_inferences":["Retrieval-augmented generation pipelines that treat attribution scores as trust or citation signals will systematically under-weight documents that duplicate training data.","The same failure mode likely affects any removal- or ablation-based explanation method, not only the four tested here.","A useful next method would emit two scores per segment—contextual marginal effect and parametric coverage—rather than one scalar.","If pretraining data provenance ever becomes auditable, the protocol could be re-run without fine-tuning proxies and would become a stronger acceptance test for attribution tools."],"forward_implications":["Low attribution scores cannot be read as ‘irrelevant’ when the same knowledge may already be in the weights.","Likelihood-based checks (log-prob drop, LDS) can rank methods highly even when task-level attribution under overlap fails.","Systems that discard low-scoring evidence or reward citations from attribution scores risk throwing away or mis-crediting supporting documents.","Future attribution methods need primitives that quantify both contextual contribution and parametric coverage, not removal alone.","WMDP-Cyber++ and the BCS/CAC/APS/SSP suite give a concrete testbed for any new method that claims source-aware attribution."],"fun_headline_variants":["Attribution scores fail under in-weight overlap","Four methods can't separate context from model weights","Contributive scores turn unfaithful when knowledge is already known","New metrics show attribution breaks on training-data overlap","Source separation stays near random for leading attribution methods"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That teaching the model the material with LoRA fine-tuning is a fair stand-in for the knowledge that would already sit in its weights from real pretraining, and that simple score thresholds can fairly test whether those methods can separate sources.","fun_headline_variants_meta":{"raw":{"variants":["Attribution scores fail under in-weight overlap","Four methods can't separate context from model weights","Contributive scores turn unfaithful when knowledge is already known","New metrics show attribution breaks on training-data overlap","Source separation stays near random for leading attribution methods"]},"model":"grok-4.5","effort":"low","cost_usd":0.003479,"raw_usage":{"total_tokens":1161,"prompt_tokens":763,"num_sources_used":0,"completion_tokens":57,"cost_in_usd_ticks":34788000,"prompt_tokens_details":{"text_tokens":763,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":341,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":763,"tokens_out":57,"duration_ms":6323,"temperature":1.0,"reasoning_tokens":341,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T11:54:29.996926+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-run the same four methods on a model whose pretraining corpus is fully known and public, with true overlap segments taken from that corpus rather than from LoRA, and check whether CAC rises substantially above ~0.5 and SSP rises well above chance; if both stay low, the claim holds under real in-weight knowledge.","supporting_citations":[],"review_version":1}