{"id":"d493ed5e-0efe-4487-bd2e-25681786de8c","arxiv_id":"2506.23423","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"TuCo quantifies the contribution of fine-tuning to individual LLM responses as the ratio of fine-tuning to pre-training component magnitudes, and shows jailbreak attacks lower this ratio.","lead":"This paper introduces TuCo, a per-prompt metric that measures how much of an LLM's response comes from fine-tuning versus pre-training, by comparing the model's internal activations with those of its pre-trained original. The authors show the metric drops when jailbreak attacks are used, suggesting such attacks work by weakening the fine-tuned safety behaviors.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TuCo's central interpretive step — that FTC magnitude equals fine-tuning's contribution to a response — is unvalidated; a per-prompt FTC-ablation correlation would settle it.","rationale":"The reader's weakest assumption identifies the same broad validity concern: a large FTC norm is treated as evidence of a causal role for fine-tuning, but distribution shift could produce the same pattern. I agree that this is the load-bearing point. I would sharpen it: the real question is not only whether low TuCo tracks distribution shift, but whether TuCo predicts the actual causal effect of removing FTC on a per-prompt basis. That is what the alpha-scaling experiments fail to test, since they report aggregate accuracy deltas rather than per-prompt sensitivity. The proposed ablation correlation is a single, feasible computational check that would decide between TuCo-as-contribution and TuCo-as-anomaly-score. If it fails, the paper's headline interpretation is not supported; if it passes, the distribution-shift concern is largely defused, because low-TuCo prompts would demonstrably be ones where fine-tuning changes the output least. The paper remains a useful contribution either way, so the reader's CONDITIONAL verdict is appropriate; no adjustment is needed.","tokens_in":40504,"tokens_out":7922,"duration_ms":86557,"concrete_test":"On a held-out set of prompts spanning the high- and low-TuCo regimes (e.g. from Sections 5.2-5.4), compute TuCo and also the per-prompt change in output distribution when FTC is removed (FTC alpha-scaling with alpha = 0, i.e. run x_{l+1} = x_l + PTC(x_l,l)). Measure the KL divergence or argmax-token probability shift between the fine-tuned and FTC-removed next-token distributions. If TuCo measures fine-tuning contribution, prompts with low TuCo should show systematically smaller output changes under FTC removal, yielding a significant positive Spearman correlation across prompts. If the correlation is weak or absent, TuCo reflects internal difference magnitude rather than contribution to the response, and the jailbreak-attenuation claim loses its stated meaning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The algebraic decomposition in Proposition D.3 is exact, but the paper's central claim depends on a further interpretive step: that the norm of the fine-tuning component FTC measures how much fine-tuning contributes to the response. Nothing in the paper validates this step. TuCo is a norm ratio of layer-wise differences evaluated along the fine-tuned trajectory; Appendix B's own example gives TuCo = 1/4 on an input where the fine-tuned and pre-trained models produce identical final hidden states, so TuCo is not a measure of output change. Conversely, the Section 5.2 result (OpenWebText below HH-RLHF) and the conjugate-prompting language ordering are exactly what a distribution-shift probe would produce: prompts far from the fine-tuning distribution tend to have smaller layer-wise differences. The FTC alpha-scaling experiments show that the FTC direction carries behaviorally relevant signal in aggregate, but they do not establish that a prompt's TuCo value predicts how much fine-tuning actually changes that prompt's output. The jailbreak interpretation — that attacks succeed by attenuating the fine-tuning component — therefore rests on an unvalidated equivalence between activation-space magnitude and causal contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TuCo, a per-prompt metric intended to quantify how much fine-tuning contributes to an LLM's individual responses. The method assumes access to both a pre-trained model and its fine-tuned counterpart. The core algebraic observation, formalized in Proposition D.3, is that at every residual layer the fine-tuned layer output can be written exactly as the pre-trained layer output evaluated on the fine-tuned trajectory (PTC) plus the difference between the fine-tuned and pre-trained layer outputs (FTC). The paper proves a discrete Grönwall bound (Proposition 4.2) showing that the maximum over layers of a certain ratio of cumulative FTC and PTC norms controls the distance between final hidden states of the two models. TuCo is then defined in Definition 4.3 as the norm of the cumulative last-token FTC divided by the sum of the norms of the cumulative last-token PTC and FTC. Empirically, the paper shows that scaling FTC by a factor α can modulate model behavior on Model-Written Evaluations and MMLU; that TuCo is lower for OpenWebText than for HH-RLHF chats; that three jailbreak attacks (GCG, conjugate prompting, many-shot) are reported to reduce TuCo; and that TuCo is lower on successful jailbreaks for most tested models. The paper interprets these results as quantitative evidence that jailbreaks attenuate the effect of fine-tuning during the forward pass.","tokens_in":40703,"tokens_out":7368,"duration_ms":86242,"significance":"If the central interpretive claim were validated, TuCo would be a valuable, inexpensive interpretability tool: it requires only two forward passes and no training, and the layer-wise decomposition is exact and model-agnostic. The alpha-scaling experiments are a genuinely interesting intervention, showing that the FTC direction carries behaviorally relevant signal for several behaviors and tasks. The paper is also commendable for releasing code and for being explicit about the requirements and limitations of the method. However, the significance is currently limited by an unvalidated leap from 'FTC norm is large' to 'fine-tuning contributes to this response.' The paper's own example in Appendix B shows that TuCo can be nonzero when the fine-tuned and pre-trained models produce identical final hidden states (and hence identical sampling distributions), which directly contradicts the name and interpretation of the metric as measuring contribution to a response. Several empirical claims, notably the claim that all three jailbreak attacks reduce TuCo, are not supported by the reported per-model AUCs.","major_comments":[{"comment":"Proposition 4.2 bounds the final-state discrepancy in terms of β = max_{0≤l<L} ∥FTC_l∥ / (∥PTC_l∥ + ∥FTC_l∥), but TuCo is defined using β_L (the final-layer cumulative ratio) and only the last token. The theorem does not imply a bound or guarantee for TuCo. The discussion in §4.5 and Appendix A justifies these changes by symmetry and interpretability, but no matching theorem or even a formal heuristic is supplied. As written, the theoretical analysis does not support the claim that a low TuCo means fine-tuning has little effect on the response; it supports a statement about a different quantity. Please either prove a similar bound for TuCo (or a related quantity), or clearly state in the main text that TuCo is a heuristic metric motivated by, but not implied by, Proposition 4.2.","section":"§4.4–4.5, Proposition 4.2 and Definition 4.3"},{"comment":"Example B.1 is a two-layer construction where the fine-tuned and pre-trained models have identical final hidden states, yet TuCo = 1/4. Since the final hidden state determines the next-token distribution, the response is identical in both models, so fine-tuning makes no causal contribution to the response on this input. The paper embraces this as a feature (capturing internal activity), but it undermines the metric's name and its central interpretation as measuring the contribution of fine-tuning to a response. The alpha-scaling experiments in §5.1 show that the FTC direction contains task-relevant signal in aggregate, but they do not establish that a prompt's TuCo value predicts how much fine-tuning actually changes that prompt's output. Please add a per-prompt validation: for example, correlate TuCo with the change in output distribution when FTC is scaled or removed, or with the actual distance between fine-tuned and pre-trained output distributions across a diverse prompt set.","section":"§4.1 and Appendix B, Example B.1"},{"comment":"The sentence 'We find that all three attacks significantly decrease TuCo when applied to harmful prompts' is not supported by the CP columns of Table 1. For Gemma 7B the Japanese-vs-low-resource AUC is 0.12, for Zephyr Gemma the Japanese AUC is 0.23 and Hungarian is 0.19, and for Llama 3 8B the Hungarian AUC is 0.40. These values indicate the opposite direction for a substantial fraction of model/language pairs. The clean web-share ordering is demonstrated only for Llama 2 13B. Please report per-language-pair statistics with confidence intervals, and either soften the claim to the models/languages where the effect holds, or explain why the reversed AUCs are consistent with the overall conclusion.","section":"§5.3, Table 1"},{"comment":"The empirical comparisons are confounded by distribution shift and prompt length. OpenWebText and HH-RLHF differ in many ways beyond 'pre-training-like' versus 'chat-like'; many-shot jailbreaks increase prompt length; conjugate prompting changes language and token distribution; GCG adds an adversarial suffix. Since TuCo is lower on OpenWebText, the observed decreases under jailbreaks could be an anomaly-score effect — prompts far from the fine-tuning distribution tend to have smaller layer-wise differences — rather than attenuation of fine-tuning's causal role. The statement in §5.4 that TuCo is 'not merely reflecting spurious aspects of the prompt (e.g. length or perplexity)' is not backed by any control. Please include explicit controls (e.g., length-matched prompts, perplexity-matched non-attack prompts, or regression analyses partialling out these variables) so that the causal interpretation is not confounded.","section":"§5.2–5.4"}],"minor_comments":[{"comment":"The text says the AUC is above 0.80 for all but two models, but the table shows only Zephyr Gemma (0.63) below 0.80; please correct the count.","section":"§5.2, Table 1"},{"comment":"The statistical significance claims for MWE and MMLU are reported without standard errors, p-values, or multiple-testing correction; please provide these details or clarify the test used.","section":"§5.1, Appendix F.1"},{"comment":"Algorithm 1's comment 'Compute PTC for layer l' could be misinterpreted as running the pre-trained model's own forward pass; the fact that PTC is evaluated on the fine-tuned trajectory is explained in the text but should be emphasized in the algorithm caption as well.","section":"Algorithm 1"},{"comment":"There are several typographical issues in the proof of Corollary D.5 (e.g., malformed summation indices) and in Appendix D.5 ('traingle inequality', 'Propostion'); please proofread the appendix carefully.","section":"Appendix D.4"},{"comment":"For many-shot jailbreaking, the paper reports a downward trend in TuCo with the number of shots, but the prompt length also increases with the number of shots; a length-controlled comparison would strengthen the interpretation.","section":"§5.3"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid algebraic core and an interesting intervention study, but the central metric's causal interpretation is not validated. The issues raised in major comments 1–2 are load-bearing for the paper's headline claim; major comments 3–4 concern empirical overclaiming and confounds. These are fixable with additional experiments and careful rewriting, so I recommend major revision rather than rejection. The paper would benefit from a per-prompt ablation correlating TuCo with actual output sensitivity, and from controls for distribution shift and prompt length."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper before reading it. First, the core tool is simple and sensible: with access to the pretrained model, define the fine-tuning component layer-wise as the difference between fine-tuned and pretrained layer outputs, accumulate those differences along the residual stream, and take the ratio of norms on the last token. The decomposition is exact by construction, costs two forward passes, and the paper ships code. Second, the headline interpretation — that TuCo measures how much fine-tuning causally contributes to a response — is not established. The paper shows that three jailbreak families lower TuCo and that TuCo is lower on successful attacks, but that pattern is exactly what a distribution-shift probe would produce, since the comparison is always between the fine-tuned model's trajectory and what the pretrained model would do from the same states. The authors never validate that a prompt's TuCo value tracks whether fine-tuning actually changed that prompt's output.\n\nWhat the paper does well: the alpha-scaling experiments are the most convincing part. Scaling FTC up or down shifts behavior and MMLU accuracy in an out-of-sample cross-validated setup, which shows the FTC direction carries real behavioral signal in aggregate. The jailbreak results are also broad — seven open models, three attack families, AUC numbers mostly above 0.8 — and the many-shot dose-response curves are clean. The Gronwall bound is a correct textbook application, though the paper is honest that the actual TuCo metric replaces the max-over-layers beta with the final-layer ratio and restricts to last-token norms, so the theorem does not directly justify the specific formula.\n\nThe soft spots are real but not fatal. The Appendix B example illustrates the key problem: TuCo can be nonzero while final hidden states are identical, so it is not a measure of output change. The claim that TuCo reflects \"fine-tuning's contribution to the response\" therefore rests on an unstated equivalence between activation-space magnitude and causal contribution. The empirical sections also lack confidence intervals, and the paper overstates the \"exactness\" of a decomposition that is exact only in an algebraic sense, not in the sense of isolating a causal mechanism.\n\nWho this is for: interpretability and safety researchers who want a cheap, per-prompt diagnostic and are willing to treat TuCo as an indicator of \"how pre-training-like the forward pass is\" rather than a literal measure of fine-tuning's causal role. It deserved a serious referee then, and I would send it to review now, with the main request being a direct validation: ablate FTC at varying intensities per prompt and show that the resulting change in output distribution correlates with TuCo. Without that, the paper's strongest conclusion should be softened to \"jailbreaks move the forward pass toward the pretrained model's behavior,\" which is still interesting.","headline":"TuCo is a genuinely useful per-prompt metric with strong cross-model empirical results, but the paper's central claim that it measures fine-tuning's causal contribution is unvalidated and partly confounded with distribution shift.","tokens_in":41228,"tokens_out":1405,"would_cite":false,"duration_ms":21103,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims every fine-tuned LLM decomposes exactly into pre-training and fine-tuning parts, and the ratio TuCo measures how much fine-tuning shaped each response.","keywords":["Tuning Contribution","fine-tuning attribution","residual stream decomposition","interpretability","jailbreak attacks","many-shot jailbreaking","conjugate prompting","LLM safety"],"falsifier":"Take a harmless prompt and its translation into a low-resource language such as Swahili or Malayalam; if TuCo drops just as much for the benign translation as for the harmful one while the model's helpfulness and refusal behavior are unchanged, then TuCo is tracking distribution shift rather than the causal contribution of fine-tuning.","tokens_in":40304,"feed_emoji":"📉","tokens_out":12778,"duration_ms":117009,"temperature":0.7,"pith_summary":"The paper claims that any fine-tuned transformer LLM can be decomposed exactly, layer by layer, into a pre-training component (the output of the corresponding pre-trained layer) and a fine-tuning component (the difference between the fine-tuned and pre-trained layer outputs). Building on that identity, it defines TuCo as the ratio of the accumulated last-token magnitudes of the fine-tuning component to the pre-training component, and interprets TuCo as the fraction of the response attributable to fine-tuning. Experimentally, the paper shows that scaling the fine-tuning component during the forward pass changes model behavior and benchmark accuracy, and that three jailbreak techniques—adversarial suffix attacks, translation to low-resource languages, and many-shot prompting—all reduce TuCo, more so when the attack succeeds. If correct, TuCo provides a per-prompt, inference-time measurement of fine-tuning's influence, with direct relevance to interpretability and to understanding how jailbreaks bypass safety training.","feed_headline":"One number, TuCo, measures fine-tuning's role in each LLM reply","feed_subtitle":"Splitting each forward pass into pre-training and fine-tuning parts, TuCo drops when jailbreak attacks succeed.","key_machinery":"The load-bearing object is the canonical decomposition of a fine-tuned transformer into two generalized components: PTC, the output of the pre-trained layer on the same residual-stream input, and FTC, the difference between the fine-tuned and pre-trained layer outputs. Because every residual layer adds its output to the stream, the accumulated sums $\\mathrm{PTC}_l$ and $\\mathrm{FTC}_l$ make the final hidden state an exact sum of a pre-training and a fine-tuning contribution. A discrete Grönwall inequality converts the ratio of these accumulated magnitudes into a bound on how far the fine-tuned final state can drift from the pre-trained one, and TuCo is the last-token form of that ratio, chosen so that pre-training and fine-tuning contributions sum to one.","core_discovery":"The central discovery is the exact canonical decomposition $f^{\\mathrm{FT}}_\\Theta(x,l) = \\mathrm{PTC}(x,l) + \\mathrm{FTC}(x,l)$ with $\\mathrm{PTC}(x,l) = f^{\\mathrm{PT}}_\\phi(x,l)$ and $\\mathrm{FTC}(x,l) = f^{\\mathrm{FT}}_\\Theta(x,l) - f^{\\mathrm{PT}}_\\phi(x,l)$. Because transformer layers write additively to the residual stream, the final hidden state satisfies $x_L = x_0 + \\mathrm{PTC}_L + \\mathrm{FTC}_L$, and a discrete Grönwall bound shows that the ratio $\\beta_l = \\|\\mathrm{FTC}_l\\|_1/(\\|\\mathrm{PTC}_l\\|_1 + \\|\\mathrm{FTC}_l\\|_1)$ controls the distance between the pre-trained and fine-tuned final hidden states. TuCo is the last-token version of this ratio, $\\mathrm{TuCo}(x) = \\|\\mathrm{proj}_n(\\mathrm{FTC}_L)\\|_1/(\\|\\mathrm{proj}_n(\\mathrm{PTC}_L)\\|_1 + \\|\\mathrm{proj}_n(\\mathrm{FTC}_L)\\|_1)$. The paper's empirical claim is that this scalar is behaviorally meaningful: multiplying the fine-tuning component by $\\alpha \\in [0.75, 1.25]$ shifts MMLU accuracy by up to about 5% and agreement with Christian beliefs by up to 24% on Llama 2 13B, and three jailbreak families make TuCo drop, with AUC 0.87 separating successful from unsuccessful attacks on Llama 2 13B.","pith_inferences":["If TuCo is read as an anomaly score rather than a causal measure, the conjugate-prompting result predicts that any out-of-distribution input, even a harmless one, should lower TuCo; this is directly testable on benign prompts translated to Swahili or Malayalam.","The FTC-scaling result opens the possibility of an inference-time 'fine-tuning dial' for open models, but the paper does not propose it as a defense, and it would not transfer to closed models where the pre-trained base is unavailable.","A natural extension is to compute layer-level or token-level TuCo maps to localize where fine-tuning's contribution enters the forward pass; the paper only defines the scalar per-prompt value."],"forward_implications":["Scaling the fine-tuning component during the forward pass is a control knob: on MMLU, 71% of tasks improve out-of-sample with average gains of 1.03–2.69%, and agreement with Christian beliefs rises by 24% for Llama 2 13B when the component is scaled by $\\alpha = 1.25$.","All three jailbreak families reduce TuCo, and increasing many-shot attack intensity lowers TuCo monotonically, which the paper interprets as the attack implicitly applying FTC-scaling with $\\alpha < 1$.","Successful jailbreaks have lower TuCo than unsuccessful ones (AUC 0.87 for Llama 2 13B and above 0.8 for most other tested models), so TuCo can be used as an analysis signal for where safety-tuning has lost its grip.","TuCo separates pre-training-like web text from chat-like fine-tuning inputs with AUC above 0.8 for eight of nine models, supporting the claim that it tracks the fine-tuning regime rather than prompt length or perplexity.","TuCo is computable at inference time from two forward passes, so it can be applied to open billion-parameter models without training probes or gradient access."],"supporting_citations":[{"why":"Supplies the residual layer structure that makes the layerwise additive decomposition exact.","marker":"[Vaswani et al., 2017]"},{"why":"Supplies the discrete Grönwall inequality used in the bound on final-state discrepancy.","marker":"[Clark, 1987]"},{"why":"Supplies the GCG attack, the harmful-instruction dataset, and the refusal-detection protocol used in the jailbreak experiments.","marker":"[Zou et al., 2023b]"},{"why":"Supplies conjugate prompting via low-resource-language translation and the claim that such prompts push models toward pre-training behavior.","marker":"[Kotha et al.]"},{"why":"Supplies the many-shot jailbreaking technique and the relationship between number of shots and harmful response rate.","marker":"[Anil et al., 2024]"},{"why":"Supplies the competing-objectives and mismatched-generalization hypothesis that the jailbreak results are said to give quantitative evidence for.","marker":"[Wei et al., 2024]"},{"why":"Supplies the HH-RLHF chat dataset used as the fine-tuning-like input condition.","marker":"[Bai et al., 2022a]"},{"why":"Supplies OpenWebText as the pre-training-like input condition for the web-versus-chat comparison.","marker":"[Gokaslan & Cohen, 2019]"},{"why":"Supplies the MMLU benchmark used to show that FTC scaling changes model capabilities.","marker":"[Hendrycks et al., 2020]"},{"why":"Supplies the Model-Written Evaluations used to show that FTC scaling changes stances and behaviors.","marker":"[Perez et al., 2023]"}],"fun_headline_variants":["TuCo: one metric for fine-tuning's effect on each LLM response","New metric TuCo tracks fine-tuning's role in every LLM reply","TuCo drops during jailbreaks, revealing fine-tuning's safety link","Splitting LLM hidden states: TuCo measures fine-tuning per prompt","Jailbreaks reduce TuCo, a new measure of fine-tuning influence"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the magnitude of the fine-tuning component—the difference between fine-tuned and pre-trained layer outputs—reflects how much fine-tuning actually causes the response, rather than merely how unusual the input is for the fine-tuned model.","fun_headline_variants_meta":{"raw":{"variants":["TuCo: one metric for fine-tuning's effect on each LLM response","New metric TuCo tracks fine-tuning's role in every LLM reply","TuCo drops during jailbreaks, revealing fine-tuning's safety link","Splitting LLM hidden states: TuCo measures fine-tuning per prompt","Jailbreaks reduce TuCo, a new measure of fine-tuning influence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000439,"raw_usage":{"total_tokens":2344,"prompt_tokens":1179,"completion_tokens":1165,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":795,"completion_tokens_details":{"reasoning_tokens":1068}},"tokens_in":795,"tokens_out":1165,"duration_ms":8885,"temperature":1.0,"reasoning_tokens":1068,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:43:36.550742+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a harmless prompt and its translation into a low-resource language such as Swahili or Malayalam; if TuCo drops just as much for the benign translation as for the harmful one while the model's helpfulness and refusal behavior are unchanged, then TuCo is tracking distribution shift rather than the causal contribution of fine-tuning.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the discrete Grönwall inequality used in the bound on final-state discrepancy."},{"cited_title":"Many-shot jailbreaking","cited_arxiv_id":null,"evidence_quote":"Supplies the many-shot jailbreaking technique and the relationship between number of shots and harmful response rate."},{"cited_title":"Measuring massive multitask language understanding","cited_arxiv_id":null,"evidence_quote":"Supplies the MMLU benchmark used to show that FTC scaling changes model capabilities."},{"cited_title":"Discovering language model behaviors with model-written evaluations","cited_arxiv_id":null,"evidence_quote":"Supplies the Model-Written Evaluations used to show that FTC scaling changes stances and behaviors."}],"review_version":1}