{"id":"e796a1f7-8fa2-4a72-ab49-749fdfc9c88a","arxiv_id":"2505.22572","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Fusion Steering tunes per-prompt activation injections against ground-truth answers and reports 25.4% accuracy on the same 260 SimpleQA prompts, a gain that is compromised by circular evaluation.","lead":"Fusion Steering guides a 2B language model toward factual answers by injecting activation vectors built from the ground-truth answer plus a model explanation, with per-prompt weights tuned by Optuna. On 260 SimpleQA prompts where the base model failed, segmented steering lifts a composite accuracy from 3.5% to 25.4%, but tuning and evaluation share the same ground-truth labels, so the gain largely reflects fitting rather than generalization.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central accuracy comparison is compromised because per-prompt optimization (Eq. 5) and evaluation (Eq. 6) share the same ground-truth token-overlap signal on the same 260 prompts; without held-out evidence, the 25.4% vs. 3.5% claim does not establish a generalizable factual-accuracy improvement.","rationale":"The reader's weakest assumption correctly identifies the circular evaluation: per-prompt hyperparameter optimization uses ground-truth token overlap, and the same signal is the principal component of the evaluation metric on the same prompts. My independent reading of §2.2, §2.4, and §2.6 confirms that there is no held-out prompt set and no ablation that removes the ground-truth-derived reference activations. The strict SimpleQA rubric result is also weakened by subset selection on baseline INCORRECT labels and by the shared ground-truth signal. The paper is transparent about the requirement of ground-truth references in its limitations section, and it provides code and deterministic generation details; however, the central empirical claim of factual-accuracy improvement is unsupported as stated. Therefore the reader's REJECT verdict remains appropriate.","tokens_in":9507,"tokens_out":5129,"duration_ms":65847,"concrete_test":"Split the 260 failed prompts into a tuning set (e.g., 180) and a held-out set (e.g., 80). Use only the tuning set to set λ and the α/γ search bounds. On the held-out set, run two conditions: (A) the reported per-prompt Optuna protocol with reference activations from ground-truth answer plus explanation, and (B) an ablation where reference activations come from a model-generated explanation only (no ground-truth answer). If condition B collapses to baseline (≈3.5% composite, 0% SimpleQA), or if condition A on held-out prompts shows no advantage over fixed-parameter steering, then the reported gains are attributable to ground-truth leakage into the reference activations and the Eq. 5 objective, not to a generalizable steering mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result is not an uncontrolled experiment. In §2.4, α and γ are optimized per prompt with Optuna by maximizing Eq. 5, where token overlap with the ground-truth answer is the dominant term (λ = 0.7). In §2.6.1, Eq. 6 scores each output with 0.5·TokenOverlap + 0.5·(LLM Grade/5), using token overlap against the same ground-truth answer. The reference activations in §2.2 are captured from a prompt that explicitly contains the ground-truth answer and an explanation, so the injected steering signal already encodes the target. Reporting 25.4% vs. 3.5% on these same 260 prompts therefore compares a test-label-fitting procedure against a baseline that never sees the labels. The strict SimpleQA rubric result (0.0% → 13.1%) is not immune: the subset was selected because the baseline was already INCORRECT under that rubric, and the same ground truth drives both the token-overlap objective and the rubric grader. The paper's limitations section acknowledges that ground-truth answers are required at inference, but the abstract and conclusion still claim a general factual-accuracy improvement. The load-bearing assumption is that Eq. 5 optimization is a legitimate intervention rather than label leakage; as written, the evaluation cannot distinguish these.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Fusion Steering, an activation steering method for factual QA. It captures reference activations from a prompt containing the ground-truth answer and a model-generated explanation, then injects them across all transformer layers during inference, with per-prompt hyperparameters α and γ optimized via Optuna for full-layer and segmented (early/middle/late) configurations. On 260 SimpleQA prompts where the baseline Gemma-2-2B-IT produced incorrect answers, segmented steering is reported to improve a composite token-overlap/LLM-grade accuracy from 3.5% to 25.4% and strict SimpleQA-rubric correctness from 0% to 13.1%. The paper claims these results demonstrate improved factual accuracy.","tokens_in":9781,"tokens_out":4973,"duration_ms":55252,"significance":"If the result held on unseen questions, it would be a useful step toward inference-time activation control for knowledge-intensive QA, and the paper deserves credit for releasing code, using deterministic greedy decoding, and publishing all 260 outputs in supplementary files. However, the central empirical claim is not supported as stated because the per-prompt optimization and the evaluation share the same ground-truth token-overlap signal on the same 260 prompts, and the reference activations are derived from the ground-truth answer. The paper is best read as a study of answer-conditioned steering in an oracle setting; its general factual-accuracy claim requires additional evidence.","major_comments":[{"comment":"The Optuna objective in Eq. (5) maximizes λ·TokenOverlap − (1−λ)·Perplexity with λ=0.7, where token overlap is computed against the SimpleQA ground-truth answer, and Eq. (6) evaluates accuracy using 0.5·TokenOverlap + 0.5·LLMGrade/5 against the same ground-truth answer on the same 260 prompts. Therefore the reported 25.4% vs. 3.5% gain measures how well per-prompt hyperparameter optimization can make the steered output reproduce the known target tokens, not whether steering improves factual accuracy on questions whose answers are not given. To support the abstract's claim of improved factual accuracy, the authors need a held-out evaluation (e.g., optimizing on a subset and evaluating on a disjoint subset) or an optimization objective that does not use the ground-truth answer.","section":"§2.4, Eq. (5); §2.6.1, Eq. (6)"},{"comment":"The reference activations in Eq. (1) are computed from an enriched prompt containing 'Question: [prompt] Answer and Explanation: [ground truth + explanation]', so the injected signal already contains the answer to be retrieved. Section 5 correctly acknowledges that precise ground-truth answers are required at inference, but this contradicts the Section 1 statement that steering guides the model 'from the question alone.' As written, the method is an answer-conditioned steering oracle; the manuscript must reframe its contribution or add experiments for questions without known answers.","section":"§2.2; §1; §5"},{"comment":"The strict SimpleQA rubric result (0.0% to 13.1%) does not avoid the label-leakage problem. The 260-prompt subset was selected because the baseline was labeled INCORRECT by the same rubric-based Mistral grader described in Appendix A, and the same gold targets are used in reference-activation construction, in per-prompt optimization, and in the final rubric grading. Comparing the steered outputs against the training labels on a subset chosen because the baseline failed those labels is not evidence of generalization to new factual questions.","section":"§3.1; §2.1"},{"comment":"The paper states that statistical significance testing was not performed because decoding is deterministic; this conflates reproducibility with statistical generalizability. The 260 prompts are a sample from SimpleQA, and the question of whether the improvement generalizes across prompts is a statistical one. A paired McNemar test or bootstrap confidence interval over prompts would be straightforward and would strengthen the comparison.","section":"§3.2, Note"}],"minor_comments":[{"comment":"The accuracy thresholds (composite ≥ 0.6 and token overlap ≥ 0.5) are introduced without justification; because the evaluation is threshold-sensitive, please state how these thresholds were chosen and whether the conclusions are robust to nearby values.","section":"§3.1"},{"comment":"Figure 2 and Figure 3 appear out of numerical order in the text (Figure 3a is cited before Figure 2); please renumber figures and ensure all panels are referenced in order.","section":"Figures 2 and 3"},{"comment":"The same Mistral Small 3.1 instance and rubric used for initial incorrect-label selection are reused for the SimpleQA-rubric evaluation; please discuss potential grader bias or use a different grader for final evaluation.","section":"§2.6.2"},{"comment":"The limitations paragraph on the need for ground-truth answers at inference is a significant caveat; it should be stated in the abstract or introduction rather than only at the end.","section":"§5"},{"comment":"There are minor typographical issues, e.g., 'whether baseline, full-layer steering, or segmented steering—are produced' in §2.5 is missing a dash or comma; please proofread.","section":"§2.5"}],"recommendation":"reject","confidential_remarks":"This is a clear case where the headline result is an artifact of label leakage. The method may have value as an oracle-style study, but the manuscript's central claim is not supported as written. Rejection is appropriate; if the authors later add held-out experiments and reframe the contribution, a new submission could be considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me get straight to the point. The paper's central empirical claim collapses under the evaluation design. They tune α and γ per prompt by maximizing Eq. (5), which is essentially token overlap with the ground-truth answer minus perplexity. They then evaluate on the same 260 prompts with Eq. (6), which is half token overlap against the same ground truth. The reference activations also come from a prompt that explicitly contains the correct answer. In other words, segmented steering is fitting the test answers, and the baseline has no access to them. So the 25.4% vs 3.5% and 13.1% vs 0% numbers are the expected result of a per-instance function fit, not a measure of improved factual accuracy. The limitations section does say ground-truth answers are required at inference, but the abstract and conclusion still present this as a generalizable factual-accuracy improvement. That tension is the load-bearing flaw.\n\nWhat's worth crediting: the paper assembles full-layer and segmented steering, per-prompt Optuna tuning, and interpolation of original and steered activations into a coherent pipeline. The code and data are public, and greedy decoding makes the whole pipeline deterministic and reproducible. The qualitative examples, including the fluency breakdown case, are honestly discussed. They don't hide the failure mode. The topic distribution stratification check is also decent.\n\nThe soft spots beyond circularity: no held-out evaluation, no error bars or significance testing (determinism does not address optimizer variance), a small filtered subset, and dependence on an LLM grader that was also used to construct the subset. The comparison against a baseline that is forced to have 0% strict-rubric correctness inflates the apparent gain. The novelty is incremental—it combines known techniques without a new mechanism, and the per-prompt optimization is really a form of test-time adaptation with labels.\n\nWho is this for? If reframed as a diagnostic tool for studying how activation injection can force a target answer into a failing prompt, it might interest interpretability researchers. As a general factual-QA method, it doesn't hold.\n\nRecommendation: this should not go to peer review as is. The central claim is invalidated by label leakage. If the authors can add a held-out set (or at least a per-prompt tuning set that doesn't overlap the eval set) and reframe the contribution as calibration, it might become a footnote-worthy result. That's a substantial rewrite.","headline":"The paper's headline accuracy win is a test-label fitting artifact; per-prompt optimization and evaluation share the same ground-truth token overlap, so the central claim does not survive.","tokens_in":10318,"tokens_out":3808,"would_cite":false,"duration_ms":39535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fusion Steering claims that injecting per-prompt, layer-group-specific activation deltas derived from the correct answer and an explanation raises factual accuracy on previously failed SimpleQA questions from 3.5% to 25.4% without…","keywords":["activation steering","segmented steering","factual accuracy","question answering","inference-time intervention","per-prompt optimization","layer-group steering","SimpleQA"],"falsifier":"Split the 260 prompts into a tuning set and a held-out evaluation set. Optimize the per-prompt steering weights using only tuning-set ground-truth answers, then apply the same recipe to the held-out prompts. If held-out accuracy does not exceed the baseline's 3.5% composite rate (or 0.0% under the strict rubric), the in-sample gains are label-fitting, not a general steering effect.","tokens_in":9268,"feed_emoji":"🎯","tokens_out":7314,"duration_ms":73792,"temperature":0.7,"pith_summary":"The paper tries to establish that a language model's factual answers on questions it previously got wrong can be repaired at inference time by steering its internal activations. The steering signal is built from the model's own activations on a prompt that already contains the ground-truth answer and an explanation, then injected across all transformer layers with per-prompt, per-layer-group weights. On 260 SimpleQA prompts where the baseline was incorrect, this segmented steering lifts composite accuracy from 3.5% to 25.4% and fully correct answers from 0.0% to 13.1%. The method matters because it offers a fine-tuning-free route to factual control, with the caveat that the same ground-truth labels drive both the per-prompt parameter search and the final evaluation.","feed_headline":"Layer-group steering fixes 7x more failed facts","feed_subtitle":"On 260 questions the model got wrong, segmented activation steering lifts accuracy to 25.4% without fine-tuning.","key_machinery":"The key object is the per-layer reference activation vector $h_l$, the mean activation over tokens aligned with the ground-truth answer and explanation. The intervention combines additive steering, $\\Delta_l = \\gamma(h_l - \\mu_l)$ where $\\mu_l$ is the mean inference activation, with an interpolation step $\\alpha$ that fuses steered and original activations. Segmenting layers into early (0-8), middle (9-17), and late (18-25) groups and tuning $(\\alpha,\\gamma)$ separately for each group is what the paper credits for the accuracy gains.","core_discovery":"The central discovery is that segmenting the transformer into early, middle, and late layer groups, each with its own optimized fusion weight and steering strength per prompt, produces larger factual-accuracy gains than either no steering or a single global steering vector. Reference activations are captured as the per-layer mean activation over the ground-truth answer and explanation tokens; at inference the model sees only the question, and the shift $\\Delta_l = \\gamma(h_l - \\mu_l)$ is added to every token's activation and then blended with the original activation through $\\alpha$. The paper reports that this segmented, prompt-specific injection raises accuracy on 260 previously failed SimpleQA prompts from 3.5% to 25.4% under a composite token-overlap/LLM-grade metric, and from 0.0% to 13.1% under a strict correct/incorrect rubric.","pith_inferences":["Because the per-prompt parameter search and the final evaluation both score token overlap against the same ground-truth answers, the reported gains are in-sample and likely overstate generalization to unseen questions.","A fair test would tune $(\\alpha,\\gamma)$ on one set of prompts and evaluate on a disjoint set; until that is done, the method should be viewed as label-conditioned repair rather than general factual improvement.","The method could be adapted to open-domain settings by replacing the ground-truth-based reference activations with activations derived from a retrieved candidate answer, turning the approach into a retrieval-steered corrector.","The fluency collapse observed in one example suggests adding a repetition or coherence penalty to the optimization objective would reduce the worst-case quality degradation and might make the gains more usable."],"forward_implications":["Per-prompt, per-layer-group activation steering can correct a substantial fraction of factual errors in a 2B-parameter model without any weight updates.","Segmented steering outperforms both the unsteered baseline and full-layer uniform steering on composite accuracy, token overlap, and strict SimpleQA grading.","Steering trades fluency for factual alignment, and can produce degenerate repetitions such as a stream of '7' tokens even when the correct answer is reached.","Because the method operates entirely at inference time on modest hardware and 8-bit weights, it is presented as a low-resource alternative to fine-tuning for targeted question repair.","The requirement of ground-truth answers to build reference activations means the method repairs known questions rather than correcting open-domain queries on its own."],"supporting_citations":[{"why":"Supplies the SimpleQA benchmark and its ground-truth answers that define the task and drive both optimization and evaluation.","marker":"[18]"},{"why":"Defines the base model, Gemma-2-2B-IT, whose activations are captured and steered throughout the experiments.","marker":"[19]"},{"why":"Provides the LLM-based grader used to classify baseline failures and to assign correctness and quality scores in evaluation.","marker":"[20]"},{"why":"Supplies the hyperparameter search algorithm that tunes the fusion weight and steering strength per prompt.","marker":"[21]"},{"why":"Provides the perplexity measure used in the optimization objective as a fluency proxy.","marker":"[22]"},{"why":"Supports the claim that transformer layers form functional groups, motivating the early/middle/late segmentation.","marker":"[23]"},{"why":"Represents the prior activation-steering approach that Fusion Steering contrasts with and extends to multi-layer, prompt-specific control.","marker":"[7]"}],"fun_headline_variants":["Segmented steering lifts failed QA accuracy from 3.5% to 25.4%","Layer-group steering: 0% to 13.1% correct answers on hard prompts","Prompt-specific activation control fixes 7x more QA facts","Steering by layer groups outperforms full-layer and baseline","Segmented activation steering raises QA accuracy without fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that optimizing the steering weights per prompt against ground-truth token overlap on a question, and then scoring that same question with token overlap, measures the method's ability to improve factual accuracy rather than its ability to fit the test labels.","fun_headline_variants_meta":{"raw":{"variants":["Segmented steering lifts failed QA accuracy from 3.5% to 25.4%","Layer-group steering: 0% to 13.1% correct answers on hard prompts","Prompt-specific activation control fixes 7x more QA facts","Steering by layer groups outperforms full-layer and baseline","Segmented activation steering raises QA accuracy without fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000571,"raw_usage":{"total_tokens":2737,"prompt_tokens":1016,"completion_tokens":1721,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":1626}},"tokens_in":632,"tokens_out":1721,"duration_ms":15558,"temperature":1.0,"reasoning_tokens":1626,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:03:03.538452+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Split the 260 prompts into a tuning set and a held-out evaluation set. Optimize the per-prompt steering weights using only tuning-set ground-truth answers, then apply the same recipe to the held-out prompts. If held-out accuracy does not exceed the baseline's 3.5% composite rate (or 0.0% under the strict rubric), the in-sample gains are label-fitting, not a general steering effect.","supporting_citations":[{"cited_title":"Gemma 2: Improving open language models at a practical size, 2024","cited_arxiv_id":null,"evidence_quote":"Defines the base model, Gemma-2-2B-IT, whose activations are captured and steered throughout the experiments."},{"cited_title":"Mistral small 3.1, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the LLM-based grader used to classify baseline failures and to assign correctness and quality scores in evaluation."},{"cited_title":"Language models are unsupervised multitask learners, 2019","cited_arxiv_id":null,"evidence_quote":"Provides the perplexity measure used in the optimization objective as a fluency proxy."},{"cited_title":"Transformer layers as painters, 2025","cited_arxiv_id":null,"evidence_quote":"Supports the claim that transformer layers form functional groups, motivating the early/middle/late segmentation."}],"review_version":1}