{"id":"96cef5c8-3969-4c2e-a5ba-0d48a990220e","arxiv_id":"2607.20469","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Decode-time hidden states contain a low-dimensional, cross-task shared subspace whose removal during decoding degrades decisions far more than matched random or prefill-derived controls.","lead":"This paper identifies a compact set of directions in a language model's internal states that many tasks use at the moment each answer token is chosen, and shows that removing only that subspace during decoding damages accuracy far more than removing random or prefill-derived directions. The result points to decode-time, KV-cached states—not prefill activations—as the regime where steering and interpretability should be evaluated.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'shared across tasks' claim is not established by the majority-threshold operationalization: the all-tasks intersection is empty for Llama-2, loosened thresholds produce sets that fail the scramble null, and the reported causal tests use only the default m=8 set.","rationale":"The reader's weakest assumption is that the protocol equates the 'shared decode channel' with PCA directions selected by a per-task variance threshold, and the reader flags the empty all-tasks intersection for Llama-2 and the threshold sensitivity. My re-reading of the paper confirms this is the most load-bearing concern. The central hypothesis is about a subspace shared across tasks, but the evidence for sharing is threshold-dependent: strict all-tasks sharing fails for one of four models, and relaxed thresholds produce sets that are not significant under the scramble null. The causal tests are not run on the strict all-tasks sets, so they cannot rescue the stronger claim. The energy-matched controls, LOTO checks, and patchback specificity are genuinely strong and support the existence of a causally important decode-time subspace, but they do not settle whether the selected set is 'the' shared channel rather than a threshold artifact. I therefore agree with the reader's conditional verdict rather than moving to accept or reject. The concrete test proposed would directly address the gap by checking whether the causal effect survives the strict all-tasks definition or any non-empty threshold-valid all-tasks set for Llama-2.","tokens_in":45158,"tokens_out":8660,"duration_ms":109725,"concrete_test":"Re-run the H2 decode-only ablation using the strict all-tasks shared set (m=13) at layer 10 for Qwen2.5-7B, Llama-3.1-8B, and Falcon-7B, where Table 16 reports non-empty sets. For Llama-2-7B, where the all-tasks set is empty at τ=10^-3, find the smallest τ that yields a non-empty all-tasks set while still passing both nulls, and test that set. Compare the shared-vs-control accuracy gap against the default m=8 results in Figure 7/Table 31. If the gap is not preserved in these strict all-tasks sets—or if no non-empty all-tasks set passes the scramble null for Llama-2—the reported causal effect depends on the majority threshold rather than on a genuinely task-shared subspace.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central hypothesis (Section 1) is that a small subspace S is 'shared across tasks' and causally relevant at decode time. The operationalization in §2.3 defines S_ℓ(τ,m) as PCA directions whose per-task relative variance exceeds τ for at least m of 13 tasks, with defaults τ=10^-3, m=8. Two facts from the paper show this does not actually establish a task-universal shared channel. First, Table 16 reports that under the strict all-tasks criterion (m=13), the shared set is empty for Llama-2-7B (|S|=0, p1=p2=1.0). Thus there is no direction that is above threshold for every task in that model, directly contradicting the literal 'shared across tasks' reading of the hypothesis. Second, the loosened-threshold check in Table 17 (τ=3×10^-4, m=8) makes the shared set expand to 23.8% and 21.4% of the PCA dimension for Llama-3.1-8B and Falcon-7B, yet these sets fail the orthogonal-scramble null (p2=1.0). So the statistical significance is confined to a narrow hand-set threshold band. The H2 causal evidence (Figure 7, Tables 30–31) is reported only for the default m=8 majority set; it does not show that a genuinely shared all-task subspace exists or retains the causal gap. Therefore the headline claim is not fully tested: the paper demonstrates that a threshold-selected set of PCA directions, shared by a majority of tasks, is causally important, but not that a natural, task-general shared decode subspace exists. The paper's own Appendix B.1.4 acknowledges this by reframing the claim as 'broadly reused across tasks,' but that is a weaker claim than the one in the abstract and introduction. The load-bearing gap is that strict all-tasks sharing fails on one model and the causal effect is not shown to be robust to the threshold choices that define 'shared.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DecodeShare, a protocol for identifying a low-dimensional subspace of LLM decode-time hidden states that is shared across tasks, and for testing its causal role under KV-cached inference. Decode-time hidden states are collected across tasks, pooled via PCA, and a 'shared' set S_l(τ,m) is selected as directions whose per-task relative variance exceeds τ for at least m of 13 tasks. The paper then performs decode-only projection removal of this subspace and compares against dimension- and energy-matched non-shared controls, with leave-one-task-out re-estimation and patchback. The main claims are (H1) such a shared structure exists beyond chance, (H2) removing it during decoding causes a large performance drop relative to controls, and (H3) prefill-estimated subspaces do not reproduce the decode-time causal effect. The paper also connects the decode-shared subspace to activation-steering interference and argues that decode-time validation ranks steering vectors better than prefill-based proxies.","tokens_in":45523,"tokens_out":3644,"duration_ms":47327,"significance":"If the central claim holds, the paper makes a useful protocol contribution: it provides a way to identify and causally test a compact, cross-task subspace directly in the KV-cached decode-time representation, rather than using prefill activations as a proxy. The empirical effort is substantial: energy- and dimension-matched controls, leave-one-task-out estimation, patchback with several specificity diagnostics, permutation/scramble nulls, multiple models and layers, and an open-source code link. The reported effects are often large (e.g., average LOTO generation accuracy 27.4 to 15.1 in Table 31) and patchback rescue rates are high with near-zero non-shared rescue, which is evidence in favor of direction-specific causal relevance. However, the significance is qualified by the mismatch between the hypothesis as stated ('shared across tasks') and the operationalization (majority of tasks under a hand-set threshold); the all-tasks intersection is empty for Llama-2, and a loosened threshold fails the scramble null. The paper is a strong candidate after the central claim is reframed or further supported.","major_comments":[{"comment":"The central hypothesis states that next-token decisions depend on a subspace S 'shared across tasks.' The operational definition, however, is a majority threshold: S_l(τ,m) with m=8 of 13 tasks. Table 16 shows that under the strict all-tasks criterion (m=13) the shared set is empty for Llama-2-7B (|S|=0, p1=p2=1.0), and App. B.1.4 explicitly reframes the claim as a 'compact core shared by many tasks.' This is load-bearing: all causal H2 tests use the default m=8 set, so they do not establish a genuinely task-universal channel. Combined with Table 17, where loosening τ to 3×10^-4 produces large sets that fail the scramble null for two models, the evidence supports only a threshold-dependent majority-shared subspace. Please either (a) report H2 causal results under the strict all-tasks set where it is nonempty, or (b) revise the abstract, hypothesis, and title to say 'shared by a majority","section":"§2.3, Table 16, App. B.1.4"},{"comment":"The paper claims a 'consistent performance collapse across tasks and models' (H2.1), but Table 7 shows StratQA with Dec-est 52.8 vs baseline 49.6, i.e., an improvement under decode-estimated shared-subspace removal, and Table 21 shows LogiQA 25.8 vs baseline 22.7, also an increase. These are not isolated rounding artifacts; they are per-task rows in the main evidence tables. The aggregate mean drops are real, but the word 'consistent' overstates the per-task pattern. Please report the number of tasks where the shared intervention is significantly harmful vs. unchanged/beneficial, and soften the claim accordingly or explain why these exceptions do not affect the central conclusion.","section":"Table 7, §3.2, Table 21"},{"comment":"There is a self-referential element: the subspace is selected by a variance-based sharedness criterion in pooled PCA, and then the causal test removes exactly that subspace. The energy-matched controls and nulls address the alternative that the effect is merely 'high-energy' or 'low-rank,' but they do not establish that the selected directions correspond to a natural, pre-existing task-general channel. The all-tasks empty set and the threshold sensitivity in Table 17 make this more than a philosophical concern. I would like to see either an independent operationalization of 'shared' (e.g., based on intersection of per-task bases, or a task-general probe) or an explicit limitation statement that DecodeShare characterizes the causal import of the threshold-selected subspace, not necessarily the unique 'shared decode channel.'","section":"§2.3, §3.1, App. B.1.5"}],"minor_comments":[{"comment":"The GitHub URL in the abstract is broken across a line: 'https://github.com/Zishan-Sha o/decodeshare.git' should be one continuous string.","section":"Abstract/URL"},{"comment":"Figures 5 and 16 are identical, and Figures 6 and 17 are identical. Please remove the duplicates in the appendix or cite them once.","section":"Figures 5/16 and 6/17"},{"comment":"The text references 'Appendix 6' for formal definitions, but no such appendix is present in the visible manuscript; the definitions appear in unnumbered text after Table 8. Please fix the cross-reference.","section":"§2.2 and §2.3"},{"comment":"The H1 p-values appear inconsistent across tables for the same model/setting: Table 10b reports p_perm <10^-3 and p_scr=0.0196 for Qwen2.5, while Table 11 reports p_perm ≤5.0×10^-4 and p_scr ≤5.0×10^-3. Clarify whether these are different null baselines or different runs.","section":"Tables 10b and 11"},{"comment":"p1 and p2 are used in Tables 16–17 without being defined in the main text; define them as permutation and scramble p-values at first use.","section":"Table 16/17 notation"}],"recommendation":"major_revision","confidential_remarks":"The paper is empirically rich and the controls are generally well designed, but the central terminology needs to be aligned with the actual operationalization. The all-tasks-empty result and the StratQA improvement are important caveats that the authors appear to acknowledge only in the appendix; they should be addressed in the main text. With a revised framing and transparent per-task reporting, the paper could be a solid contribution to the interpretability/steering literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a solid protocol paper with an overstated central claim. The decode-time causal effect is real and unusually well-controlled. Pooled PCA on KV-cached decode states, decode-only projection removal, energy- and dimension-matched controls, LOTO, and patchback are the right tools, and the paper uses them carefully. The prefill-vs-decode estimator swap is the most novel piece; the result that prefill-estimated bases fail to reproduce decode-time causal effects is worth taking seriously.\n\nThe aggregate evidence for a high-leverage decode-only subspace is strong: generation LOTO drops from 27.4 to 15.1, patchback rescue is dramatic, and non-shared controls stay near baseline. The Qout/Qcore split and the verbalizer-remap control are also good faith checks.\n\nNow the soft spots. The headline says 'shared across tasks,' but the operational definition is a majority of 13 tasks (m=8). Table 16 shows the strict all-tasks intersection is empty for Llama-2-7B. That is not a nitpick; it changes the claim. The paper's own Appendix B.1.4 walks it back to 'broadly reused across tasks,' but the abstract and introduction still say shared. Second, the significance is threshold-fragile: loosening tau to 3e-4 makes the shared set huge and it fails the scramble null on two of four models. So the causal story is only established for a narrow band of hyperparameters. Third, there are per-task exceptions that undercut the 'consistent collapse' language: in Table 7, StratQA improves when the decode-shared subspace is removed (49.6 to 52.8), and in Table 23 CSQA shows the opposite of H3 (prefill-estimated removal hurts more than decode-estimated). These are not fatal, but they matter.\n\nReproducibility is close but incomplete: code exists, but no pinned commit or one-command script, which is annoying for an explicitly protocol paper.\n\nBottom line: the paper deserves a serious referee. The measurements look honest and the controls are better than most in this literature. But the authors need to restate the hypothesis as 'a subspace shared by a majority of tasks,' report causal results for stricter sharing criteria, and show threshold robustness. If they do that, this becomes a useful reference for anyone doing steering or representation work under KV-cached serving.","headline":"Good protocol, overclaimed headline: the decode-time causal effect is real and well-controlled, but 'shared across tasks' holds only for a majority threshold, not literally across tasks.","tokens_in":46149,"tokens_out":1277,"would_cite":true,"duration_ms":18902,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Under KV-cached decoding, next-token decisions depend on a small subspace of the hidden state shared across tasks; removing it only at decode time collapses accuracy far beyond matched non-shared or prefill-estimated controls","keywords":["KV-cached decoding","decode-time hidden states","shared subspace","pooled PCA","causal ablation","activation steering","prefill-decode mismatch","LLM interpretability"],"falsifier":"Hold the decode-only intervention and energy-matched controls fixed but swap the shared-set rule for the strict all-tasks intersection: for Llama-2-7B that set is empty (reported in the appendix), so the protocol predicts no causal collapse where the majority rule produced drops of tens of points—whichever behavior the experiment matches would separate a genuine cross-task channel from a tuned threshold. The complementary check replaces the PCA-threshold estimator with a threshold-free or nonlinear one (e.g., canonical correlations across task state matrices) and asks whether decode-only remov","tokens_in":44996,"feed_emoji":"🔍","tokens_out":15312,"duration_ms":149687,"temperature":0.7,"pith_summary":"Under KV-cached decoding—the regime where a model has already read the whole prompt and then emits one token at a time over cached attention states—this paper claims that next-token decisions run through a small subspace of the hidden state that is shared across many tasks. It contributes a protocol that estimates this subspace from decode-time activations across 13 benchmarks, then ablates it only during decoding, leaving the prompt-processing prefill pass untouched. Removing the tiny shared set (a few percent of the layer width) collapses decision accuracy, while dimension- and energy-matched non-shared directions and prefill-estimated directions stay near baseline; restoring only the removed component rescues the lost decisions. The paper then draws practical consequences for activation steering: steering vectors overlap the shared channel, so projecting them off gives a robustness knob, and validating steering candidates at decode time ranks held-out utility better than prefill-based proxies. A reader should care because the claim, if right, replaces 'the decode-time state' with a concrete, compact, causally tested object that can be monitored or repaired without touching prefill.","feed_headline":"Remove ~3% of one layer and LLM decoding accuracy collapses","feed_subtitle":"It beats energy-matched controls; prefill-derived directions miss it, so validate steering at decode time.","key_machinery":"Three pieces carry the argument. First, the decode-aligned shared basis: pooled PCA of task-centered decode states gives a cross-task basis, and a per-task 'relative variance' threshold τ plus a 'used by at least m tasks' rule selects a compact shared set. Second, decode-only projection removal (h̃ = h − αQQᵀh) applied solely on single-token decode steps, prefill untouched, with dimension- and energy-matched controls that rule out a high-variance confound. Third, patchback: record the shared component during normal decoding, restore only that component in the ablated run, with random, energy-matched, and non-shared patches as specificity controls.","core_discovery":"The central claim: during KV-cached decoding, next-token decisions causally depend on a small subspace S of the decode-time hidden state shared across tasks. S comes from pooled PCA of task-centered decode states, keeping directions whose per-task relative variance clears a threshold for most tasks. Though a few percent of layer width, decode-only removal of S collapses accuracy far beyond dimension- and energy-matched controls; leave-one-task-out re-estimation keeps the effect, and patching back only the removed component rescues flips, even with donors from other tasks. The channel is decode-specific: prefill-estimated bases are near-orthogonal to it and behave like random controls.","pith_inferences":["A threshold-free or nonlinear re-estimation of shared structure—e.g., canonical correlation between per-task state matrices—would show whether the PCA-plus-threshold step is essential or merely a convenient probe; the paper's own sensitivity tables show shared-set size swings sharply with τ, so the causal signature under alternative estimators is the natural next test.","If the channel is as task-general as claimed, projection energy onto it at decode time could serve as a lightweight serving-time monitor for distribution shift or decision-regime failures—a use the paper's machinery makes immediate but does not propose.","Because prefill and decode shared subspaces are near-orthogonal, KV-cache compression and pruning choices validated on prefill statistics may be silently removing exactly the directions that matter at decision time; re-testing those methods with decode-only causal ablations transfers this protocol to the systems setting the paper cites but does not run."],"forward_implications":["Removing the shared decode subspace during decoding collapses accuracy across tasks and model families, while dimension- and energy-matched controls stay near baseline—the effect is direction-specific, not a matter of how much energy is removed.","Prefill-estimated shared bases are near-orthogonal to the decode-time channel and fail the same decode-only causal test, so estimates must be collected from the decoding regime they are meant to explain.","Steering vectors carry a substantial component in the shared channel; removing that overlap is an offline knob that typically trades a small average-effect loss for lower template sensitivity and fewer worst-case failures.","Decode-stage validation of steering candidates ranks held-out KV-cached utility better than prefill or mixed prefill/decode validation, with higher utility, lower flip rate, and lower regret for the vectors actually deployed.","The channel is internally structured: a 29-dimensional residual core carries the causal weight and encodes procedural and symbolic tokens, while a 3-dimensional readout slice is near-inert when ablated."],"fun_headline_variants":["Decode-time subspace is the key lever for LLM accuracy","Tiny shared subspace steers LLM decisions; prefill misses it","Remove 3% layer at decode and accuracy falls far below controls","Steering vectors work best at decode time, not prefill","LLM decode shares a small subspace that controls task decisions"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the task-general decision channel is exactly the small set of directions a hand-set variance cutoff picks out inside a common coordinate system built from the model's decode-time states; if that structure is not linear, or the cutoff mostly reflects the particular tasks and model used to set it, the experiments describe the chosen directions rather than a naturally occurring shared channel.","fun_headline_variants_meta":{"raw":{"variants":["Decode-time subspace is the key lever for LLM accuracy","Tiny shared subspace steers LLM decisions; prefill misses it","Remove 3% layer at decode and accuracy falls far below controls","Steering vectors work best at decode time, not prefill","LLM decode shares a small subspace that controls task decisions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1355,"prompt_tokens":711,"completion_tokens":644,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":556}},"tokens_in":455,"tokens_out":644,"duration_ms":6759,"temperature":1.0,"reasoning_tokens":556,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T13:21:21.309758+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold the decode-only intervention and energy-matched controls fixed but swap the shared-set rule for the strict all-tasks intersection: for Llama-2-7B that set is empty (reported in the appendix), so the protocol predicts no causal collapse where the majority rule produced drops of tens of points—whichever behavior the experiment matches would separate a genuine cross-task channel from a tuned threshold. The complementary check replaces the PCA-threshold estimator with a threshold-free or nonlinear one (e.g., canonical correlations across task state matrices) and asks whether decode-only remov","supporting_citations":[],"review_version":1}