{"id":"d87590c2-2674-42da-889d-e4b388a29bf8","arxiv_id":"2509.22854","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ICR extracts shared attention directions from in-context learning and routes them at inference time, enabling zero-shot reuse across tasks.","lead":"Researchers trained a small router that adjusts a frozen large language model's attention using patterns extracted from a few labeled examples, so the model behaves as if it had seen those examples without putting them in the prompt. The approach generalizes to new tasks without retraining, beating earlier implicit in-context learning methods on out-of-domain benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"OOD evaluation uses only close variants of training tasks, so 'diverse new tasks' claim is unsupported","rationale":"The reader's weakest assumption was theoretical: that domain-specific directions cancel in pooled PCA. I find a more load-bearing vulnerability in the empirical scope of the OOD evaluation. The central claim is that ICR generalizes to 'diverse new tasks,' but all seven OOD datasets are close relatives of the five training domains—same language, same classification/QA format, similar label spaces. Consequently, the observed OOD gains may be explained by task-family overlap rather than by recovery of a domain-general ICL subspace. This concern directly undermines the strongest claimed contribution. I do not think it warrants rejection; the method still shows promise on the tested tasks, and the theoretical framework could be sharpened. However, it reinforces the reader's CONDITIONAL verdict: the paper should either temper the 'diverse new tasks' language or add a genuinely varied OOD benchmark. Since the reader already set CONDITIONAL, the verdict is unchanged. I disagree with the reader's choice of weakest assumption because the spiked-covariance cancellation, even if imperfect, is not as damaging as the lack of evidence for true task diversity.","tokens_in":24877,"tokens_out":6647,"duration_ms":66787,"concrete_test":"Train ICR exactly as in the paper (PIDs from AGNews/SST-2/TREC/CSQA/PIQA; router on 25k queries). Evaluate on held-out tasks from unrelated families: CoNLL-2003 NER (sequence labeling), MathQA (math word problems), HumanEval (code completion, multiple-choice formatted), and a non-English sentiment benchmark (e.g., Spanish TASS). Compare zero-shot vs ICR accuracy on these tasks. If the mean gain over zero-shot is not clearly positive (or is below the mean gain reported on the paper's 7 OOD datasets), the 'train once, reuse everywhere' claim fails for genuinely diverse tasks.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Sec. 4.2 that ICR is 'the first implicit ICL method that can be directly adopted for zero-shot inference in diverse new tasks' rests on the seven OOD datasets in Table 1. But these datasets are not diverse in any strong sense: SST-5 and MR are sentiment analysis tasks essentially identical to training domain SST-2; MRPC and CB are English sentence-pair classification; COPA, CREAK, and AI2SciE are multiple-choice commonsense/science QA directly analogous to training domains CSQA and PIQA. Thus the evaluation only shows interpolation across a narrow family of English classification/QA tasks. It never tests reuse on structurally different tasks such as sequence labeling, code generation, math word problems, or non-English text. The theoretical argument in Sec. 2.3 (Eq. 6) requires domain-specific variations to cancel in pooled PCA, but with D=5 domains drawn from the same task families, the 'diversity' assumed by the theory is untested. If ICR only helps tasks that share label format and reasoning style with the training set, the claimed 'generalizable ICL pattern' may simply reflect task-family overlap rather than a reusable attention-routing mechanism.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes In-Context Routing (ICR), an implicit in-context learning method that operates on attention logits rather than on residual-stream vectors. ICR first extracts Principal ICL Directions (PIDs) by applying PCA to last-token query and key projections collected from explicit ICL prompts across five training domains, then trains a query-conditioned router with a small MLP to produce low-rank, head-gated additive biases for the attention logits of the last third of layers. The router is trained on zero-shot queries with cross-entropy, confidence-alignment, and sparsity losses while the backbone is frozen. The paper evaluates ICR on five in-domain and seven out-of-domain datasets using Llama2-7B, Qwen2.5-7B, and Llama3.1-8B, reporting consistent gains over prior vector-based implicit ICL baselines and claiming a train-once-and-reuse framework that generalizes to diverse new tasks without retrieval or retraining.","tokens_in":25214,"tokens_out":5110,"duration_ms":42795,"significance":"If the central claims hold, ICR is a useful step beyond vector-based implicit ICL: it moves the intervention into the attention geometry, makes the routing query-conditioned, and shows evidence that low-rank PCA directions extracted from multiple domains can be reused without per-task retraining. The paper has clear strengths: evaluation across 12 datasets and three LLMs, comprehensive ablations (PCA rank, random bases, loss components, domain alignment, routing layers), efficiency analysis, and released code. The main weaknesses are (i) the theoretical justification for why PIDs generalize is informal and rests on an unproven cancellation assumption, (ii) the reported numerical margins are sometimes very small with no error bars or significance tests, and (iii) the 'diverse new tasks' claim is not supported by the actual OOD benchmark, which consists of close variants of the training task families.","major_comments":[{"comment":"The claim that ICR 'consistently outperforms' baselines is not backed by inferential statistics. The setup says each experiment is run with three seeds and averages are reported, but no standard deviations, confidence intervals, or significance tests are given. Several reported differences are tiny or even reversed: in Table 1 on Llama2-7B, SST-2 is 86.4 for both ICR and M2IV, and CSQA is 24.8 for both; on Qwen2.5-7B, TREC is 70.6 for ICR vs 70.8 for M2IV. The collapse counts (0 vs 2-4) could flip with a single seed. Please report per-seed results or at least error bars/paired tests for the headline comparisons.","section":"Sec. 4.2 / Tables 1 and 7"},{"comment":"The theoretical argument is load-bearing because it justifies PIDs as reusable directions, but Eq. (18) is stated without derivation. The term ρ_D is never defined, and the assertions that the numerator decreases with D and that gap_Q increases with N and D are not proven. Moreover, Eq. (6) already assumes that the domain-specific terms B_{q,d}Γ_{q,d}B^T_{q,d} 'average out toward isotropy'; this is precisely the condition that must hold for pooled PCA to recover the shared subspace, yet no diagnostic or experiment verifies it for the five chosen domains. Please provide a formal derivation or clearly frame the argument as a heuristic supported by the MATCHED-5 vs MATCHED-3 experiment.","section":"Sec. 2.3 / Appendix A.3, Eq. (18)"},{"comment":"The abstract and Sec. 4.2 describe the evaluation as spanning 'diverse new tasks' and claim that ICR is directly adoptable for 'zero-shot inference in diverse new tasks.' The seven OOD datasets do not support this wording. SST-5 and MR are sentiment classification, near-identical to training-domain SST-2; MRPC and CB are sentence-pair classification; COPA, CREAK, and AI2SciE are multiple-choice comprehension/reasoning tasks directly analogous to CSQA and PIQA. There are no structurally different tasks such as sequence labeling, generation, math/code, or non-English text. The evidence therefore supports interpolation across task-family variants, not generalization across diverse task types. Either add structurally out-of-domain tasks or limit the claim accordingly.","section":"Sec. 4.2 and Sec. D.3.1"}],"minor_comments":[{"comment":"The 'ICLness' token analysis is meant to support interpretability, but many of the top-50 tokens are subword fragments or generic tokens (e.g., dep, forme, fro, bro, catt, cyl, duch) rather than clearly reasoning-related words. The 'highlighted in red' formatting is lost in the text, and no quantitative comparison against a random or function-word baseline is provided. Please substantiate the claim that the selected tokens are especially tied to ICL.","section":"Appendix H / Table 12"},{"comment":"Equation (3) writes ΔA^l ∈ R^{T×T} as a layer-level bias shared across heads, while Eq. (10) applies a head-specific gate γ_{l,h}. Please clarify the dimensionality and broadcasting of the bias, especially how the shared low-rank term is combined with per-head gates.","section":"Eq. (3) vs Eq. (10)"},{"comment":"The score definition uses 'stability_v = mean_v / (std_v + ε)' but ε is never specified. Please state its value or remove it.","section":"Sec. 5.1 / Appendix H"},{"comment":"The right panel showing Spearman correlations has small, overlapping labels and the cluster structure is hard to read. Consider a larger figure or a dendrogram-style plot.","section":"Figure 4"},{"comment":"The paper reports using a single V100 GPU and deterministic settings, which is good for reproducibility, but the wall-clock time for PIDs extraction and router training is not reported. Including it would strengthen the efficiency discussion.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's contribution is plausible and the experiments are extensive, but the central generalization claim is currently broader than the evidence. The theoretical section also needs to either be made rigorous or repositioned as a heuristic. I would ask for revisions along the major comments before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: ICR is a real step beyond residual-stream shift vectors. Routing attention via PCA-extracted Q/K directions with a query-conditioned router is new as far as I know, and the paper earns its main claim: it outperforms vector-based implicit ICL baselines on ID and on the OOD sets tested, with no collapse below zero-shot.\n\nBest parts: clean mechanism, sensible ablations (random basis, rank, layer placement), three LLMs, twelve datasets. The random-basis ablation matters: it shows the PCA direction content, not just low-rank bias, drives OOD transfer. The router conditioning ablation shows query-dependent allocation helps. That is honest supporting evidence.\n\nSoft spots: (1) No error bars or significance tests anywhere. Three seeds are averaged; we do not know whether +3.0/+6.5 points is stable. (2) Theory is sketchy. Eq. 6 assumes domain-specific subspaces cancel in the pooled covariance; that is an assumption, not a result. Eq. 18 is stated without derivation, and the claim that the eigengap grows with N and D is hand-wavy. It reads like intuition, fine, but it does not do the work. (3) The OOD label is generous. The seven OOD sets are near variants: SST-5/MR from SST-2, sentence-pair and MCQ families from training. No sequence labeling, generation, non-English, or genuinely different structure. So 'diverse new tasks' (abstract, Sec. 4.2) overstates what the data shows. The method may still transfer broadly; the paper just has not shown it.\n\nThe stress-test note is right about the narrowness, but it does not sink the paper. ICR's advantage over baselines on these variants is still evidence of cross-task pattern reuse, not just label-format interpolation. The claim should be softened to 'close variants' and tested on harder OOD. The \"ICLness\" token analysis is a minor post-hoc curiosity; the top-50 list includes French/German fragments, so it is not a clean story, but the paper acknowledges the pitfall and the point is not load-bearing.\n\nWho it is for: people working on implicit ICL and representation engineering, not the broad LLM training community. The efficiency story (fewer cached params, faster than few-shot) is useful.\n\nRecommendation: send to peer review. The mechanism deserves referee time, and the experiments are reproducible enough (code provided, deterministic settings, single V100). Request error bars, a derivation or a clear statement of assumptions for the PCA claim, and a harder OOD set before acceptance.","headline":"A genuinely new mechanism for implicit ICL—attention logit routing—with solid but narrow OOD evidence; the 'diverse new tasks' claim outruns the evaluation.","tokens_in":25661,"tokens_out":1982,"would_cite":false,"duration_ms":21492,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One training run can equip an LLM to do in-context learning on new tasks at zero-shot cost.","keywords":["implicit in-context learning","attention routing","principal ICL directions","attention logits","zero-shot generalization","large language models","low-rank bias","out-of-distribution transfer"],"falsifier":"Run pooled PCA on Q/K projections from two domains that share a strong spurious direction (e.g., both tasks are corrupted by the same input feature), then remove one domain and re-run PCA: if the top-r subspace shifts substantially, the shared-subspace assumption fails. Testing ICR on such paired domains should show its OOD gain disappear, while a per-task vector baseline remains competitive.","tokens_in":24804,"feed_emoji":"🧭","tokens_out":3106,"duration_ms":27656,"temperature":0.7,"pith_summary":"The paper proposes In-Context Routing (ICR), an implicit in-context learning method that captures a generalizable ICL pattern in the attention logits of large language models. Instead of injecting task-specific shift vectors into residual streams, ICR extracts low-dimensional 'Principal ICL Directions' from query and key projections across multiple domains, then uses a small trainable router to add a query-conditioned low-rank bias to attention logits during zero-shot inference. This enables a train-once-and-reuse framework: after one round of training on a few domains, ICR transfers directly to new, even out-of-domain, tasks without retrieval or retraining. If correct, this would give LLMs few-shot-like performance at zero-shot cost while avoiding the brittleness and task-specificity of prior implicit ICL approaches.","feed_headline":"One training run steers LLMs on new tasks via attention routing","feed_subtitle":"Trained once on five tasks, a small router adds a low-rank bias to attention logits, giving zero-shot ICL gains on unseen tasks.","key_machinery":"The core mechanism is the PID-based attention routing: at each layer, a low-rank bias ΔA = γ·(Q U_q) diag(α) (K U_k)⊤ is added to the attention logits, where U_q and U_k are the top-r PCA directions of pooled last-token query/key projections (the Principal ICL Directions), γ is a per-head gate, and α is a per-layer routing vector produced by a small two-branch MLP conditioned on a frozen text-encoder representation of the query. This reparameterizes the attention kernel as I + U_q diag(α) U_k⊤, confining the intervention to a low-dimensional subspace of the query/key geometry.","core_discovery":"ICR shows that the structural pattern underlying in-context learning can be extracted from attention projections rather than residual states. By pooling last-token query and key projections from ICL prompts on multiple domains and taking their top principal components, the method obtains reusable Principal ICL Directions (PIDs). A query-conditioned router then composes these directions into a low-rank bias added to attention logits, effectively reparameterizing the attention kernel. The paper argues this is the first implicit ICL method that can be directly adopted for zero-shot inference in diverse new tasks without retrieval or retraining, and reports consistent gains over vector-based bas","pith_inferences":["If PIDs are truly domain-agnostic, the paper suggests that in-context learning in LLMs has a universal geometric signature in query/key space; one could attempt to transfer PIDs across models of different sizes or families, potentially amortizing the extraction step further.","The spiked covariance assumption implies a testable prediction: the top-r eigen-directions of pooled Q/K covariance should be stable under domain removal, and the eigengap should increase with the number of domains. Measuring this directly would indicate how many and which domains are needed for generalizable routing.","The router's input-conditioning opens a possible route to on-the-fly task identification: routing vectors might cluster by task type, and feeding an unlabeled query from a novel task could be used to infer which ICL direction to emphasize, without any labeled examples.","One could extend ICR to continual learning scenarios: as new tasks arrive, update only the router and possibly the PID subspace via incremental PCA, preserving earlier capabilities while adding new ones—this would test whether the shared ICL subspace truly remains stable under distribution shift."],"forward_implications":["ICR outperforms prior implicit ICL baselines that require task-specific retrieval or training, on both in-domain and out-of-domain datasets, and shows no performance collapse below zero-shot on any evaluated task.","ICR closely matches and sometimes surpasses few-shot prompting on in-domain tasks while using zero-shot prompts and cached PIDs plus a small router, making inference cheaper than explicit few-shot prompting.","The framework is train-once-and-reuse: one round of training on five domains transfers to seven unseen domains, including tasks with different label spaces, formats, and reasoning demands.","Ablations show that OOD robustness depends on the PCA-meaningful directions, not just low-rank routing; random orthogonal directions retain in-domain accuracy but collapse on OOD tasks.","The router's query-conditioned modulation is interpretable: it consistently upweights tokens related to reasoning and structural explanation across datasets, and layer/head/PID importance profiles are shared across semantically related tasks."],"fun_headline_variants":["Attention routing: train once, zero-shot ICL on new tasks","Reusable attention directions enable zero-shot ICL","Train a router, steer attention, generalize ICL","Attention logits routing yields generalizable ICL","One-time router training, universal ICL reuse"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that domain-specific query/key variations cancel out when covariances are pooled, so that PCA on multi-domain projections recovers a shared, transferable ICL subspace; if those variations align instead, the extracted PIDs will not generalize to new tasks.","fun_headline_variants_meta":{"raw":{"variants":["Attention routing: train once, zero-shot ICL on new tasks","Reusable attention directions enable zero-shot ICL","Train a router, steer attention, generalize ICL","Attention logits routing yields generalizable ICL","One-time router training, universal ICL reuse"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000753,"raw_usage":{"total_tokens":3191,"prompt_tokens":755,"completion_tokens":2436,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":2374}},"tokens_in":499,"tokens_out":2436,"duration_ms":15180,"temperature":1.0,"reasoning_tokens":2374,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T14:46:35.288779+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run pooled PCA on Q/K projections from two domains that share a strong spurious direction (e.g., both tasks are corrupted by the same input feature), then remove one domain and re-run PCA: if the top-r subspace shifts substantially, the shared-subspace assumption fails. Testing ICR on such paired domains should show its OOD gain disappear, while a per-task vector baseline remains competitive.","supporting_citations":[],"review_version":1}