{"id":"868c42fe-4244-47d0-9d56-46ae00e3e352","arxiv_id":"2505.12584","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A single learned steering vector added to one transformer layer improves multilingual task performance without fine-tuning, and transfers between related languages.","lead":"The paper shows that adding a single learned vector to the internal activations of a large language model, at just one layer, improves how the model answers questions in other languages, with no fine-tuning needed. The trick beats most baseline approaches and roughly matches Google Translate on several models, at a fraction of the cost.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main comparison may be inflated by selection of steering layer on test benchmarks; the layer choice for Table 2 is never disclosed despite strong layer sensitivity.","rationale":"After reading the paper, the central empirical claim is not the exact DPO loss but the aggregate superiority of one-layer additive steering. The most fragile part is not the capacity of the vector, which the authors explicitly concede, but the experimental control on the layer hyperparameter. Table 2's Base, Google, NLLB, DPO, and MSE rows are exactly recoverable from Table 6's per-model column means, but the SFT row is not, so there is additionally a reproducibility defect in one baseline. However, even if the SFT row were corrected, the headline margins over SFT, NLLB, and the closeness to Google all depend on the layer chosen. Section 6.1 shows strong layer sensitivity, so without a disclosed, test-blind selection protocol the reported results are not yet trustworthy. The reader already flagged this as a load-bearing premise, and I agree. I would keep the verdict at CONDITIONAL rather than REJECT because the issue is addressable by disclosure and a validation-based re-run, and the per-model table suggests the effect is not entirely an artifact. The missing Wang et al. baseline is relevant to the novelty claim but secondary to whether the intervention works.","tokens_in":18542,"tokens_out":13935,"duration_ms":142409,"concrete_test":"Ask the authors to disclose the layer index and selection rule for every model and language in Table 2. Then re-run the DPO and MSE steering protocols with L chosen on a held-out validation split that is not used for any test-benchmark decision, for example reserving 20% of each language's training corpus for layer selection and evaluating only on the untouched MGSM, XNLI, XCOPA, M3Exam, and MMLU sets. Compare the validation-selected layer with the reported layer and recompute the Table 2 aggregates. If the validation-selected layer differs, or the DPO-vs-baseline margins shrink materially (for instance, the SFT or NLLB gain drops by more than a few points, or the Google gap widens), the central claim is inflated. This single check settles whether the undisclosed layer choice was effectively a test-set selection.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's strongest claim is that adding one learned vector to one layer's residual stream (Eqs. 4 and 6) beats SFT, 5-shot, XLT, NLLB, and approaches Google Translate (Table 2). This claim requires a fixed choice of layer L for each model. But the main experiments never state which layer produced Table 2, how L was chosen, or whether a validation split was used. Section 6.1 shows the effect is sharply layer-dependent, with Aya23 best in early layers, Llama2 best in early/middle layers for low-resource languages, and final layers useless. If L was picked after inspecting the same benchmarks reported in Table 2, the reported margins (DPO vs SFT, vs NLLB, and the narrow Google gap) are upper bounds from test-set selection rather than estimates of the method with a pre-registered or validation-selected layer. The Limitations section concedes the vector is a fixed linear correction, so the sweet-spot layer is precisely a hyperparameter that can move results substantially. This is load-bearing because it affects every entry in Table 2, not one baseline, and the mechanistic interpretation that one layer captures fine-tuning-like alignment depends on the intervention rather than on the selection process.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a lightweight multilingual intervention: for each target language, a single steering vector is learned from paired English/non-English corpora (MSVAMP for math, Tatoeba for general text) and added to the residual stream at one selected layer during inference (Eqs. 4 and 6). The method is evaluated on five open-source LLMs (LLaMA2-7B, Aya23-8B, LLaMA3-8B, Gemma-7B, Qwen1.5-7B) across MGSM, XNLI, XCOPA, M3Exam, and MMLU, against seven baselines including basic prompting, Google Translate, NLLB, 5-shot ICL, XLT, and SFT. The authors report that DPO-based steering improves over SFT by about 26.7%, over 5-shot ICL by about 38.8%, over NLLB by about 25.4%, and approaches Google Translate within about 3%. They also analyze layer sensitivity, cross-lingual transfer of steering vectors, and internal alignment via logit lens, concluding that single-layer steering can reproduce part of the effect of multilingual fine-tuning.","tokens_in":18720,"tokens_out":10337,"duration_ms":96815,"significance":"If the empirical claims hold, the paper is a useful contribution to multilingual representation engineering: a single fixed additive vector at one layer can recover a meaningful portion of the gains from full multilingual fine-tuning, at much lower computational cost, and the layer-dependence results are informative for mechanistic interpretability. The paper is commendably transparent in its Limitations section about the fixed linear direction, the cost of layer sweeping, and the distribution sensitivity of the method, and it provides per-model tables (Table 6) that make the aggregate claims checkable. The main risk is experimental protocol: the headline results are trustworthy only if the steering layer was chosen on a proper validation split and if the DPO objective is specified consistently; both points currently require clarification.","major_comments":[{"comment":"The headline results in Table 2 do not state which layer L was used for each model or language, nor how L was chosen. Section 6.1 shows that the effect of steering is sharply layer-dependent: Aya23 benefits most from early layers, LLaMA2 from early and middle layers for low-resource languages, and final layers give little or no improvement. If L was selected by inspecting the same benchmarks reported in Table 2, the reported margins over SFT, NLLB, and the 3.08% gap to Google Translate are upper bounds from test-set selection rather than estimates of the method with a pre-specified or validation-selected layer. The authors should disclose L for every model and language group, choose L on a held-out validation split (for example, a portion of MSVAMP/Tatoeba not used for training the vector), and report the sensitivity of Table 2 to the choice of L, such as the mean and standard deviation over layers or over a small set of pre-registered layers.","section":"Section 6.1 and Table 2"},{"comment":"The paper defines the DPO objective in Eq. (7) with a reference-model probability ratio, a temperature beta, and a sign flip d, but Algorithm 1 (line 12) optimizes L_DPO = -E_t log sigma(ell_en_t - ell_t), which contains no reference model, no beta, and no d, and treats logits as if they were probabilities. These two objectives are not equivalent, so the learned steering vector is not reproducible from the text as written. Please provide a single, precise loss with all terms defined, and clarify how the logits ell and ell_en are obtained from the altered and English hidden states, including whether the model is run from layer L+1 onward after the intervention.","section":"Appendix A, Eq. (7) versus Algorithm 1"},{"comment":"The additive intervention is underspecified at the sequence level. The paper defines token-wise residual activations x_i^(l), but Eqs. (4) and (6) add a single vector v to 'x_target', and Algorithms 1-2 call HiddenState(M,q,L) and add v to the whole hidden state. If v is added to every token position, the training pairs (q, q_en) have different lengths and the loss must specify a per-position or pooled alignment between the two sequences. If v is instead added to a pooled representation, that contradicts Figure 1, which states that v is added to each token's residual stream. The authors should specify the exact tensor shape, the pooling or alignment procedure, and how the altered hidden state is propagated through the remaining layers.","section":"Section 3, Eqs. (4)-(6) and Algorithms 1-2"},{"comment":"The MGSM result may overestimate the method because the math steering vector is trained on MSVAMP, a multilingual math-word-problem corpus that is distributionally very close to MGSM. The paper itself shows in Section 5.2 and Figure 8 that steering gains are largest when the test distribution closely matches the training distribution, so the MGSM row in Table 2 is not an out-of-domain test of the method. The authors should quantify the overlap between the MSVAMP training subset and MGSM, for example by checking for duplicate or near-duplicate questions, and should report MGSM results with a vector trained on Tatoeba only, or on another non-math corpus, as a robustness check.","section":"Section 4.1, training and evaluation datasets"},{"comment":"The 5-shot and XLT baselines produce implausibly low scores that are inconsistent with typical behavior of these methods, for example Aya23 XLT on XCOPA scores 12.1 versus 81.6 for the basic prompt, and LLaMA2 5-shot on MGSM scores 12.2 versus 19.6 for the basic prompt. Because one headline claim is a 38.8% improvement over 5-shot ICL, the construction of these baselines must be described in detail, including the number and language of the examples, the prompt templates, and how the translation step is performed for XLT. As reported, the comparison to 'competitive baselines' is not yet credible.","section":"Section 4.2 and Table 6"}],"minor_comments":[{"comment":"The caption says 'Three models' but the table contains five models (Aya23-8B, LLaMA2-7B, LLaMA3-8B, Gemma-7B, Qwen1.5-7B); correct the caption and state the language set used for each row.","section":"Table 6 caption"},{"comment":"Table 1 and Table 6 report different MGSM averages for the same models, for example LLaMA2-7B Basic Prompt is 13.4 in Table 1 but 19.6 in Table 6; clarify whether these use different language subsets or different task configurations.","section":"Tables 1 and 6"},{"comment":"The caption refers to 'Aya32-8B' and should read 'Aya23-8B'.","section":"Figure 4 caption"},{"comment":"Values such as '.36' and '.01' should be written as '0.36' and '0.01' for readability.","section":"Table 1"},{"comment":"The notation Logits(M, ~h) is undefined; specify whether the model is run from layer L+1 onward with the altered hidden state replacing the residual stream at that layer.","section":"Algorithm 1"},{"comment":"The term 'probing' is used to describe steering in the Limitations section and elsewhere; since probing usually means reading or analyzing representations, clarify the terminology to avoid confusion.","section":"Throughout"},{"comment":"Spelling of the model name is inconsistent ('LLama2', 'Llama2', 'LLaMA2'); choose one convention and use it consistently.","section":"Throughout"},{"comment":"The symbol d is used both as a direction coefficient and as the hidden dimension; use a different symbol, such as s, for the direction coefficient.","section":"Appendix A, Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"I want to flag to the editor that the undisclosed layer-selection protocol is the key risk in this paper. If the authors cannot provide a validation-based layer selection procedure, or code that shows how the layer was chosen, the headline comparison should be regarded as a search over the test benchmarks. I would ask for the code, the exact layer choices per model, and the full DPO loss specification before any further decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The paper has a genuinely interesting idea—a single fixed steering vector added to one layer's residual stream—and the per-model tables give it some empirical support. But the headline comparison is not reliable as stated. Table 2's SFT row does not match Table 6: I re-averaged Table 6's SFT numbers per task and got (40.5, 68.7, 55.9, 47.8, 43.2), while Table 2 lists (34.7, 55.2, 46.1, 39.9, 35.4). Every other method (Base, Google, NLLB, 5-shot, XLT, DPO, MSE) is consistent between the two tables. So either the SFT row is from a different run or it's a typo, and it's the exact row that supports the 26.7% claim. With the consistent numbers, DPO beats SFT by about 4–5%, not 26.7%.\n\nThe layer-selection gap is also real. The paper never says which layer L produced Table 2. Section 6.1 shows the effect is sharply layer-dependent, and the Limitations concede the fixed linear vector is limited. If L was chosen on the test benchmarks, the margins are upper bounds. The stress-test note holds up.\n\nWhat I like: the layer-sensitivity analysis is informative and mechanistic—Aya23 works best early, Llama2 early/middle for low-resource languages, final layers useless. The cross-lingual transfer results (Spanish vector helps German/French/Russian but not Japanese) are a clean sanity check. The logit-lens comparison between steering and fine-tuning is a useful qualitative observation. The paper is also honest in its limitations section about the fixed-direction constraint.\n\nWhat's missing: no direct baseline against Wang et al. 2024, so the single-layer vs all-layer question is not actually answered; no code/data; no error bars. Those are minor-to-moderate, but they add uncertainty.\n\nOverall, this deserves a serious referee: the per-model data are substantial and the layer analysis is valuable, so it would be a shame to desk-reject. But the authors need to fix the SFT row, disclose how L was chosen, and ideally add error bars and code before the claims can be trusted. If the SFT row is a simple error, the paper lands as a modest improvement over prompting/translation baselines rather than a leap over fine-tuning.","headline":"The SFT comparison is internally inconsistent and the layer-selection protocol is undisclosed, so the headline claim is not reliable; the per-model data and layer analysis are still worth a serious look.","tokens_in":19351,"tokens_out":11516,"would_cite":false,"duration_ms":84029,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single learned vector added at one layer aligns multilingual LLMs","keywords":["multilingual language models","representation steering","activation intervention","residual stream","cross-lingual transfer","direct preference optimization","low-resource languages"],"falsifier":"Re-run the headline comparison selecting the steering layer and vector hyperparameters only on a held-out validation split, never on the test benchmarks; if the reported margins over supervised fine-tuning vanish or reverse under that protocol, the central comparison is an artifact of optimistic layer selection. A second check is to train the steering vector without any English translations and see whether task gains remain, which would reveal how dependent the method is on English supervision.","tokens_in":1417,"feed_emoji":"🌐","tokens_out":1927,"duration_ms":72285,"temperature":0.7,"pith_summary":"The paper sets out to show that multilingual failures in large language models are largely a representational alignment problem, and that correcting it does not require further training. Its method learns one steering vector per language and adds it to the residual stream at a single layer during inference. Across 14 languages and five tasks, this intervention beats supervised fine-tuning, 5-shot in-context learning, cross-lingual prompting, and the NLLB translation pipeline, and comes within about three points of Google Translate in aggregate. If true, the result recasts a major piece of multilingual capability as a single-layer geometric correction rather than a data-hungry training problem.","feed_headline":"One learned vector steers LLMs near Google Translate","feed_subtitle":"A single-layer activation fix outperforms supervised fine-tuning and 5-shot prompting across 14 languages.","key_machinery":"The steering vector $v \\in \\mathbb{R}^d$ is a single learnable direction added to the residual stream at the entry of layer $L$, so that each token's activation becomes $x_{\\text{target}} + v$. It is learned from English-target sentence pairs either by a direct preference optimization objective that raises the probability of English-aligned responses and lowers the probability of target-language responses, or by minimizing mean squared error between the aligned and English activations. Its work is to shift the whole target-language activation distribution into the model's English-representation subspace in one step, which the paper claims mimics the representational alignment that fine-tuning produces over many epochs.","core_discovery":"The paper claims that a fixed additive vector $v$, applied as $x_{\\text{altered}} = x_{\\text{target}} + v$ to every token's residual stream at one selected layer, aligns non-English representations with English closely enough to improve task performance across five open-source LLM families and 14 languages. The vector learned by a directional preference objective (DPO) consistently outperforms MSE-based alignment, transfers between linguistically related languages, and produces internal alignment effects similar to full supervised fine-tuning, as shown by logit-lens inspection. The paper also reports that the best layer is architecture-specific and diagnostic: multilingual models like Aya23 benefit most from early-layer steering, while Llama2 needs early or middle intervention for low-resource languages and is disrupted by late-layer changes.","pith_inferences":["If one vector per language suffices, multilingual models may encode a shared English-like subspace plus language-specific offsets, making cross-lingual alignment a low-rank correction problem rather than a deep architectural one.","The layer-sensitivity result could be turned into a pre-training diagnostic: measuring how far a language's early activations are from the model's dominant-language subspace might predict downstream performance gaps without running full benchmarks.","A natural extension is to replace the fixed vector with a per-token or per-prompt vector; the paper's own limitation section notes that a fixed linear direction misses contextual variation, so context-dependent steering should close more of the gap to translation-based systems.","Since the method works best on languages whose representations already partially overlap English, it is most plausibly paired with data interventions for low-resource languages, not treated as a complete substitute for multilingual pretraining."],"forward_implications":["Multilingual improvement becomes possible without gradient updates: only a small bilingual corpus, a few forward passes, and one stored vector are needed at inference time.","A steering vector learned on one language transfers to related languages within the same family, so a small set of vectors could cover language families rather than individual languages.","The layer at which steering helps is diagnostic of how a model organizes languages: early-layer effectiveness indicates a shared language-neutral space, while late-layer ineffectiveness indicates the model has already committed to an interpretation.","Because steering and fine-tuning both act by aligning non-English representations with English, steering can be composed with fine-tuning, or used in settings where fine-tuning data are scarce.","The method offers a cheaper, model-native alternative to external translation pipelines for low- and mid-resource languages."],"supporting_citations":[{"why":"Supplies the prior inference-time cross-lingual intervention that learns an alignment vector over all layers; this paper narrows that intervention to a single layer.","marker":"Wang et al., 2024"},{"why":"Provides the bi-directional preference optimization (DPO/BiPO) procedure used to learn the steering vector.","marker":"Cao et al., 2024"},{"why":"Supplies the linear representation hypothesis that justifies learning an additive mapping between language representations.","marker":"Park et al., 2023"},{"why":"Establishes that LLMs internally translate non-English tokens toward English, defining the representational target the steering vector aims for.","marker":"Wendler et al., 2024"},{"why":"Analyzes how LLMs handle multilingual inputs and connects representation alignment to performance gaps.","marker":"Zhao et al., 2024"},{"why":"Defines the NLLB open-source translation baseline that the paper reports outperforming with steering.","marker":"Costa-jussà et al., 2022"}],"fun_headline_variants":["Single steering vector boosts multilingual LLMs effectively","LLMs get multilingual boost from one learned vector","Representation steering: one vector improves 14 languages","A single vector aligns multilingual LLM representations","Steering aligns LLM representations, rivals translation"],"cache_read_input_tokens":21376,"weakest_assumption_plain":"A single additive vector, identical for every token position and every prompt in a language, can capture the mapping from that language's internal representations to English well enough to improve task behavior; the authors concede that this fixed linear direction limits the mapping's capacity.","fun_headline_variants_meta":{"raw":{"variants":["Single steering vector boosts multilingual LLMs effectively","LLMs get multilingual boost from one learned vector","Representation steering: one vector improves 14 languages","A single vector aligns multilingual LLM representations","Steering aligns LLM representations, rivals translation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000369,"raw_usage":{"total_tokens":1925,"prompt_tokens":841,"completion_tokens":1084,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":1014}},"tokens_in":457,"tokens_out":1084,"duration_ms":8675,"temperature":1.0,"reasoning_tokens":1014,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:33:56.647275+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the headline comparison selecting the steering layer and vector hyperparameters only on a held-out validation split, never on the test benchmarks; if the reported margins over supervised fine-tuning vanish or reverse under that protocol, the central comparison is an artifact of optimistic layer selection. A second check is to train the steering vector without any English translations and see whether task gains remain, which would reveal how dependent the method is on English supervision.","supporting_citations":[],"review_version":1}