Pith. sign in

REVIEW 4 major objections 5 minor 51 references

Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that strengthening a small set of sparse, target-language-specific features, identified by contrasting SAE activations on parallel sentences and decoded into a steering vector, improves multilingual reasoning accuracy at…

desk verdict A useful training-free multilingual steering idea with held-out gains, but the abstract/table mismatch and a missing random-feature control need attention before the core claim is fully convincing. read the letter →

arxiv 2608.04904 v1 pith:AJK55OOA submitted 2026-08-05 cs.CL

Hongsheng Wang , Phlipp Koehn This is my paper
classification cs.CL
keywords multilingualLLMssparseautoencodersactivationsteeringlanguage-specificfeaturesinference-timeinterventioncross-lingualtransferreasoningGemma-3-12B-it
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a model's language identity is carried by a small set of sparse, language-specific features, and that deliberately strengthening those features at inference time improves multilingual task performance without any training or parameter updates. The method runs parallel sentences through a sparse autoencoder (SAE), averages the feature activations per language, subtracts the average of the non-target languages as a language-neutral semantic centroid, and uses the largest target-versus-centroid differences to build a steering vector. The vector is decoded from SAE features and added to the hidden state at the final prompt token, with a strength $\alpha$ chosen on a validation split. The paper reports average gains of 10.9, 5.3, and 1.9 percentage points on XCOPA, XNLI, and MGSM (the detailed table gives 11.3, 4.1, and 0.6 for the same setting), with all nine XCOPA languages improving.

What carries the argument

The central object is the per-layer target–centroid activation difference $\Delta^l_{\tau,\mathrm{multi}} = \mu^l_\tau - \mu^l_{\mathrm{multi}}$, where $\mu^l_\tau$ is the average SAE activation of the target language and $\mu^l_{\mathrm{multi}}$ is the average over all non-target languages on parallel sentences. A sparse autoencoder is a network that represents a hidden state as a small set of interpretable features, and the multilingual centroid is treated as the shared, language-neutral semantic content of the aligned sentences. The method ranks SAE features by $|\Delta^l_{\tau,\mathrm{multi}}|$, keeps the top $k$ with their signed differences, sets all other dimensions to zero, and decodes this sparse code with the SAE decoder to produce the steering vector added to the hidden state with strength $\alpha$. This pipeline is what converts the hypothesis 'language is a sparse, steerable feature set' into a concrete intervention.

What would settle it

Run feature selection on FLORES-200 sentence groups after permuting the translations so the sentences are no longer semantically aligned across languages, then apply the same steering to the same test sets; if the accuracy gains persist, the selected features are not actually isolating language while controlling for content, whereas if they vanish, semantic alignment is the load-bearing ingredient.

Watch

Extended reading notes

Core claim

The central claim is that strengthening target-language features, rather than aligning representations or retraining, is enough to narrow the multilingual performance gap. Using model-matched pretrained SAEs on Gemma-3-12B-it, the paper identifies, for each layer, the features whose activation differs most between the target language and the average of the other languages on 500 parallel sentence groups. Those top-$k$ features are decoded into a steering vector and added to the residual stream at the final prompt position before the model generates its answer. The paper argues the gains are causal: the intervention improves accuracy on commonsense reasoning, natural language inference, and math word problems, and ablations show that scaling hidden states or steering raw hidden dimensions does not reproduce the effect.

Load-bearing premise

The method assumes that averaging the SAE activations of the non-target languages yields a language-neutral semantic centroid, so that subtracting it from the target-language average leaves purely language-specific features rather than accidental content or style differences.

Editorial extensions

If this is right

  • Multilingual reasoning can be improved on any model with compatible SAEs using only a few hundred parallel sentences, no gradient updates, and no language-specific training data.
  • Across the nine XCOPA languages the gain is uniformly positive (Table 1, average +11.3 points), showing the effect is not confined to high-resource languages.
  • Since XCOPA and XNLI answers are label tokens A/B/C, the improvement cannot be explained by steering the model toward English surface output; it reflects changed processing of the non-English input.
  • A fixed steering strength of $\alpha = 0.6$ still improves average accuracy on all three datasets, so per-language hyperparameter search is not essential to the benefit.
  • The ablation result that hidden-state scaling and raw hidden-dimension steering barely move scores implies the sparse SAE code, not activation magnitude, is what carries the language signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If this holds, language identity in multilingual models behaves like a small, sparse, causally effective feature set, which suggests the same contrastive SAE recipe could steer other categorical properties such as dialect, register, or domain.
  • A direct test of the centroid assumption: use a parallel corpus that is topic-matched but not sentence-aligned; if the gains persist, content alignment is not the mechanism and the centroid may be capturing a broader language prior.
  • Both an early layer (6) and a late layer (47) support effective steering, so target-language features are distributed; combining steering across multiple layers is a natural extension that could exceed single-layer gains.
  • The English-relative variant performs nearly as well as the multilingual variant, which is consistent with English-centered internal representations in this model family, but evaluating on a model trained on more balanced multilingual data would separate reference choice from the model's own inductive bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an inference-time, training-free method for improving multilingual LLM performance. Using parallel sentences from FLORES-200, the method compares SAE activations of target and reference languages, selects the top-k features with the largest activation differences, decodes those differences into a steering vector via the SAE decoder, and adds the vector to the hidden state at the final prompt position of a chosen transformer layer. Experiments with Gemma-3-12B-it and Gemma Scope 2 SAEs report average accuracy gains on XCOPA, XNLI, and MGSM, alongside ablations comparing reference representations (multilingual centroid vs. English), signal content (language-only vs. full representation), and non-SAE controls (hidden-state scaling, direct hidden-dimension steering, least-squares English projection). The paper claims that strengthening target-language-related SAE features improves held-out multilingual task performance without parameter updates.

Significance. If the central causal claim holds, this is a useful contribution: a lightweight, training-free method for multilingual inference that leverages independently pretrained SAEs, with evaluation on held-out test splits and hyperparameters selected on validation splits. The use of a parallel corpus separate from the downstream evaluation set, the validation-based tuning of the steering coefficient, and the public release of code are strengths. However, the missing control with random or least-discriminative SAE features leaves the mechanistic interpretation underdetermined, and several reporting inconsistencies in the headline numbers need correction. The method may still be practically useful even if the language-specificity explanation is not fully established, but the current evidence does not support all the paper's interpretive claims.

major comments (4)
  1. [§6, Figure 4, Table 3] The central causal claim that language-discriminative feature selection drives the gains is not established, because no control uses the same SAE-based steering pipeline with random or least-discriminative features. All four SAE-based strategies improve over baseline, but a non-sparse, non-SAE baseline (least-squares English projection) reaches 68.4 on XCOPA, above the main method's 65.8. This leaves open the possibility that the improvement comes from adding any sparse SAE-decoded vector at layer 47 rather than from the language-specificity of the selected dimensions. I recommend adding a random-feature control (e.g., top-k features with the smallest absolute activation difference, or randomly sampled SAE features) and reporting whether it also improves performance; this directly tests the paper's mechanistic interpretation.
  2. [§5, Table 1 vs. Abstract and Appendix G] The reported gains are internally inconsistent. The abstract states average gains of 10.9, 5.3, and 1.9 percentage points on XCOPA, XNLI, and MGSM, while Table 1 reports 11.3, 4.1, and 0.6. Additionally, the Appendix G text says the target-language signal on MGSM achieves 83.6 against an 81.7 baseline, whereas Table 1 and Figure 10 report 81.1 against 80.5; Figure 10's caption also says the average is over German, Spanish, and French, but Table 1 includes Japanese. These discrepancies must be resolved before the results can be assessed, and the abstract should match the table.
  3. [§5, Table 1] The claim of 'consistent improvements' is too strong given the per-language results and the absence of error bars or significance tests. On XNLI, German decreases by 2.5 points; on MGSM, Japanese decreases by 0.5 points and the average gain is 0.6 points. With no repeated runs, confidence intervals, or significance tests, the reader cannot distinguish a real effect from tuning noise, especially for MGSM. I recommend reporting bootstrap or standard-error estimates, or at minimum clearly flagging the language-level variability and softening the consistency claim.
  4. [§3.2, Eqs. (5)–(6)] The method assumes that the average SAE activation of non-target languages is a language-neutral semantic centroid, so that subtracting it isolates language-specific features. This assumption is load-bearing: if the centroid retains language-specific information or is affected by semantic misalignment in the parallel sentences, the selected features will not be purely target-language features, and the steering signal will have an unintended component. The paper does not validate this assumption. A concrete test would be to check the stability of the selected features under different subsets of non-target languages, or to compare the centroid with a leave-one-out estimate; the current English-reference variant only partially addresses this concern.
minor comments (5)
  1. [§4.3, Figure 2] The 'Random feature' control is said to be close to zero and not visible at the figure's scale; showing it in an inset or on a log scale would make the contrast informative rather than invisible.
  2. [§4.4] Layer 47 is selected based on XCOPA validation performance and then used for all datasets. The paper should state whether this introduces selection bias and, ideally, show layer-wise results for XNLI and MGSM in addition to XCOPA.
  3. [§4.3] The number of selected features is fixed at k=3 without a sensitivity analysis. Since k is a free parameter, reporting results for k=1, 5, 10 would help establish robustness.
  4. [Appendix G, Figure 10] The caption and the text disagree about which languages are averaged in the MGSM ablation; please correct the caption to match the data reported in Table 1.
  5. [§2, Related Work] The paper cites two preprints (Kirtane and Huang, 2026; Sterz et al., 2025) to justify the claim that averaging across languages attenuates language-specific variation. If these are not yet publicly available or peer-reviewed, please provide additional detail or a more standard citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: features come from the independent FLORES-200 corpus, tuning is confined to validation splits, and the reported held-out gains reduce to no fitted quantity.

full rationale

Walking the paper's derivation chain: the target-language features are selected exclusively from SAE activations of FLORES-200 parallel sentences (Eqs. 3-8); the steering vector is the deterministic SAE decode of the resulting sparse code of activation differences (Eqs. 10-11); and the only tuned quantities are the per-language steering coefficient alpha (validation split, Section 4.5) and the intervention layer (XCOPA validation set, Section 4.4). All reported gains (Table 1; Figures 4, 9, 10) are measured on held-out test splits that are never touched by feature selection or tuning, so no equation in the paper reduces a claimed improvement to a fitted parameter or to the definition of a feature. The load-bearing assumption that the multilingual centroid (Eq. 5) approximately captures language-neutral semantics is an empirical hypothesis, explicitly probed by the English-reference, full-representation, hidden top-k, scaling, and least-squares controls (Section 6); it is not an algebraic identity. No load-bearing citation involves the present authors (Deng et al. 2025, Chou et al. 2025, Kirtane and Huang 2026, Sterz et al. 2025, and Wang et al. 2025 are all external), so there is no self-citation chain. The manuscript's Limitations paragraph honestly notes that results may not generalize to other models without matched SAEs, which supports a low circularity reading. Reporting inconsistencies exist -- the abstract's 10.9/5.3/1.9 percentage points do not match Table 1's 11.3/4.1/0.6, and Appendix G's text (83.6/81.7) disagrees with Figure 10 (81.1/80.5) -- but these are correctness defects, not circular reductions. The missing random-feature steering control and the validity of the language-neutral centroid assumption are underdetermination concerns, not cases where the derivation reduces to its inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method introduces no new entities. It relies on several domain assumptions about the language-neutrality of the multilingual centroid, the semantic alignment of parallel sentences, the faithfulness of the pretrained SAEs, and the validity of activation steering. These assumptions are stated in the paper but not independently verified.

free parameters (3)
  • Steering coefficient alpha (per language, per dataset) = See Table 1 (e.g., XCOPA: 0.8-2.5; XNLI: 0.0-1.0; MGSM: 0.0-0.9)
    Selected by grid search (0.0-2.5 for XCOPA, 0.0-1.0 for XNLI/MGSM, step 0.1) on the validation split. This is a hyperparameter tuned to data.
  • Number of top features k = 3
    The authors state 'we select the three features with the largest absolute differences at each layer' (Section 4.3). No ablation over k is shown.
  • Intervention layer l = 47
    Chosen because layer 6 and 47 had the highest validation gains on XCOPA (13.9 and 13.4 points); layer 47 was selected as it 'directly precedes the model output' (Section 4.4). This is a design choice informed by validation data.
assumptions (5)
  • domain assumption The multilingual centroid of non-target languages approximates a language-neutral semantic representation.
    Section 3.2, Eq. 5-6. The definition of target-language features relies on this centroid being language-neutral so that the difference isolates language identity.
  • domain assumption Parallel sentences in the FLORES-200 corpus are semantically aligned across languages, so that differences in SAE activations reflect language rather than content.
    Section 3.2: 'Because the sentences are semantically aligned across languages, we treat this multilingual centroid as an approximation of their shared, language-neutral semantic representation.' This is assumed and not validated.
  • domain assumption The pretrained Gemma Scope 2 SAEs provide a sufficiently faithful sparse decomposition of the residual stream for the intervention to be meaningful.
    Section 4.1: SAEs are used as fixed encoders/decoders. The paper does not report reconstruction metrics or intervention reliability checks.
  • domain assumption Adding a decoded sparse code to the hidden state at the final prompt position steers the model's next-token distribution toward the target language.
    Section 3.3: The intervention is applied at the final prompt position 'directly determines the model's first response token'. This is a standard activation-steering assumption.
  • domain assumption The English-centered representation hypothesis motivates the English-reference condition but is not required for the main method.
    Section 3.2 cites prior work (Wendler et al., 2024; Schut et al., 2025). The main method uses the multilingual centroid, so this is only relevant to the English-reference ablation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference." pith.science (2026). https://pith.science/paper/AJK55OOA

@misc{pith2026260804904,
  author       = {Pith},
  title        = {Pith review of: Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AJK55OOA}},
  note         = {Machine review of arXiv:2608.04904}
}
read the original abstract

Multilingual large language models exhibit substantial performance differences across languages, while existing adaptation methods often require parameter updates and considerable multilingual training data. We propose an inference-time multilingual steering method that uses pretrained sparse autoencoders to identify and strengthen target-language-related features. Using multilingual parallel sentences, we compare SAE activations across languages and select a small number of layer-specific features associated with each target language. These features are decoded into steering signals and injected into the model's hidden states without additional training. Experiments with Gemma-3-12B-it show average accuracy improvements of 10.9 percentage points on XCOPA, 5.3 points on XNLI, and 1.9 points on MGSM.

Figures

Figures reproduced from arXiv: 2608.04904 by the authors.

Figure 1
Figure 1. Overview of target-language feature identification. At each transformer layer, hidden states from aligned [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Target-language SAE activation differences relative to the multilingual centroid at layer 47. For each [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Accuracy gain over the non-steered baseline [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Average test accuracy of the intervention [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Target-language SAE activation differences [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 7
Figure 7. Figure 7: Accuracy gain over the non-steered base [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Validation accuracy gain under intervention [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 10
Figure 10. Figure 10: Average accuracy of the intervention strate [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 28 canonical work pages

  1. [1]

    Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, et al. 2023. Mega: Multilingual evaluation of generative ai. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4232--4267

    work page 2023

  2. [2]

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2(5):6

    work page 2023

  3. [3]

    Tyler A Chang, Zhuowen Tu, and Benjamin K Bergen. 2022. The geometry of multilingual language model representations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 119--136

    work page 2022

  4. [5]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 8440--8451

    2020

  5. [6]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2475--2485

    work page 2018

  6. [9]

    John Dang, Arash Ahmadian, Kelly Marchisio, Julia Kreutzer, Ahmet \"U st \"u n, and Sara Hooker. 2024. Rlhf can speak many languages: Unlocking multilingual preference optimization for llms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13134--13156

    work page 2024

  7. [10]

    Boyi Deng, Yu Wan, Baosong Yang, Yidan Zhang, and Fuli Feng. 2025. Unveiling language-specific features in large language models via sparse autoencoders. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4563--4608

    work page 2025

  8. [11]

    Zihao Feng, Hailong Cao, Wang Xu, and Tiejun Zhao. 2025. https://aclanthology.org/2025.coling-main.138/ Word-level cross-lingual structure in large language models . In Proceedings of the 31st International Conference on Computational Linguistics, pages 2026--2037, Abu Dhabi, UAE. Association for Computational Linguistics

    work page 2025

Show all 51 references
  1. [14]

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, J \'a nos Kram \'a r, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. In Proceedings of the 7th BlackboxN...

    2024

  2. [15]

    Danni Liu and Jan Niehues. 2025. Middle-layer representation alignment for cross-lingual transfer in fine-tuned llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15979--15996

    2025

  3. [16]

    Callum McDougall, Arthur Conmy, J \'a nos Kram \'a r, Tom Lieberum, Senthooran Rajamanoharan, and Neel Nanda. 2025. https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-beha...

    2025

  4. [17]

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. 2023. Crosslingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of th...

    2023

  5. [18]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  6. [19]

    Edoardo Maria Ponti, Goran Glava s , Olga Majewska, Qianchu Liu, Ivan Vuli \'c , and Anna Korhonen. 2020. Xcopa: A multilingual dataset for causal commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2362--2376

    2020

  7. [20]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728--53741

    2023

  8. [23]

    Hannah Sterz, Fabian David Schmidt, Goran Glava s , and Ivan Vulic. 2025. Recover the target language: Language steering without sacrificing task performance. Preprint

    2025

  9. [24]

    Nishant Subramani, Nivedita Suresh, and Matthew E Peters. 2022. Extracting latent steering vectors from pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 566--581

    2022

  10. [25]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, ...

    2025 arXiv

  11. [27]

    Weixuan Wang, Minghao Wu, Barry Haddow, and Alexandra Birch. 2025. Bridging the language gaps in large language models with inference-time cross-lingual intervention. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2025

  12. [28]

    Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do llamas work in english? on the latent language of multilingual transformers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

    2024

  13. [31]

    arXiv preprint arXiv:2309.08600 , year=

    Sparse autoencoders find highly interpretable features in language models , author=. arXiv preprint arXiv:2309.08600 , year=

    arXiv

  14. [32]

    Transformer Circuits Thread , volume=

    Towards monosemanticity: Decomposing language models with dictionary learning , author=. Transformer Circuits Thread , volume=

  15. [33]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Unveiling language-specific features in large language models via sparse autoencoders , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  16. [34]

    arXiv preprint arXiv:2507.13410 , year=

    Causal language control in multilingual transformers via sparse feature steering , author=. arXiv preprint arXiv:2507.13410 , year=

  17. [35]

    arXiv preprint arXiv:2411.02193 , year=

    Improving steering vectors by targeting sparse autoencoder features , author=. arXiv preprint arXiv:2411.02193 , year=

    arXiv

  18. [36]

    arXiv preprint arXiv:2602.02326 , year=

    Language Steering for Multilingual In-Context Learning , author=. arXiv preprint arXiv:2602.02326 , year=

  19. [37]

    Preprint , year=

    Recover the target language: Language steering without sacrificing task performance , author=. Preprint , year=

  20. [38]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Do llamas work in english? on the latent language of multilingual transformers , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  21. [39]

    arXiv preprint arXiv:2502.15603 , year=

    Do multilingual llms think in english? , author=. arXiv preprint arXiv:2502.15603 , year=

    arXiv

  22. [40]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Middle-layer representation alignment for cross-lingual transfer in fine-tuned llms , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  23. [41]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Bridging the language gaps in large language models with inference-time cross-lingual intervention , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  24. [42]

    arXiv preprint arXiv:2308.10248 , year=

    Steering language models with activation engineering , author=. arXiv preprint arXiv:2308.10248 , year=

    arXiv

  25. [43]

    arXiv preprint arXiv:2310.01405 , year=

    Representation engineering: A top-down approach to ai transparency , author=. arXiv preprint arXiv:2310.01405 , year=

    arXiv

  26. [44]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

  27. [45]

    Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages=

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2 , author=. Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages=

  28. [46]

    2025 , month = sep, type =

    Gemma Scope 2: Technical Paper , author =. 2025 , month = sep, type =

    2025

  29. [47]

    arXiv preprint arXiv:2207.04672 , year=

    No language left behind: Scaling human-centered machine translation , author=. arXiv preprint arXiv:2207.04672 , year=

    arXiv

  30. [48]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    XCOPA: A multilingual dataset for causal commonsense reasoning , author=. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    2020

  31. [49]

    Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    XNLI: Evaluating cross-lingual sentence representations , author=. Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    2018

  32. [50]

    arXiv preprint arXiv:2210.03057 , year=

    Language models are multilingual chain-of-thought reasoners , author=. arXiv preprint arXiv:2210.03057 , year=

    arXiv

  33. [51]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Mega: Multilingual evaluation of generative ai , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  34. [52]

    Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

    Unsupervised cross-lingual representation learning at scale , author=. Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

  35. [53]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Crosslingual generalization through multitask finetuning , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  36. [54]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Rlhf can speak many languages: Unlocking multilingual preference optimization for llms , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  37. [55]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  38. [56]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  39. [57]

    Findings of the Association for Computational Linguistics: ACL 2022 , pages=

    Extracting latent steering vectors from pretrained language models , author=. Findings of the Association for Computational Linguistics: ACL 2022 , pages=

    2022

  40. [58]

    On the Language Neutrality of Pre-trained Multilingual Representations

    Libovick \'y , Jind r ich and Rosa, Rudolf and Fraser, Alexander. On the Language Neutrality of Pre-trained Multilingual Representations. Findings of the Association for Computational Linguistics: EMNLP 2020. 2020. doi:10.18653/v1/2020.findings-emnlp.150

    2020 doi

  41. [59]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    The geometry of multilingual language model representations , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  42. [60]

    Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations

    Xie, Zhihui and Zhao, Handong and Yu, Tong and Li, Shuai. Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.379

    2022 doi

  43. [61]

    Word-level Cross-lingual Structure in Large Language Models

    Feng, Zihao and Cao, Hailong and Xu, Wang and Zhao, Tiejun. Word-level Cross-lingual Structure in Large Language Models. Proceedings of the 31st International Conference on Computational Linguistics. 2025

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.