Pith. sign in

REVIEW 3 major objections 4 minor 51 references

Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that boosting a small set of target-language features, identified by comparing sparse autoencoder activations across parallel sentences, and adding them to the final prompt hidden state improves multilingual task accuracy…

desk verdict Clean idea with decent controls, but the headline numbers don't match across abstract, Table 1, and Appendix G – needs revision before the claim is evaluable. read the letter →

arxiv 2608.04904 v2 pith:AJK55OOA submitted 2026-08-05 cs.CL

Hongsheng Wang , Philipp Koehn This is my paper · ORCID
classification cs.CL
keywords sparseautoencodersmultilingualinferenceactivationsteeringtarget-languagefeaturesinference-timeadaptationparallelsentencesGemma-3languagerepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a language's identity inside a multilingual model is concentrated in a few sparse-autoencoder features, and that amplifying them is enough to improve how the model answers in that language. The proposed method computes, for each transformer layer, the average SAE activation of the target language on parallel sentences and subtracts the average over the other languages, then keeps the three features with the largest difference, decodes them back into hidden space, and adds the result to the final prompt token's hidden state. No parameters are updated. On Gemma-3-12B-it the authors report average held-out accuracy improvements on XCOPA, XNLI, and MGSM, with the largest stated gain around 10.9 points, and ablations indicating that the sparse SAE representation, rather than activation scaling or raw hidden dimensions, is what carries the effect.

What carries the argument

The load-bearing object is the target--multilingual-centroid activation difference, $\Delta_{\tau,\mathrm{multi}}^l=\mu_\tau^l-\mu_{\mathrm{multi}}^l$, where $\mu_\lambda^l$ is the sentence- and token-average SAE activation for language $\lambda$ at layer $l$ and $\mu_{\mathrm{multi}}^l$ averages over all non-target languages. The method keeps the three entries of $\Delta$ with the largest absolute value, sets everything else to zero, and maps the sparse code back with the SAE decoder $g^l$ to form the steering vector $v^l$. The intervention is $\tilde h_q^l = h_q^l + \alpha v^l$ applied at the final prompt position of layer 47, after which the forward pass continues normally.

What would settle it

Shuffle the alignments of the parallel sentences so that supposedly parallel sentences do not match semantically, then repeat feature selection and steering; if held-out gains largely persist, the difference vector is not cleanly 'target language minus shared meaning'. Independently, run the same pipeline on a second model family with its own matched pretrained SAEs; if the average gains disappear, the effect is tied to this model's SAE decomposition rather than a general property of multilingual representations.

Watch

Extended reading notes

Core claim

The central claim is that steering by target-language SAE features is a causal intervention: the top-three features with the largest target-minus-centroid activation difference at layer 47, when decoded and added with strength $\alpha$ to the final prompt hidden state, shift the model's next-token prediction toward the correct answer in that language. The authors define the multilingual centroid as the average SAE activation over all non-target languages and treat it as the language-neutral semantic content of the parallel sentences. They report that this intervention improves average accuracy on all three datasets, that a single shared coefficient $\alpha=0.6$ still gives positive average gains, and that the language-only sparse signal is a competitive or better steering vector compared with full-representation and non-SAE alternatives.

Load-bearing premise

The load-bearing premise is that the average of non-target-language SAE activations is a faithful language-neutral semantic centroid, so the target-minus-centroid difference isolates language-specific content; if the centroid leaks semantic or third-language signal, the steering vector is not doing what the paper claims.

Editorial extensions

If this is right

  • Held-out accuracy rises on average on XCOPA, XNLI, and MGSM, with the largest gains on XCOPA and on specific languages such as Italian, Indonesian, and Turkish.
  • Using the same steering coefficient for every language and dataset still yields positive average gains, so the effect does not depend on per-language tuning.
  • The SAE-based sparse signal outperforms steering in raw hidden dimensions and hidden-state scaling, indicating that the sparse decomposition is what isolates a useful language signal.
  • Because the intervention changes no parameters and needs only a few hundred parallel sentences, it can be applied to any model that has compatible pretrained SAEs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to compare the selected features with those from contrasting two non-target languages; if the same features appear, the centroid difference may capture general multilingual variation rather than target-language identity alone.
  • The similar performance of the multilingual and English reference variants suggests the gain does not depend on an English-centered internal space; using a non-English high-resource language as reference would test this directly.
  • If the effect transfers across model families, it would make gradient-free language adaptation feasible for low-resource languages using only a few hundred translated sentences, a testable claim for future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes an inference-time method for improving multilingual task performance in LLMs. Using frozen layer-specific sparse autoencoders (SAEs) from Gemma Scope 2, the authors identify the top-k SAE features whose activations differ most between a target language and a multilingual centroid (or an English reference) computed from parallel FLORES-200 sentences. These features are decoded into a sparse steering vector and added to the hidden state at the final prompt position of a chosen layer. Experiments on Gemma-3-12B-it report average held-out accuracy gains on XCOPA, XNLI, and MGSM, with ablations comparing multilingual versus English references, language-only versus full-representation signals, and non-SAE controls. The paper claims the method requires no parameter updates and improves multilingual performance across all three benchmarks.

Significance. If the reported results are accurate, the method is a lightweight, no-training alternative to multilingual adaptation that leverages SAE feature interpretability for downstream task performance rather than only language generation. The paper has concrete strengths: steering coefficients are selected on validation splits before held-out evaluation, a shared-coefficient robustness check is included, several control interventions (hidden-state scaling, direct hidden-dimension steering, least-squares English projection) are compared, and the code is publicly released. The central empirical claim is not obviously circular, since the steering vectors are computed from FLORES-200 parallel sentences and the test sets are held out. However, the manuscript currently contains mutually inconsistent headline numbers across the abstract, Section 5, Table 1, and Appendix G, so the exact effect sizes claimed by the paper cannot be verified as written.

major comments (3)
  1. [Abstract and Section 5 versus Table 1 and Appendix G] The paper's central quantitative claims are internally inconsistent. The abstract and the introduction report average gains of +10.9 on XCOPA, +5.3 on XNLI, and +1.9 on MGSM; Section 5 and Table 1 report +11.3, +4.1, and +0.6 for the same settings. Appendix G, in turn, states for MGSM that the target-language signal achieves 83.6 against an 81.7 baseline, a gain of 1.9 that matches neither the Table 1 values (80.5 to 81.1) nor the three-language averages computed from Table 1 (80.3 to 81.3). Because the contribution is precisely a quantitative improvement claim, every reported number must be traceable to one experimental configuration. The authors must reconcile these discrepancies or clearly identify which configuration underlies each table and figure; as written, the reader cannot determine which results are being claimed.
  2. [Appendix G, MGSM paragraph and Figure 10] The Appendix G MGSM text says that 'all intervention settings improve over the baseline,' but Figure 10 shows the English-relative language-only strategy at 78.2 and the English full-representation strategy at 79.1, both below the printed baseline of 80.5 (and also below the three-language average of 80.3). The caption states that the averages are over German, Spanish, and French, yet the displayed values match a four-language average including Japanese. This is not a typo in one digit: the text, table, and figure each tell a different story about the MGSM ablation. These numbers must be aligned before the paper's empirical claims can be assessed.
  3. [Section 3.2, Equations (5)-(6)] The mechanistic interpretation of the method rests on the assumption that the multilingual centroid approximates language-neutral semantic content, so that the target-minus-centroid difference isolates target-language features. This assumption is not directly validated. The parallel-sentence control reduces but does not eliminate the possibility that the top features capture translation residuals or third-language signal, especially because the centroid averages over only the eight or three non-target languages used in each benchmark and because the selected features are identified by activation contrast alone. I recommend adding a control that disrupts the language pairing (for example, computing the centroid from randomly mismatched parallel sentences and comparing the selected features and downstream gains), or otherwise providing direct evidence that the selected features are language-discriminative rather than semantic. This would materially strengthen the paper's central interpretation that the steering signal is a target-language feature, rather than merely a beneficial hidden-state perturbation.
minor comments (4)
  1. [Section 4.3 and Section 3.2] The text in Section 4.3 says the feature-identification procedure is described in Section 3.3, but the procedure is in Section 3.2; the cross-reference should be corrected.
  2. [Appendix B] Appendix B presents a layer-20 analysis in Figure 5 but concludes that the effect is 'not limited to layer 30'; the layer number cited in the sentence should match the figure (either layer 20 or an additional layer-30 figure should be provided).
  3. [Appendix G, MGSM caption] The Figure 10 caption says the MGSM averages are over German, Spanish, and French, but the numbers shown match the four-language averages in Table 1, which include Japanese; the caption and the underlying data should be made consistent.
  4. [References] Several references contain spacing artifacts (e.g., 'V os' in the Shi et al. entry and 'Cideron, Jean bastien' in the Gemma Team entry); these should be cleaned up before final submission.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the steering vectors are computed from FLORES-200 activation statistics and evaluated on held-out downstream tasks, with no fitted parameter renamed as a prediction.

full rationale

The paper's derivation chain is self-contained and externally benchmarked. Language-discriminative SAE features are selected by comparing SAE activations of FLORES-200 parallel sentences against a multilingual centroid (Eqs. 4-8), and the steering vector is the SAE-decoded sparse difference (Eqs. 10-11). This vector is then added to the hidden state at the final prompt position of layer 47. The downstream evaluations on XCOPA, XNLI, and MGSM use held-out test examples, and the steering coefficient alpha is selected on validation splits only. No test labels are used to fit the reported gains, and no parameter in the method is defined in terms of the evaluation outcome. The multilingual-centroid assumption (Eq. 5) is an interpretational premise about what the averaged reference represents, not a circular reduction: even if the centroid retained language-specific information, the method's empirical claim about accuracy would remain independently testable. The paper contains no load-bearing self-citations; the cited prior work on SAE language features and activation steering supports the setup but is not required to make the evaluation results follow. The observed internal inconsistencies among the abstract, Table 1, and Appendix G are a reporting and verifiability concern, not evidence of circularity, because they do not show that any claimed result is equivalent to its inputs by construction. Therefore the correct circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method rests on three numerically tuned or hand-chosen settings (alpha, k, intervention layer) and four domain assumptions about SAE feature semantics and the multilingual centroid. It introduces no invented entities.

free parameters (3)
  • steering coefficient alpha = per-language values; e.g., XCOPA alpha* 0.8-2.5, XNLI alpha* 0.0-1.0, MGSM alpha* 0.0-0.9
    Tuned on the validation split for each language and dataset; controls intervention strength (Eq. 9).
  • number of features k = 3
    Chosen by hand, no tuning or sensitivity analysis reported (Section 4.3).
  • intervention layer = 47
    Selected from a validation sweep; layer 6 actually scored higher (13.9 vs 13.4) but layer 47 was used in main experiments (Section 4.4).
assumptions (4)
  • domain assumption The multilingual centroid of non-target languages approximates the shared language-neutral semantic representation of parallel sentences.
    Eqs. (5)-(6) and the surrounding text in Section 3.2; if false, the difference vector does not isolate target-language features.
  • domain assumption SAE features are sparse, interpretable, and language-specific, and intervening on them can causally affect model behavior.
    Invoked in Sections 2 and 3.1, relying on Deng et al. (2025) and Chou et al. (2025); the method's validity depends on this.
  • domain assumption Averaging SAE activations over tokens and over 500 parallel FLORES-200 sentences yields stable language-level statistics.
    Section 4.2 uses only 500 aligned sentence groups; low-resource languages may have small effective token counts.
  • domain assumption The final prompt position is the right place to intervene, and layer-wise SAEs are compatible with this intervention point.
    Section 3.3 applies the steering vector at the final prompt position; no justification for why this position or layer is best beyond validation results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference." pith.science (2026). https://pith.science/paper/AJK55OOA

@misc{pith2026260804904,
  author       = {Pith},
  title        = {Pith review of: Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AJK55OOA}},
  note         = {Machine review of arXiv:2608.04904}
}
read the original abstract

Multilingual large language models exhibit substantial performance differences across languages, while existing adaptation methods often require parameter updates and considerable multilingual training data. We propose an inference-time multilingual steering method that uses pretrained sparse autoencoders to identify and strengthen target-language-related features. Using multilingual parallel sentences, we compare SAE activations across languages and select a small number of layer-specific features associated with each target language. These features are decoded into steering signals and injected into the model's hidden states without additional training. Experiments with Gemma-3-12B-it show average accuracy improvements of 10.9 percentage points on XCOPA, 5.3 points on XNLI, and 1.9 points on MGSM.

Figures

Figures reproduced from arXiv: 2608.04904 by the authors.

Figure 1
Figure 1. Overview of target-language feature identification. At each transformer layer, hidden states from aligned [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Target-language SAE activation differences relative to the multilingual centroid at layer 47. For each [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Accuracy gain over the non-steered baseline [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Average test accuracy of the intervention [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Target-language SAE activation differences [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 7
Figure 7. Figure 7: Accuracy gain over the non-steered base [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Validation accuracy gain under intervention [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 10
Figure 10. Figure 10: Average accuracy of the intervention strate [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 8 linked inside Pith

  1. [2]

    Transformer Circuits Thread , volume=

    Towards monosemanticity: Decomposing language models with dictionary learning , author=. Transformer Circuits Thread , volume=

  2. [3]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Unveiling language-specific features in large language models via sparse autoencoders , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  3. [5]

    arXiv preprint arXiv:2411.02193 , year=

    Improving steering vectors by targeting sparse autoencoder features , author=. arXiv preprint arXiv:2411.02193 , year=

    arXiv

  4. [7]

    Preprint , year=

    Recover the target language: Language steering without sacrificing task performance , author=. Preprint , year=

  5. [8]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Do llamas work in english? on the latent language of multilingual transformers , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  6. [10]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Middle-layer representation alignment for cross-lingual transfer in fine-tuned llms , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  7. [11]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Bridging the language gaps in large language models with inference-time cross-lingual intervention , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  8. [14]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

Show all 51 references
  1. [15]

    Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages=

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2 , author=. Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pages=

  2. [16]

    2025 , month = sep, type =

    Gemma Scope 2: Technical Paper , author =. 2025 , month = sep, type =

    2025

  3. [18]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    XCOPA: A multilingual dataset for causal commonsense reasoning , author=. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    2020

  4. [19]

    Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    XNLI: Evaluating cross-lingual sentence representations , author=. Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    2018

  5. [21]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Mega: Multilingual evaluation of generative ai , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  6. [22]

    Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

    Unsupervised cross-lingual representation learning at scale , author=. Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

  7. [23]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Crosslingual generalization through multitask finetuning , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  8. [24]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Rlhf can speak many languages: Unlocking multilingual preference optimization for llms , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  9. [25]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  10. [26]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  11. [27]

    Findings of the Association for Computational Linguistics: ACL 2022 , pages=

    Extracting latent steering vectors from pretrained language models , author=. Findings of the Association for Computational Linguistics: ACL 2022 , pages=

    2022

  12. [29]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    The geometry of multilingual language model representations , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  13. [31]

    Word-level Cross-lingual Structure in Large Language Models

    Feng, Zihao and Cao, Hailong and Xu, Wang and Zhao, Tiejun. Word-level Cross-lingual Structure in Large Language Models. Proceedings of the 31st International Conference on Computational Linguistics. 2025

    2025

  14. [32]

    Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, et al. 2023. Mega: Multilingual evaluation of generative ai. In Proceedings of the 2023 Conference on Empirical Methods in Natura...

    2023

  15. [33]

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2(5):6

    2023

  16. [34]

    Tyler A Chang, Zhuowen Tu, and Benjamin K Bergen. 2022. The geometry of multilingual language model representations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 119--136

    2022

  17. [35]

    Cheng-Ting Chou, George Liu, Jessica Sun, Cole Blondin, Kevin Zhu, Vasu Sharma, and Sean O'Brien. 2025. Causal language control in multilingual transformers via sparse feature steering. arXiv preprint arXiv:2507.13410

    2025

  18. [36]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th annual me...

    2020

  19. [37]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages...

    2018

  20. [38]

    Marta R Costa-Juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    2022 arXiv

  21. [39]

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600

    2023 arXiv

  22. [40]

    John Dang, Arash Ahmadian, Kelly Marchisio, Julia Kreutzer, Ahmet \"U st \"u n, and Sara Hooker. 2024. Rlhf can speak many languages: Unlocking multilingual preference optimization for llms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Process...

    2024

  23. [41]

    Boyi Deng, Yu Wan, Baosong Yang, Yidan Zhang, and Fuli Feng. 2025. Unveiling language-specific features in large language models via sparse autoencoders. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4...

    2025

  24. [42]

    Zihao Feng, Hailong Cao, Wang Xu, and Tiejun Zhao. 2025. https://aclanthology.org/2025.coling-main.138/ Word-level cross-lingual structure in large language models . In Proceedings of the 31st International Conference on Computational Linguistics, pages 2026--2037, Abu Dhabi, ...

    2025

  25. [43]

    Neeraja Kirtane and Kuan-Hao Huang. 2026. Language steering for multilingual in-context learning. arXiv preprint arXiv:2602.02326

    2026

  26. [44]

    Jind r ich Libovick \'y , Rudolf Rosa, and Alexander Fraser. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.150 On the language neutrality of pre-trained multilingual representations . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1663-...

    2020 doi

  27. [45]

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, J \'a nos Kram \'a r, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. In Proceedings of the 7th BlackboxN...

    2024

  28. [46]

    Danni Liu and Jan Niehues. 2025. Middle-layer representation alignment for cross-lingual transfer in fine-tuned llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15979--15996

    2025

  29. [47]

    Callum McDougall, Arthur Conmy, J \'a nos Kram \'a r, Tom Lieberum, Senthooran Rajamanoharan, and Neel Nanda. 2025. https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-beha...

    2025

  30. [48]

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. 2023. Crosslingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of th...

    2023

  31. [49]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  32. [50]

    Edoardo Maria Ponti, Goran Glava s , Olga Majewska, Qianchu Liu, Ivan Vuli \'c , and Anna Korhonen. 2020. Xcopa: A multilingual dataset for causal commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2362--2376

    2020

  33. [51]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728--53741

    2023

  34. [52]

    Lisa Schut, Yarin Gal, and Sebastian Farquhar. 2025. Do multilingual llms think in english? arXiv preprint arXiv:2502.15603

    2025 arXiv

  35. [53]

    Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2022. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057

    2022 arXiv

  36. [54]

    Hannah Sterz, Fabian David Schmidt, Goran Glava s , and Ivan Vulic. 2025. Recover the target language: Language steering without sacrificing task performance. Preprint

    2025

  37. [55]

    Nishant Subramani, Nivedita Suresh, and Matthew E Peters. 2022. Extracting latent steering vectors from pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 566--581

    2022

  38. [56]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, ...

    2025 arXiv

  39. [57]

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248

    2023 arXiv

  40. [58]

    Weixuan Wang, Minghao Wu, Barry Haddow, and Alexandra Birch. 2025. Bridging the language gaps in large language models with inference-time cross-lingual intervention. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2025

  41. [59]

    Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do llamas work in english? on the latent language of multilingual transformers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

    2024

  42. [60]

    Zhihui Xie, Handong Zhao, Tong Yu, and Shuai Li. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.379 Discovering low-rank subspaces for language-agnostic multilingual representations . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,...

    2022 doi

  43. [61]

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. 2023. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.