Pith. sign in

REVIEW 4 major objections 6 minor 70 references

From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a model's cross-lingual ability can be read off from which neurons fire on parallel sentences, and that 100 sentence pairs suffice to predict benchmark performance.

desk verdict The headline 0.9556 correlation is a within-model, across-language statistic, not a model-level proxy; the paper still has a solid core for within-model alignment analysis but overstates the headline claim. read the letter →

arxiv 2507.14900 v2 pith:MLVMPGT7 submitted 2025-07-20 cs.CL

Chongxuan Huang , Yongshi Ye , Biao Fu , Qifeng Su , Xiaodong Shi This is my paper
classification cs.CL
keywords cross-lingualalignmentmultilinguallargelanguagemodelsneuronactivationstatesfeedforwardnetworksparallelsentenceszero-shottransferevaluationrepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes NeuronXA, a way to score how well a large language model aligns two languages by looking at which neurons fire rather than at the model's sentence embeddings. The claim is that with only 100 parallel sentence pairs, the NeuronXA score correlates at 0.9556 (Pearson) with the model's performance on multilingual benchmarks and at 0.8514 with its ability to transfer knowledge from one language to another. If true, this gives a cheap, label-free proxy for a model's multilingual ability, since ranking models would no longer require running full benchmarks. The paper also reports that neuron-based sentence representations improve cross-lingual retrieval and reduce the asymmetry between retrieving into English and out of English.

What carries the argument

The load-bearing object is the NeuronXA score, defined in Equation 4 as the proportion of parallel sentence pairs $i$ for which $c_{ii} > c_{ij}$ and $c_{ii} > c_{ji}$ for all $j \neq i$ in the cosine-similarity matrix $C^{(l)}$ of neuron-state sentence representations at layer $l$. The sentence representation (Equation 3) is a position-weighted average $N_l = \sum_t w_t n_{lt}$ with weights $w_t = t / \sum_k k$, applied to binary neuron activation states (activation value greater than zero) or to absolute activation values, so that early tokens do not dominate the sentence vector. This machinery turns the neurobiological idea that similar information activates overlapping neural regions into a testable statistic: if two languages are well aligned, a sentence and its translation should be mutual nearest neighbours in neuron-activation space.

What would settle it

Compute NeuronXA scores after randomly shuffling the alignment between the 100 English sentences and their target-language translations; if the shuffled scores stay close to the true-pair scores, or if they still correlate with benchmark performance, then the metric is not measuring semantic equivalence.

Watch

Extended reading notes

Core claim

The central discovery is that cross-lingual semantic alignment can be measured in the activation patterns of feedforward neurons rather than in the embedding space. Concretely, NeuronXA converts each sentence into a position-weighted average of binary neuron activation states (or absolute activation magnitudes) across the transformer's feedforward layers, forms the cosine-similarity matrix between parallel sentences in two languages, and scores alignment as the proportion of parallel pairs whose similarity is the maximum in both its row and its column (Equation 4). Averaged over layers, this score correlates strongly with downstream multilingual performance and with zero-shot cross-lingual transfer, and it outperforms the embedding-based MEXA baseline. The paper further finds that alignment is highest in middle layers and lowest in bottom and top layers, consistent with lower layers mapping languages into a shared space and upper layers generating language-specific tokens.

Load-bearing premise

The load-bearing premise is that, from only 100 parallel sentence pairs, the proportion of translations that are mutual nearest neighbours in binary neuron-activation space truly measures semantic cross-lingual alignment, rather than superficial token or language statistics.

Editorial extensions

If this is right

  • NeuronXA can rank or screen multilingual LLMs using 100 parallel sentences and no task labels, which would make alignment evaluation much cheaper than running full benchmarks.
  • Because alignment correlates with zero-shot transfer, NeuronXA can be used during development to predict whether a model will transfer to a new language before any fine-tuning on that language is done.
  • The finding that middle layers carry the highest alignment suggests layer-specific interventions: aligning or freezing middle-layer neurons may matter more for multilingual ability than bottom or top layers.
  • The reported correlation with COMET and CometKiwi scores during fine-tuning implies that tracking NeuronXA during training could serve as a monitor of emerging translation quality.
  • Non-English pivot languages such as German, French, and Italian give similar results, so the method is not tied to English as the reference language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the score is truly semantic, it should predict alignment on language pairs not seen in the benchmark set; a direct test would compute NeuronXA on a held-out low-resource language pair and compare it with human translation judgments.
  • Editorial extension: because the score is computed from internal states, it can also be used to localize which neurons or layers carry a model's multilingual ability, potentially guiding pruning or sparse interventions.
  • Editorial extension: the 100-pair result invites a stress test—randomly permuting the target-language sentences should drop the score toward the chance level derived in Appendix E; if it does not, the metric may be capturing language-level regularities rather than meaning.
  • Editorial extension: one could test whether NeuronXA tracks meaning or lexical overlap by evaluating parallel sentences with no shared cognates or by comparing scores across different scripts, such as Hindi versus Urdu.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes NeuronXA, a cross-lingual alignment score computed from feed-forward neuron states of decoder-only LLMs. For a pair of languages, it represents each sentence by a position-weighted average of neuron activations (binary or absolute-value), builds a cosine-similarity matrix over parallel sentences, and defines alignment as the proportion of sentences whose cosine similarity is a mutual row/column maximum (Eq. 4). The authors evaluate NeuronXA on nine LLMs across three multilingual benchmarks (m-ARC, m-MMLU, Belebele) and two zero-shot transfer tasks (XNLI, BMLAMA-53), reporting Pearson correlations between alignment scores and task performance, and claim that with only 100 parallel sentence pairs NeuronXA attains average correlations of 0.9556 and 0.8514 with downstream performance and transferability, respectively.

Significance. If the reported correlations were unbiased and the score genuinely captured cross-lingual semantic alignment, NeuronXA would be a useful internal-state proxy for multilingual ability that is cheap to compute on a small parallel corpus. The paper's strengths include evaluation across a broad set of models and languages, multiple ablations over representation types and pooling strategies, and a formal robustness check against random matrices (Appendix E). The central limitation is that the headline correlations are computed within models across languages, selected post hoc as the best of several configurations, and never compared against the closest prior neuron-based method SADS. These issues currently prevent the paper from supporting its advertised utility as a model-comparison tool.

major comments (4)
  1. [Section 4.3, Tables 3 and 4, Conclusion] The headline claim that NeuronXA achieves Pearson correlations of 0.9556 and 0.8514 is an average of per-model correlations computed across languages within each model. This establishes only that, within a given model, languages with higher NeuronXA scores tend to have higher benchmark accuracy; it does not establish that a model with a higher mean NeuronXA score is more multilingual-capable than another model. No per-model aggregate NeuronXA score is ever compared against per-model benchmark accuracy. Because the abstract and conclusion present NeuronXA as a method for evaluating a model's multilingual capabilities, a model-level analysis is required to support that claim.
  2. [Section 4.3, paragraph beginning 'Across all settings'] The default configuration (position-weighted average plus NASCA) was adopted because it produced the highest average correlations. The reported headline numbers are therefore the maximum over a grid of configurations (three pooling methods times two score types), making them optimistically biased estimates of the metric's predictive power. The paper should either pre-specify the configuration, use a selection-corrected procedure such as cross-validation over languages or models, or report the full distribution of correlations across configurations. Without this, the 0.9556 and 0.8514 figures overstate the reliability of NeuronXA.
  3. [Related Work and Appendix C.2] The closest prior method, SADS (Zeng et al., 2025), also computes cross-lingual alignment from neuron activation values, yet it is only discussed in Related Work and never included in the empirical baseline tables (Tables 8-10 list CKA, SVCCA, ANC, MEXA, NASCA, and NAVCA). Since SADS is the most directly comparable neuron-based approach, the absence of a direct comparison means the claimed advantage of NeuronXA over existing neuron-based alignment evaluation is not established.
  4. [Section 2.2, Eq. (4), and Appendix E] The metric's core assumption is that diagonal dominance in the neuron-state cosine-similarity matrix indicates semantic equivalence of parallel sentences. Appendix E only bounds the probability of high scores under an i.i.d. uniform model; it does not validate that binary neuron states from 100 sentences capture shared cross-lingual semantics rather than language-specific token statistics. The authors should include a direct validation of the score against an independent alignment measure, such as retrieval accuracy on held-out parallel data or human semantic similarity judgments, to support the semantic interpretation.
minor comments (6)
  1. [Section 3.2 and Tables 3-4] The manuscript never states the number of data points used for each Pearson correlation coefficient, such as the number of languages included and whether all 203 FLORES-200 languages are used, nor does it report p-values or confidence intervals. Without this information, the statistical significance of the reported correlations cannot be assessed.
  2. [Section 3.1] The term 'max-pooling' is introduced but not defined; please clarify whether it is applied across layers, across tokens, or across neuron dimensions.
  3. [Section 2.2 and elsewhere] There are several typographical errors, including 'the the dimension' in the text below Eq. (4), 'pivo language' in Section 4.2, 'a lignment' in the abstract, and an unwanted line-break hyphen in 'Muen- nighoff' in the references.
  4. [Section 3.2] The acronyms NASCA and NAVCA are used before their meanings are explicitly defined; please define them at first mention in the main text.
  5. [Appendix E] The binomial calculation in Eq. (5) assumes independence of the n diagonal events, but rows and columns of the similarity matrix overlap, so the events are not strictly independent. The approximation should be stated explicitly, and the sensitivity of the conclusion to this assumption should be checked.
  6. [Table 2, Section 4.2] For the reported NASCA scores in Table 2, it is unclear whether the scores are averaged over layers or taken from a single representative layer; please specify the pooling method used.

Circularity Check

1 steps flagged · score 4.0 of 10

Headline 0.9556/0.8514 correlations are post-hoc selected maxima over method configurations; the underlying NeuronXA-benchmark comparison is otherwise not circular.

  1. fitted input called prediction [Section 4.3, 'Analysis of different sentence representation calculation methods'; reported in Conclusion.]
    "Across all settings, the best overall results (higher correlation) were achieved when embeddings were computed using a weighted average and alignment scores were computed using NASCA, so we adopted this configuration as the default for NeuronXA."

    The default NeuronXA configuration is selected by scanning the correlation tables (Tables 3 and 4) for the variant with the highest Pearson coefficients ('best overall results (higher correlation)'), and the abstract/conclusion then presents those selected numbers ('0.9556' multilingual, '0.8514' transferability) as evidence of NeuronXA's effectiveness. This is the fitted-input-called-prediction pattern: the method variant (weighted average + NASCA) is fit to the very benchmark correlations that are later reported as the headline result. The comparison is not out-of-sample; alternative configurations in the same tables yield visibly lower correlations (e.g., average-pooled NASCA averages about 0.936 across the three multilingual tasks vs.

full rationale

NeuronXA's score (Eq. 4) is defined from neuron activation states of 100 parallel sentences and is computed without using any downstream benchmark label; the correlation with Belebele, m-ARC, m-MMLU, XNLI, and BMLAMA is an external, falsifiable check. There is no self-citation chain, no imported uniqueness theorem, and no equation-level identity between the alignment score and the benchmark outcomes. The only circular step is the explicit post-hoc choice of the weighted-average + NASCA configuration 'best overall results (higher correlation)' followed by reporting the resulting correlation as the paper's headline result. That makes the headline number a fitted maximum, not a pre-specified prediction. The missing model-level correlation (within-model, across-language correlations averaged over models) is a validity and correctness limitation, not circularity, so it does not raise the score further. Score 4 reflects one partial fitted-choice step while the central derivation remains independent.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method adds no trained parameters; the scores are computed from frozen model internals. The main design choices are 100 parallel pairs, the position-weighted averaging rule, and the choice of English as pivot. These are not fitted to the downstream data in a formal sense, but the default configuration was selected post hoc as the best-correlated variant, which contributes to some circularity. The axioms are domain assumptions about neuron interpretability rather than standard math.

free parameters (3)
  • number of parallel sentence pairs = 100
    Evaluation-set size used for all alignment scores; not fitted but a design choice. The robustness analysis (Appendix E) argues 100 is sufficient, but no sweep of n is reported.
  • default representation: position-weighted average = weighted average with NASCA
    Selected post hoc because it gave the highest average correlation (Section 4.3), so the headline 0.9556 is the best of several configurations rather than an independent prediction.
  • pivot language English = English
    The default pivot for computing alignment scores; the paper tests German, French, and Italian as alternatives, but does not test low-resource pivots.
assumptions (4)
  • domain assumption FFN neurons encode semantic knowledge in a cross-lingually comparable form
    Section 2.2 motivates neuron states as intrinsic representations citing prior interpretability work; this is assumed rather than derived.
  • domain assumption Binary activation states preserve enough semantic information for cross-lingual matching
    NASCA uses 0/1 states (Section 3.2); the paper does not show that binary states retain the information needed for sentence equivalence.
  • domain assumption Cosine similarity on neuron-state vectors is an appropriate alignment metric
    Eq. 4 uses cosine similarities; no alternative metrics (e.g., L2, dot product) are explored.
  • domain assumption English is a valid pivot language for all language pairs
    Section 4.2 acknowledges variability but English is the default baseline; the paper shows German/French/Italian work, but low-resource pivots are not tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment." pith.science (2026). https://pith.science/paper/MLVMPGT7

@misc{pith2026250714900,
  author       = {Pith},
  title        = {Pith review of: From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MLVMPGT7}},
  note         = {Machine review of arXiv:2507.14900}
}
read the original abstract

Large language models (LLMs) have demonstrated remarkable multilingual capabilities, however, how to evaluate cross-lingual alignment remains underexplored. Existing alignment benchmarks primarily focus on sentence embeddings, but prior research has shown that neural models tend to induce a non-smooth representation space, which impact of semantic alignment evaluation on low-resource languages. Inspired by neuroscientific findings that similar information activates overlapping neuronal regions, we propose a novel Neuron State-Based Cross-Lingual Alignment (NeuronXA) to assess the cross-lingual a lignment capabilities of LLMs, which offers a more semantically grounded approach to assess cross-lingual alignment. We evaluate NeuronXA on several prominent multilingual LLMs (LLaMA, Qwen, Mistral, GLM, and OLMo) across two transfer tasks and three multilingual benchmarks. The results demonstrate that with only 100 parallel sentence pairs, NeuronXA achieves a Pearson correlation of 0.9556 with downstream tasks performance and 0.8514 with transferability. These findings demonstrate NeuronXA's effectiveness in assessing both cross-lingual alignment and transferability, even with a small dataset. This highlights its potential to advance cross-lingual alignment research and to improve the semantic understanding of multilingual LLMs.

Figures

Figures reproduced from arXiv: 2507.14900 by the authors.

Figure 1
Figure 1. Visualization of sentence representations for [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. NASCA scores across all layers for different languages. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Alignment Score Trends During Supervised Fine-Tuning and Continued Pre-Training of LLaMA-3.1 8B. [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Correlation coefficients between Alignment Scores and COMET/CometKiwi Scores during Supervised [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 32 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, and Sunayana Sitaram. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.258 MEGA : Multilingual evaluation of generative AI . In Proceedings of the 2023 Conference on Empirical Methods in Natural ...

    doi:10.18653/v1/2023.emnlp-main.258 2023

  4. [4]

    Mikel Artetxe and Holger Schwenk. 2019. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. Transactions of the association for computational linguistics, 7:597--610

    work page 2019

  5. [5]

    Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2024. https://doi.org/10.18653/v1/2024.acl-long.44 The belebele benchmark: a parallel reading comprehension dataset in 122 language variants . In Proceedings of the 62nd Annual Meeting of t...

    doi:10.18653/v1/2024.acl-long.44 2024

  6. [6]

    Adithya Bhaskar, Dan Friedman, and Danqi Chen. 2024. The heuristic core: Understanding subnetwork generalization in pretrained language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14351--14368

    work page 2024

  7. [7]

    Sunit Bhattacharya and Ond r ej Bojar. 2023. https://doi.org/10.18653/v1/2023.blackboxnlp-1.9 Unveiling multilinguality in transformer models: Exploring language specificity in feed-forward networks . In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 120--126, Singapore. Association for Computational...

    doi:10.18653/v1/2023.blackboxnlp-1.9 2023

  8. [8]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    arXiv 2018

Show all 70 references
  1. [9]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. https://doi.org/10.18653/v1/D18-1269 XNLI : Evaluating cross-lingual sentence representations . In Proceedings of the 2018 Conference on Empirical Methods ...

    2018 doi

  2. [10]

    Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Emerging cross-lingual structure in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6022--6034

    2020

  3. [11]

    Marta R Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    2022 arXiv

  4. [12]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. https://doi.org/10.18653/v1/2022.acl-long.581 Knowledge neurons in pretrained transformers . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2022 doi

  5. [13]

    Maksym Del and Mark Fishel. 2022. https://doi.org/10.18653/v1/2022.aacl-main.15 Cross-lingual similarity of multilingual representations revisited . In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th I...

    2022 doi

  6. [14]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  7. [16]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024 b . The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  8. [17]

    Kawin Ethayarajh. 2019. https://doi.org/10.18653/v1/D19-1006 How contextual are contextualized word representations? C omparing the geometry of BERT , ELM o, and GPT -2 embeddings . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and t...

    2019 doi

  9. [18]

    Biao Fu, Minpeng Liao, Kai Fan, Chengxi Li, Liang Zhang, Yidong Chen, and Xiaodong Shi. 2025 a . https://arxiv.org/abs/2504.09570 Llms can achieve high-quality simultaneous machine translation as efficiently as offline . Preprint, arXiv:2504.09570

    2025 arXiv

  10. [19]

    Biao Fu, Donglei Yu, Minpeng Liao, Chengxi Li, Yidong Chen, Kai Fan, and Xiaodong Shi. 2025 b . https://arxiv.org/abs/2504.11809 Efficient and adaptive simultaneous speech translation with fully unidirectional architecture . Preprint, arXiv:2504.11809

    2025 arXiv

  11. [20]

    Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tieyan Liu. 2019. https://openreview.net/forum?id=SkEYojRqtm Representation degeneration problem in training natural language generation models . In International Conference on Learning Representations

    2019

  12. [21]

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.446 Transformer feed-forward layers are key-value memories . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484--5495, ...

    2021 doi

  13. [22]

    Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Sch \"o lkopf. 2005. Measuring statistical dependence with hilbert-schmidt norms. In International conference on algorithmic learning theory, pages 63--77. Springer

    2005

  14. [23]

    Jiaxin Guo, Hao Yang, Zongyao Li, Daimeng Wei, Hengchao Shang, and Xiaoyu Chen. 2024. A novel paradigm boosting translation capabilities of large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 639--649

    2024

  15. [24]

    Wes Gurnee, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas. 2024. Universal neurons in gpt2 language models. CoRR

    2024

  16. [25]

    Katharina H \"a mmerl, Jind r ich Libovick \`y , and Alexander Fraser. 2024. Understanding cross-lingual alignment—a survey. In Findings of the Association for Computational Linguistics ACL 2024, pages 10922--10943

    2024

  17. [26]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 a . Measuring massive multitask language understanding. In International Conference on Learning Representations

    2021

  18. [27]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 b . Measuring massive multitask language understanding. In International Conference on Learning Representations

    2021

  19. [28]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations

    2022

  20. [29]

    Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. https://proceedings.mlr.press/v119/hu20b.html XTREME : A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation . In Proceedings of the 37th Inte...

    2020

  21. [30]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  22. [31]

    Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, Jana Diesner, Fran c ois Yvon, and Hinrich Sch \"u tze. 2024. Mexa: Multilingual evaluation of english-centric llms via cross-lingual alignment. arXiv preprint arXiv:2410.05873

    2024 arXiv

  23. [32]

    Takeshi Kojima, Itsuki Okimura, Yusuke Iwasawa, Hitomi Yanaka, and Yutaka Matsuo. 2024. On the multilingual ability of decoder-based pre-trained language models: Finding and controlling language-specific neurons. In Proceedings of the 2024 Conference of the North American Chap...

    2024

  24. [33]

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Similarity of neural network representations revisited. In International conference on machine learning, pages 3519--3529. PMLR

    2019

  25. [34]

    Viet Lai, Chien Nguyen, Nghia Ngo, Thuat Nguyen, Franck Dernoncourt, Ryan Rossi, and Thien Nguyen. 2023. https://doi.org/10.18653/v1/2023.emnlp-demo.28 Okapi: Instruction-tuned large language models in multiple languages with reinforcement learning from human feedback . In Pro...

    2023 doi

  26. [35]

    Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. On the sentence embeddings from pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9119--9130

    2020

  27. [36]

    Jiahuan Li, Shujian Huang, Aarron Ching, Xinyu Dai, and Jiajun Chen. 2024. Prealign: Boosting cross-lingual transfer by early establishment of multilingual alignment. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10246--10257

    2024

  28. [37]

    Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ali Payani, Ninghao Liu, and Mengnan Du. 2025. Language ranker: A metric for quantifying llm performance across high and low-resource languages. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 28186--28194

    2025

  29. [38]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2421--2425

    2024

  30. [39]

    Basel Mousi, Nadir Durrani, Fahim Dalvi, Majd Hawasly, and Ahmed Abdelali. 2024. https://doi.org/10.18653/v1/2024.acl-long.344 Exploring alignment in shared cross-lingual spaces . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volum...

    2024 doi

  31. [40]

    Niklas Muennighoff. 2022. SGPT : GPT sentence embeddings for semantic search. arXiv preprint arXiv:2202.08904

    2022 arXiv

  32. [41]

    Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML'10, page 807–814, Madison, WI, USA. Omnipress

    2010

  33. [42]

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski S...

    2022 arXiv

  34. [43]

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, et al. 2024. 2 olmo 2 furious. arXiv preprint arXiv:2501.00656

    2024 arXiv

  35. [44]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT -4 technical report. arXiv preprint arXiv:2303.08774

    2023 arXiv

  36. [45]

    Chi, Richard Futrell, and Kyle Mahowald

    Isabel Papadimitriou, Ethan A. Chi, Richard Futrell, and Kyle Mahowald. 2021. https://doi.org/10.18653/v1/2021.eacl-main.215 Deep subjecthood: Higher-order grammatical features in multilingual BERT . In Proceedings of the 16th Conference of the European Chapter of the Associat...

    2021 doi

  37. [46]

    Jirui Qi, Raquel Fern \'a ndez, and Arianna Bisazza. 2023. Cross-lingual consistency of factual knowledge in multilingual language models. In The 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  38. [47]

    Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. 2017. Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. Advances in neural information processing systems, 30

    2017

  39. [48]

    Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Wayne Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-specific neurons: The key to multilingual capabilities in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Co...

    2024

  40. [49]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  41. [50]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . In Advances in Ne...

    2017

  42. [51]

    Elena Voita, Javier Ferrando, and Christoforos Nalmpantis. 2024. Neurons in large language models: Dead, n-gram, positional. In Findings of the Association for Computational Linguistics ACL 2024, pages 1288--1301

    2024

  43. [52]

    Hetong Wang, Pasquale Minervini, and Edoardo Ponti. 2024 a . Probing the emergence of cross-lingual alignment during llm training. In Findings of the Association for Computational Linguistics ACL 2024, pages 12159--12173

    2024

  44. [53]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024 b . Improving text embeddings with large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11897--11916

    2024

  45. [54]

    Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.765 Finding skill neurons in pre-trained transformer-based language models . In Proceedings of the 2022 Conference on Empirical Methods in Natural L...

    2022 doi

  46. [55]

    Andrea W Wen-Yi and David Mimno. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.71 Hyperpolyglot LLM s: Cross-lingual interpretability in token embeddings . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1124--1131, Singapor...

    2023 doi

  47. [56]

    Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do llamas work in english? on the latent language of multilingual transformers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

    2024

  48. [57]

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2023 a . A paradigm shift in machine translation: Boosting translation performance of large language models. In The Twelfth International Conference on Learning Representations

    2023

  49. [58]

    Ningyu Xu, Qi Zhang, Jingting Ye, Menghan Zhang, and Xuanjing Huang. 2023 b . https://doi.org/10.18653/v1/2023.findings-emnlp.931 Are structural concepts universal in transformer language models? towards interpretable cross-lingual generalization . In Findings of the Associati...

    2023 doi

  50. [59]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  51. [60]

    Jian Yang, Shuming Ma, Dongdong Zhang, ShuangZhi Wu, Zhoujun Li, and Ming Zhou. 2020. https://doi.org/10.1609/aaai.v34i05.6480 Alternating language modeling for cross-lingual pre-training . Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):9386--9393

    2020 doi

  52. [61]

    Jiacheng Ye, Xijia Tao, and Lingpeng Kong. 2023. Language versatilists vs. specialists: An empirical revisiting on multilingual transfer ability. arXiv preprint arXiv:2306.06688

    2023 arXiv

  53. [62]

    Yongshi Ye, Biao Fu, Chongxuan Huang, Yidong Chen, and Xiaodong Shi. 2025. https://arxiv.org/abs/2505.19987 How well do large reasoning models translate? a comprehensive evaluation for multi-domain machine translation . Preprint, arXiv:2505.19987

    2025 arXiv

  54. [63]

    Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. CoRR

    2024

  55. [64]

    Hongchuan Zeng, Senyu Han, Lu Chen, and Kai Yu. 2025. https://aclanthology.org/2025.coling-main.707/ Converging to a lingua franca: Evolution of linguistic regions and semantics alignment in multilingual large language models . In Proceedings of the 31st International Conferen...

    2025

  56. [65]

    Yidan Zhang, Yu Wan, Boyi Deng, Baosong Yang, Haoran Wei, Fei Huang, Bowen Yu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. https://arxiv.org/abs/2411.09116 P-mmeval: A parallel multilingual multitask benchmark for consistent evaluation of llms . Preprint, arXiv:2411.09116

    2025

  57. [66]

    Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, and Jie Zhou. 2023. Emergent modularity in pre-trained transformers. In Proceedings of ACL: Findings, pages 4066--4083

    2023

  58. [67]

    Zhihao Zhang, Jun Zhao, Qi Zhang, Tao Gui, and Xuan-Jing Huang. 2024. Unveiling linguistic regions in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6228--6247

    2024

  59. [68]

    Jun Zhao, Zhihao Zhang, Qi Zhang, Tao Gui, and Xuanjing Huang. 2024 a . Llama beyond english: An empirical study on language capability transfer. arXiv preprint arXiv:2401.01055

    2024 arXiv

  60. [69]

    Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024 b . How do large language models handle multilingualism? In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  61. [70]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

    2024 arXiv

  62. [71]

    Wenhao Zhu, Shujian Huang, Fei Yuan, Shuaijie She, Jiajun Chen, and Alexandra Birch. 2024. Question translation training for better multilingual reasoning. In Findings of the Association for Computational Linguistics ACL 2024, pages 8411--8423

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.