CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning

Bing Qin; Dandan Tu; Duyu Tang; Lei Huang; Libo Qin; Weitao Ma; Xiachong Feng; Xiaocheng Feng; Xiaohui Yan; Yangfan Ye

REVIEW 3 major objections 6 minor 54 references

CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CC-Tuning claims that explicitly fusing English feed-forward activations into non-English forward passes during supervised fine-tuning improves multilingual performance beyond vanilla SFT, serving as a latent-level alternative to…

desk verdict A practical latent-fusion trick for multilingual SFT with real but uneven gains; the cross-lingual mechanism itself isn't isolated, so treat the causal claim with caution. read the letter →

arxiv 2506.00875 v1 pith:7IHVQ6GF submitted 2025-06-01 cs.CL

Yangfan Ye , Xiaocheng Feng , Zekun Yuan , Xiachong Feng , Libo Qin , Lei Huang , Weitao Ma , Yichong Huang

show 6 more authors

Zhirui Zhang Yunfei Lu Xiaohui Yan Duyu Tang Dandan Tu Bing Qin

This is my paper

classification cs.CL

keywords multilingualsupervisedfine-tuningcross-linguallatentinteractionfeed-forwardactivationsDecisionMakerTransformMatrixLLMdataaugmentationalternativecurseofmultilinguality

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CC-Tuning is a multilingual fine-tuning method that improves non-English performance by explicitly connecting English and non-English representations inside the model, rather than by adding translated training text. During training it runs each non-English input alongside its English translation, extracts feed-forward activations from the English pass, selects the most useful layer with a trainable Decision Maker, and adds that activation into the first feed-forward layer of the non-English pass. The loss function is unchanged, and the added parameters are negligible. At inference the method needs no parallel input: a Transform Matrix learned from 1,000 parallel pairs maps non-English activations into English space to reproduce the connection. Across six benchmarks and 22 languages on two 7–8B models, the paper reports that CC-Tuning outperforms vanilla supervised fine-tuning and is competitive with or better than data-level augmentation and distillation baselines.

What carries the argument

The central object is the cross-lingual connection mechanism applied to feed-forward activations just before the response start token. It has three parts: (1) an Adaptive Decision Maker, a trainable linear layer mapping combined English activations and the non-English embedding to a distribution over layers, with Gumbel-Softmax selecting a beneficial layer $s$; (2) a Latent Feed Forward Connection, the injection $\tilde{f}_{i,1} = f_{i,1} + f^{\mathrm{en}}_{i,s}$ into the first decoder layer's feed-forward activation; and (3) a Transform Matrix $W_T$, fit by least squares on 1,000 parallel pairs with closed-form solution, which maps non-English activations $F_i$ to English activations $F^{\mathrm{en}}_i$ at inference. The first two parts carry the training-time benefit; the third carries the practical, monolingual inference-time simulation.

What would settle it

A decisive test is to evaluate the Transform Matrix's per-language residual $\|f_{i,l}W_T - f^{\mathrm{en}}_{i,l}\|_2$ for languages not among the 1,000 parallel pairs used to fit it. If typologically distant or low-resource languages show residuals one order of magnitude above the average, the single linear map assumption fails and the inference-time simulation is not genuinely cross-lingual. A second check is to compare CC-Tuning's result using true English parallel activations against its result using $W_T$ language-by-language: the paper reports small average $|\Delta|$, but the claim would be falsified if the gap grows systematically with language distance.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that cross-lingual alignment can be engineered at the activation level: a model jointly fine-tuned on multilingual data benefits when the feed-forward activation of an English auxiliary input is injected into the first feed-forward layer of the non-English input at the response start token. A trainable linear Decision Maker with Gumbel-Softmax selects which English layer's activation is beneficial, making the injection adaptive. Because the training objective stays the vanilla negative log-likelihood, the model and the Decision Maker learn together to use both linguistic resources. At inference, a closed-form least-squares Transform Matrix, fit on 1,000 parallel pairs, transforms non-English activations into their English counterparts so the same fusion can be simulated on monolingual input. The paper reports that this mechanism outperforms vanilla SFT and gives a strong latent-level alternative to data-level augmentation methods, with the caveat that the gain disappears when the auxiliary input is also non-English or when English dominates the training data.

Load-bearing premise

The method assumes that one linear map, learned from 1,000 parallel pairs, can reliably turn a non-English model's internal feed-forward activations into the English activations that would have helped during training, across all layers and languages.

Editorial extensions

If this is right

Because CC-Tuning adds no training objective and only a tiny linear layer, it can be layered onto existing multilingual SFT pipelines without changing the loss or data schedule.
The Transform Matrix converges with roughly 1,000 parallel pairs, so the method remains practical in low-resource settings where full parallel corpora are unavailable.
Under the +EN setting the benefit disappears, implying the mechanism is specifically a cross-lingual bridge rather than a general activation-mixing trick.
The Decision Maker concentrates on middle layers, suggesting that cross-lingual transfer during fine-tuning is largely a middle-layer phenomenon for feed-forward activations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the linear Transform Matrix is the bottleneck, per-layer or per-language transforms (or a small learned network) should reduce the inference gap, and this is directly testable with the paper's own MSE metric.
Editorial inference: the paper's best combinations with +MT hint that latent-level fusion and data-level augmentation are complementary; a combined pipeline may push multilingual performance further than either alone.
Editorial inference: the middle-layer concentration suggests that a cheaper intervention—transplanting or fusing only a small band of layers instead of the whole stack—could capture most of the benefit at lower compute.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a read before deciding what to do with it. The core idea is new and simple: during multilingual SFT, take the FFN activations from the English translation of the training input, pick a layer with a learnable Gumbel-Softmax gate (the Decision Maker), and add that activation into the first-layer residual stream of the non-English input. At inference you don't have the parallel text, so they fit one linear transform to approximate English activations from non-English activations, and use that to simulate the connection. That's a clean, low-overhead intervention: around 1.15x training time, negligible extra parameters, and it works on both NLU and generation tasks across 22 languages.

What the paper does well: it takes the prior observation that English FFN activations can be transplanted cross-lingually (Ye et al. 2024b) and turns it into a practical training procedure with a sensible inference-time approximation. The ablations are informative — Decision Maker helps over mean/random pooling, FFN activations beat attention/block activations, and the layer-distribution analysis is a nice diagnostic. On LLaMA-3.1-8B the gains look solid: +6.54 XNLI, +5.37 XStoryCloze, +1.21 XQuAD. On Qwen2.5-7B the gains are small and sometimes negative; the paper reports them honestly rather than cherry-picking.

Soft spots, in rough order of importance. First, no error bars or multiple seeds, and some Qwen deltas are a fraction of a point, so I can't tell which of those are real. Second, the Transform Matrix's MSE is computed on the same data used to fit it, which is circular; the downstream |Δ| numbers in Table 2 are the meaningful check and those look fine. Third — and this is the one I'd want a referee to push on — nothing in the paper isolates the cross-lingual content of the auxiliary activation. The mechanism is called cross-lingual, but the ablations always use English activations as the auxiliary input. A same-language mismatched activation or a content-matched random vector would tell you whether the gains come from English linguistic information or from a generic latent perturbation that regularizes training. The +EN degradation is not a clean control because it changes the data distribution. This is a real hole in the mechanistic story, but the headline empirical claim — CC-Tuning beats vanilla SFT — would survive even if the effect turns out to be generic latent augmentation.

Who's it for: anyone working on multilingual instruction tuning or latent-space interventions. It deserves a serious referee; I'd send it out, with the missing control and variance estimates as requested revisions.

Referee Report

3 major / 6 minor

Summary. The paper proposes CC-Tuning, a multilingual supervised fine-tuning method that operates on latent feed-forward activations. For each non-English training example, an English translation is also passed through the model; a trainable Decision Maker selects one layer's feed-forward activation, and that activation is added to the first-layer feed-forward output of the non-English input before response generation (Eqs. 3-5). The training objective is unchanged from standard SFT. At inference, a Transform Matrix W_T, fit by least squares on 1,000 parallel pairs (Eqs. 7-8), maps non-English feed-forward activations into English-like activations, so that parallel inputs are not needed. The paper evaluates on six benchmarks covering 22 languages, comparing CC-Tuning with vanilla multilingual SFT and with data-augmentation baselines (+EN, +MT, +SDRRL) on LLaMA-3.1-8B and Qwen2.5-7B. The reported results show larger gains for LLaMA-3.1-8B and mixed or negligible gains for Qwen2.5-7B.

Significance. The work is significant if the mechanism is confirmed: it proposes an explicit latent-level alternative to data-level augmentation, requires no additional training loss, adds negligible parameters, and includes a practical inference-time procedure. The experimental coverage is broad, and the design is motivated by prior evidence that English feed-forward activations can improve non-English performance. However, the current evidence is not yet convincing at the claimed level. The main results lack multiple seeds or error bars, the Transform Matrix is validated in-sample on its own fitting objective, and the ablations do not isolate English-specific content from generic auxiliary or regularizing signals. These are empirical gaps that can be addressed in revision; the conceptual proposal remains interesting.

major comments (3)

[Table 1, Section 4.2] The claim that CC-Tuning 'significantly outperforms' vanilla SFT is not supported by the reported numbers. For Qwen2.5-7B, MMMLU decreases by 0.40 points and MKQA, XQuAD, and XLSum improve by only +0.11, +0.11, and +0.10 points, respectively; these differences are within the typical run-to-run variance of 7B-scale instruction tuning. No standard deviations, confidence intervals, or multiple seeds are reported for any result. The LLaMA-3.1-8B results are more favorable (e.g., XNLI +6.54, XStoryCloze +5.37), but even those lack variance estimates. I ask for at least three seeds with means and standard deviations for the main CC-Tuning vs. ML-SFT comparison, or a paired significance test across languages, before the superiority claim can be accepted.
[Section 4.3(1), Eqs. (7)-(8), Table 2] The MSE used to validate the Transform Matrix is computed on the same 1,000 parallel pairs used to fit W_T. Since W_T is the least-squares minimizer of exactly that objective, a low in-sample MSE is expected by construction and does not show that the mapping transfers to new inputs or languages. The |Δ| results compare parallel-bilingual inference with Transform-Matrix inference in the same in-sample setting. Please supplement with hold-out or cross-validated MSE and downstream results on examples not used to fit W_T. A per-language or per-layer breakdown would also help assess the single-linear-matrix assumption.
[Section 3.2.1, Eq. (5), Figures 3-4] No condition varies the source of the injected activation, so the specific 'cross-lingual' attribution is not established. The auxiliary vector is always the English feed-forward activation from the parallel translation of the same input, and the ablations only vary the pooling strategy and the activation type while keeping English as the source. The observed gains could therefore come from content-aligned auxiliary information, from any language's translation, or from a regularizing effect of injecting a task-relevant vector into the first residual stream. The +EN condition in Table 1 is not a clean control because it also changes the training data so that more than half is English. I recommend controls that replace f_en_{i,s} with: (i) a same-language activation from a different example, (ii) a norm-matched random vector, and (iii) a third-language translation of the same input.

minor comments (6)

[Eq. (8)] The summation index in the second term is written 'LX i=l'; this appears to be a typo for L and l=1, and the first term has a similar issue. Please correct the limits.
[Table 8] The language header for XQuAD (en, ar, bn, es, hi, id, ko, pt, sw, yo) is inconsistent with the language list given in Appendix A.2.1 for XQuAD (en, ar, de, el, hi, ru, th, tr, vi, zh). Please correct the table header and verify the per-language values.
[Figure 7] The t-SNE discussion is purely qualitative. Please add a quantitative measure of clustering or alignment, or clarify what 'more compact' is relative to; otherwise, avoid drawing strong conclusions from the visualization.
[Abstract] The code link is given as 'CC-Tuning' without a URL; a revision should provide the actual repository address.
[Section 3.2.2] The Transform Matrix is called 'training-free' even though it is fit on 1,000 parallel pairs. Consider calling it 'without gradient-based training' or 'post-hoc' to avoid confusion.
[Eqs. (3)-(4)] Please specify the Gumbel-Softmax temperature and whether hard or soft samples are used during training and inference; this affects the interpretation of the Decision Maker's layer selection.

Circularity Check

1 steps flagged · score 3.0 of 10

Transform Matrix alignment is verified with its own least-squares objective, but CC-Tuning's main results are independent of this fit.

fitted input called prediction [Section 3.2.2, Eqs. 7-8; Section 4.3(1), Table 2]
"To minimize the difference A and B, our objective is defined as follows (Least-Squares optimization): W∗T = argmin ... (7) ... We verify whether the Transform Matrix WT can effectively achieve the alignment by evaluating the mean squared error (MSE) between fi,l·WT and fen i,l ... The results in Table 2 show that the MSE value reaches the order of magnitude as low as 10−2, indicating that the Transform Matrix effectively transforms fi,l into fen i,l."

The Table 2 MSE is the same per-coordinate squared residual minimized in Eq. 7 to fit WT (up to the 1/(N L d) scaling), and the paper does not state that the pairs used for the MSE evaluation are disjoint from the 1,000 pairs used to fit WT. As reported, the low MSE is the in-sample training objective, so it is a restatement of the fit rather than an independent alignment check. The independent evidence is the |Δ| downstream performance gap between parallel-bilingual and Transform-Matrix inference, so this circularity affects a supporting analysis, not the paper's central CC-Tuning-versus-SFT claim.

full rationale

CC-Tuning's central claim is that the training-time fusion of English and source-language FFN activations (Eq. 5) improves multilingual SFT. This is an empirical claim: the model and Decision Maker are trained with the unchanged vanilla loss (Eq. 1), and the gains in Table 1 are measured on held-out benchmarks, so the improvement is not an artifact of the objective. The Transform Matrix is a fitted auxiliary at inference; its effectiveness is supported by the downstream |Δ| values in Table 2, which are task-performance differences independent of the least-squares fit. The only circular element I found is that the reported MSE alignment evidence is the same squared residual minimized in Eq. 7, with no stated held-out split, so that particular diagnostic is in-sample. The citation to Ye et al. (2024b) is a motivational self-citation for using FFN activations; the paper's own FFN-vs-attention-vs-block ablation (Figure 4) and main results provide independent support, so it is not load-bearing. The absent control for the source/content of the auxiliary activation is an attribution concern, not a circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The method introduces two fitted components (W_DM, W_T) plus a set of domain assumptions about activation transfer. No external benchmarks beyond the paper's experiments validate these components.

free parameters (2)

Decision Maker weights (W_DM)
Trainable linear layer in R^{d x L} that selects beneficial English FFN activations; learned during SFT (Eq. 3-4).
Transform Matrix (W_T)
d x d matrix fit by least squares on 1,000 parallel pairs to map non-English to English FFN activations (Eq. 7-8); used at inference.

assumptions (4)

domain assumption Feed-forward activations from English can enhance non-English performance when fused into the non-English stream.
Motivated by Ye et al. (2024b); assumed without independent verification in this paper.
standard math The Decision Maker's Gumbel-Softmax is differentiable and trainable end-to-end with the vanilla SFT loss.
Gumbel-Softmax is a standard reparameterization technique; the paper assumes it works in this training loop.
domain assumption A single linear map W_T can approximate the mapping from non-English to English FFN activations across all layers and languages.
This is the core assumption behind the inference-time Transform Matrix (Section 3.2.2).
domain assumption The additive fusion at the first decoder layer (Eq. 5) is a sufficient intervention point for cross-lingual transfer.
No ablation over intervention layers is provided; the authors fix the fusion to layer 1.

invented entities (2)

Decision Maker
purpose: A trainable linear layer that identifies which English FFN activation layer is beneficial for the non-English input.
It is a new model component introduced by CC-Tuning; its benefit is only demonstrated in the paper's experiments (Figure 3).
Transform Matrix
purpose: A linear map that simulates the cross-lingual connection at inference without parallel inputs.
It is fit to the paper's own activation data and has no external falsifiable handle.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning." pith.science (2026). https://pith.science/paper/7IHVQ6GF

@misc{pith2026250600875,
  author       = {Pith},
  title        = {Pith review of: CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7IHVQ6GF}},
  note         = {Machine review of arXiv:2506.00875}
}

read the original abstract

Current large language models (LLMs) often exhibit imbalanced multilingual capabilities due to their English-centric training corpora. To address this, existing fine-tuning approaches operating at the data-level (e.g., through data augmentation or distillation) typically introduce implicit cross-lingual alignment, overlooking the potential for more profound, latent-level cross-lingual interactions. In this work, we propose CC-Tuning, a novel multilingual fine-tuning paradigm that explicitly establishes a cross-lingual connection mechanism at the latent level. During training, CC-Tuning fuses the feed forward activations from both English and non-English inputs, enabling the model to benefit from both linguistic resources. This process is facilitated with a trainable Decision Maker that identifies beneficial activations. Furthermore, during inference, a Transform Matrix is utilized to simulate the cross-lingual connection under monolingual setting through representation transformation. Our experiments on six benchmarks covering 22 languages show that CC-Tuning outperforms vanilla SFT and offers a strong latent-level alternative to data-level augmentation methods. Further analysis also highlights the practicality of CC-Tuning and the potential of latent-level cross-lingual interactions in advancing the multilingual performance of LLMs.

Figures

Figures reproduced from arXiv: 2506.00875 by the authors.

**Figure 2.** Overview of the cross-lingual connection mechanism in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Performance comparisons of using Decision Maker, Mean Pooling and Random Pooling strategy on XNLI and MKQA datasets. f en i,l as well as the performance difference |∆| between using parallel bilingual inputs during inference and applying the Transform Matrix. The results in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Performance comparisons of utilizing feed [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 6.** Figure 6: The training loss curves of vanilla supervised [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: t-SNE visualizations of output representations [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: The distribution of the layer with the highest [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

**Figure 9.** Figure 9: Examples of the deviations from the prompt [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 10 canonical work pages

[1]

Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. https://doi.org/10.18653/v1/N19-1388 Massively multilingual neural machine translation . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 3874--3884, Minneapolis, M...

doi:10.18653/v1/n19-1388 2019
[2]

Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403

arXiv 2023
[3]

Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. https://doi.org/10.18653/v1/2020.acl-main.421 On the cross-lingual transferability of monolingual representations . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623--4637, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.acl-main.421 2020
[4]

Pinzhen Chen, Shaoxiong Ji, Nikolay Bogoychev, Andrey Kutuzov, Barry Haddow, and Kenneth Heafield. 2024. https://aclanthology.org/2024.findings-eacl.90/ Monolingual or multilingual instruction tuning: Which makes a better alpaca . In Findings of the Association for Computational Linguistics: EACL 2024, pages 1347--1356, St. Julian ' s, Malta. Association ...

work page 2024
[5]

Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, et al. 2023. Phoenix: Democratizing chatgpt across languages. arXiv preprint arXiv:2304.10453

arXiv 2023
[6]

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

doi:10.18653/v1/2020.acl-main.747 2020
[7]

Bowman, Holger Schwenk, and Veselin Stoyanov

Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

2018
[8]

Yiming Cui, Ziqing Yang, and Xin Yao. 2023. Efficient and effective text encoding for chinese llama and alpaca. arXiv preprint arXiv:2304.08177

arXiv 2023

Show all 54 references

[9]

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. https://doi.org/10.18653/v1/2022.acl-long.581 Knowledge neurons in pretrained transformers . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

2022 doi
[10]

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2023. https://arxiv.org/abs/2301.00234 A survey for in-context learning . ArXiv preprint, abs/2301.00234

2023 arXiv
[11]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

2024 arXiv
[12]

Gabriel Lino Garcia, Pedro Henrique Paiola, Luis Henrique Morelli, Giovani Candido, Arnaldo C \^a ndido J \'u nior, Danilo Samuel Jodas, Luis Afonso, Ivan Rizzo Guilherme, Bruno Elias Penteado, and Jo \ a o Paulo Papa. 2024. Introducing bode: A fine-tuned large language model ...

2024 arXiv
[13]

Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M

Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021. https://aclanthology.org/2021.findings-acl.413 XL -sum: Large-scale multilingual abstractive summarization for 44 languages . In Findin...

2021
[14]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

2020 arXiv
[15]

HIT-SCIR. 2024. Chinese-mixtral-8x7b: An open-source mixture-of-experts llm. https://github.com/HIT-SCIR/Chinese-Mixtral-8x7B

2024
[16]

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Info...

2025
[17]

Wenshuai Huo, Xiaocheng Feng, Yichong Huang, Chengpeng Fu, Baohang Li, Yangfan Ye, Zhirui Zhang, Dandan Tu, Duyu Tang, Yunfei Lu, et al. 2025. Enhancing non-english capabilities of english-centric large language models through deep supervision fine-tuning. In Proceedings of th...

2025
[18]

Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144

2016 arXiv
[19]

Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean

Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. https://doi.org/10.1162/tacl_a_00065 G oogle`s multilingual neural machine translatio...

2017 doi
[20]

Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023. Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation. arXiv preprint arXiv:2305.15011

2023 arXiv
[21]

o rg Tiedemann, Andr \'e FT Martins, and Hinrich Sch \

Peiqin Lin, Shaoxiong Ji, J \"o rg Tiedemann, Andr \'e FT Martins, and Hinrich Sch \"u tze. 2024. Mala-500: Massive language adaptation of large language models. arXiv preprint arXiv:2401.13303

2024 arXiv
[22]

Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O ' Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, M...

2022 doi
[23]

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1--35

2023
[24]

Shayne Longpre, Yi Lu, and Joachim Daiber. 2021. https://doi.org/10.1162/tacl_a_00433 MKQA : A linguistically diverse benchmark for multilingual open domain question answering . Transactions of the Association for Computational Linguistics, 9:1389--1406

2021 doi
[25]

Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2022. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786

2022 arXiv
[26]

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

2022
[27]

Xingyuan Pan, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, and Shanbo Cheng. 2024. https://doi.org/10.18653/v1/2024.acl-long.821 G - DIG : Towards gradient-based DI verse and hi G h-quality instruction data selection for machine translation . In Proceedings of the 62nd Annua...

2024 doi
[28]

Ramon Pires, Hugo Abonizio, Thales Sales Almeida, and Rodrigo Nogueira. 2023. Sabi \'a : Portuguese large language models. In Brazilian Conference on Intelligent Systems, pages 226--240. Springer

2023
[29]

Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S Yu. 2024. Multilingual large language model: A survey of resources, taxonomy and frontiers. arXiv preprint arXiv:2404.04925

2024 arXiv
[30]

Leonardo Ranaldi, Giulia Pucci, and Andre Freitas. 2024. https://doi.org/10.18653/v1/2024.findings-acl.473 Empowering cross-lingual abilities of instruction-tuned large language models by translation-following demonstrations . In Findings of the Association for Computational L...

2024 doi
[31]

Andrea Santilli and Emanuele Rodolà. 2023. https://arxiv.org/abs/2307.16456 Camoscio: an italian instruction-tuned llama . Preprint, arXiv:2307.16456

2023 arXiv
[32]

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili \'c , Daniel Hesslow, Roman Castagn \'e , Alexandra Sasha Luccioni, Fran c ois Yvon, Matthias Gall \'e , et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2...

2022 arXiv
[33]

Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, and Matan Eyal. 2024. https://doi.org/10.18653/v1/2024.findings-acl.136 Multilingual instruction tuning with just a pinch of multilinguality . In Findings of the Association for Computational Linguistics:...

2024 doi
[34]

Murray Shanahan. 2022. https://arxiv.org/abs/2212.03551 Talking about large language models . ArXiv preprint, abs/2212.03551

2022 arXiv
[35]

Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh F...

2024 arXiv
[36]

Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint arXiv:2008.00401

2020 arXiv
[37]

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

2023 arXiv
[38]

Ahmet \"U st \"u n, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, et al. 2024. Aya model: An instruction finetuned open-access multilingual language model. arXiv preprint arXiv:2402.07827

2024 arXiv
[39]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11)

2008
[40]

Lipton, and Yulia Tsvetkov

Zirui Wang, Zachary C. Lipton, and Yulia Tsvetkov. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.359 On negative interference in multilingual models: Findings and a meta-learning treatment . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Pro...

2020 doi
[41]

Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022 a . https://arxiv.org/abs/2206.07682 Emergent abilities of large language models . ArXiv preprint, abs/2206.07682

2022 arXiv
[42]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 b . Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824--24837

2022
[43]

Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie, et al. 2023. Polylm: An open source polyglot large language model. arXiv preprint arXiv:2307.06018

2023 arXiv
[44]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

2024 arXiv
[45]

Jiacheng Ye, Xijia Tao, and Lingpeng Kong. 2023. Language versatilists vs. specialists: An empirical revisiting on multilingual transfer ability. arXiv preprint arXiv:2306.06688

2023 arXiv
[46]

Yangfan Ye, Xiachong Feng, Xiaocheng Feng, Weitao Ma, Libo Qin, Dongliang Xu, Qing Yang, Hongtao Liu, and Bing Qin. 2024 a . https://doi.org/10.18653/v1/2024.emnlp-main.603 G lobe S umm: A challenging benchmark towards unifying multi-lingual, cross-lingual and multi-document n...

2024 doi
[47]

Yangfan Ye, Xiaocheng Feng, Xiachong Feng, Libo Qin, Yichong Huang, Lei Huang, Weitao Ma, Zhirui Zhang, Yunfei Lu, Xiaohui Yan, et al. 2024 b . Xtransplant: A probe into the upper bound performance of multilingual capability and culture adaptability in llms via mutual cross-li...

2024 arXiv
[48]

Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.491 Don ' t trust C hat GPT when your question is not in E nglish: A study of multilingual abilities and types of LLM s . In Proceedings of the 2023 Conferen...

2023 doi
[49]

Yuanchi Zhang, Yile Wang, Zijun Liu, Shuo Wang, Xiaolong Wang, Peng Li, Maosong Sun, and Yang Liu. 2024. Enhancing multilingual capabilities of large language models through self-distillation from resource-rich languages. arXiv preprint arXiv:2402.12204

2024 arXiv
[50]

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. https://arxiv.org/abs/2303.18223 A survey of large language models . ArXiv preprint, abs/2303.18223

2023 arXiv
[51]

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

2024 arXiv
[52]

Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023. Extrapolating large language models to non-english by aligning languages. arXiv preprint arXiv:2308.04948

2023 arXiv
[53]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[54]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. https://doi.org/10.18653/v1/N19-1388 Massively multilingual neural machine translation . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 3874--3884, Minneapolis, M...

doi:10.18653/v1/n19-1388 2019

[2] [2]

Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403

arXiv 2023

[3] [3]

Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. https://doi.org/10.18653/v1/2020.acl-main.421 On the cross-lingual transferability of monolingual representations . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623--4637, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.acl-main.421 2020

[4] [4]

Pinzhen Chen, Shaoxiong Ji, Nikolay Bogoychev, Andrey Kutuzov, Barry Haddow, and Kenneth Heafield. 2024. https://aclanthology.org/2024.findings-eacl.90/ Monolingual or multilingual instruction tuning: Which makes a better alpaca . In Findings of the Association for Computational Linguistics: EACL 2024, pages 1347--1356, St. Julian ' s, Malta. Association ...

work page 2024

[5] [5]

Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, et al. 2023. Phoenix: Democratizing chatgpt across languages. arXiv preprint arXiv:2304.10453

arXiv 2023

[6] [6]

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

doi:10.18653/v1/2020.acl-main.747 2020

[7] [7]

Bowman, Holger Schwenk, and Veselin Stoyanov

Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

2018

[8] [8]

Yiming Cui, Ziqing Yang, and Xin Yao. 2023. Efficient and effective text encoding for chinese llama and alpaca. arXiv preprint arXiv:2304.08177

arXiv 2023