Multilingual unification learning, which augments a tiny English chain-of-thought seed set with translated and code-switched solutions, is claimed to match or exceed larger English-only test-time scaling recipes with less data and fewer inference tokens.
LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategy
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Despite being pretrained on multilingual corpora, large language models (LLMs) exhibit suboptimal performance on low-resource languages. Recent approaches have leveraged multilingual encoders alongside LLMs by introducing trainable parameters connecting the two models. However, these methods typically focus on the encoder's output, overlooking valuable information from other layers. We propose \aname (\mname), a framework that integrates representations from all encoder layers, coupled with the \attaname mechanism to enable layer-wise interaction between the LLM and the multilingual encoder. Extensive experiments on multilingual reasoning tasks, along with analyses of learned representations, show that our approach consistently outperforms existing baselines.
fields
cs.CL 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMs
Multilingual unification learning, which augments a tiny English chain-of-thought seed set with translated and code-switched solutions, is claimed to match or exceed larger English-only test-time scaling recipes with less data and fewer inference tokens.