Pith. sign in

REVIEW 5 cited by

Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.16438 v2 pith:TNPWS2RG submitted 2024-02-26 cs.CL

Tianyi Tang , Wenyang Luo , Haoyang Huang , Dongdong Zhang , Xiaolei Wang , Xin Zhao , Furu Wei , Ji-Rong Wen This is my paper
classification cs.CL
keywords llmslanguagemultilinguallanguage-specificneuronscapabilitiesmodelslape
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) demonstrate remarkable multilingual capabilities without being pre-trained on specially curated multilingual parallel corpora. It remains a challenging problem to explain the underlying mechanisms by which LLMs process multilingual texts. In this paper, we delve into the composition of Transformer architectures in LLMs to pinpoint language-specific regions. Specially, we propose a novel detection method, language activation probability entropy (LAPE), to identify language-specific neurons within LLMs. Based on LAPE, we conduct comprehensive experiments on several representative LLMs, such as LLaMA-2, BLOOM, and Mistral. Our findings indicate that LLMs' proficiency in processing a particular language is predominantly due to a small subset of neurons, primarily situated in the models' top and bottom layers. Furthermore, we showcase the feasibility to "steer" the output language of LLMs by selectively activating or deactivating language-specific neurons. Our research provides important evidence to the understanding and exploration of the multilingual capabilities of LLMs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. One Anchor for All: Unified Multilingual and Multimodal Safety Alignment for LVLMs

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A small set of model neurons shared across languages and modalities can transfer English-only safety training to multilingual and multimodal refusal behavior.

  2. Targeted Interpretable Safety Neuron Enhancement for Multilingual Vision-Language Large Models

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    Precise Shield identifies safety neurons in VLLMs via activation contrasts and aligns only them with gradient masking, boosting safety, preserving generalization, and enabling zero-shot cross-lingual and cross-modal transfer.

  3. Sparse Neuron Ablation Triggers Catastrophic Collapse of the Language Core in Large Vision-Language Models

    cs.AI 2025-11 conditional novelty 5.0 of 10

    Ablating just four neurons in LLaVA-1.5-7b's language-model down-projection layer triggers complete output collapse, with critical neurons concentrated in the language backbone.

  4. What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations

    cs.CL 2025-07 reject novelty 5.0 of 10

    Aya-23-8B appears to activate multiple related languages internally and concentrate code-mixing neurons in final layers, but the paper's own limitations undercut the claim that these are language-specific neurons.

  5. Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Pruning the most attribution-prominent MLP neurons in a single layer, chosen via a 10-sample validation sweep, consistently improves multiple-choice accuracy across four instruction-tuned LLMs.

Pith tools