Pith. sign in

REVIEW 11 cited by

Do Multilingual LLMs Think In English?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.15603 v1 pith:SY25B76X submitted 2025-02-21 cs.CL cs.AIcs.LG

Lisa Schut , Yarin Gal , Sebastian Farquhar This is my paper
classification cs.CLcs.AIcs.LG
keywords englishllmslanguagemultilinguallanguagesrepresentationrepresentationssteering
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) have multilingual capabilities and can solve tasks across various languages. However, we show that current LLMs make key decisions in a representation space closest to English, regardless of their input and output languages. Exploring the internal representations with a logit lens for sentences in French, German, Dutch, and Mandarin, we show that the LLM first emits representations close to English for semantically-loaded words before translating them into the target language. We further show that activation steering in these LLMs is more effective when the steering vectors are computed in English rather than in the language of the inputs and outputs. This suggests that multilingual LLMs perform key reasoning steps in a representation that is heavily shaped by English in a way that is not transparent to system users.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

    cs.CL 2026-08 conditional novelty 7.0 of 10

    SAE-based steering of target-language features improves multilingual inference accuracy on XCOPA, XNLI, and MGSM without parameter updates.

  2. Verbalizable Representations Form a Global Workspace in Language Models

    cs.CL 2026-07 conditional novelty 7.0 of 10

    Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.

  3. Humans overrely on overconfident language models, across languages

    cs.CL 2025-07 conditional novelty 7.0 of 10

    LLMs produce overconfident-sounding answers in all five tested languages, and bilingual users show the highest overreliance risk in Japanese despite its frequent hedges.

  4. Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting

    cs.AI 2025-10 conditional novelty 6.0 of 10

    An early-exit rule with a zero-shot fallback, calibrated by Learn-then-Test risk control, keeps the average loss from corrupted in-context demonstrations under a preset bound.

  5. Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG

    cs.CL 2025-09 conditional novelty 6.0 of 10

    In multilingual retrieval-augmented generation, models cite English evidence more accurately than translated evidence, and this language preference can outweigh document relevance.

  6. TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla

    cs.CL 2025-09 conditional novelty 6.0 of 10

    TigerCoder is a fine-tuned Bangla code-generation LLM family that posts 0.82 Pass@1 on the new MBPP-Bangla benchmark, but its gains partly come from model selection on the test set.

  7. Under the Shadow of Babel: How Language Shapes Reasoning in LLMs

    cs.CL 2025-06 conditional novelty 6.0 of 10

    LLMs distribute attention differently for Chinese and English causal sentences, lose more accuracy on reversed Chinese order, yet converge to similar representations when they reason correctly.

  8. The Emergence of Abstract Thought in Large Language Models Beyond Any Language

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Across 20 open LLMs, shared multilingual neurons grow in number and per-neuron importance over release generations, which the authors interpret as evidence of language-agnostic abstract thought and use to guide neuron...

  9. What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations

    cs.CL 2025-07 reject novelty 5.0 of 10

    Aya-23-8B appears to activate multiple related languages internally and concentrate code-mixing neurons in final layers, but the paper's own limitations undercut the claim that these are language-specific neurons.

  10. CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation

    cs.CL 2025-07 conditional novelty 5.0 of 10

    CCL-XCoT combines curriculum-based contrastive pretraining with cross-lingual chain-of-thought fine-tuning, lifting hallucination-free rates in low-resource QA from 1-18% to 55-74%.

  11. Text2Cypher Across Languages: Evaluating and Finetuning LLMs

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A new multilingual Text2Cypher benchmark shows LLMs rank English highest, Spanish next, and Turkish lowest, and multilingual finetuning narrows the language gap more than English-only finetuning.

Pith tools