Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

The paper argues that sentence-level machine translation in large language models is carried by a sparse set of attention heads — roughly one percent — split into mostly disjoint 'language heads' that pick the output language and 'translati

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:30 UTC pith:3G7EPLPY

load-bearing objection Solid empirical MI paper; the sparse-head decomposition mostly holds up, but the corruption-based subtask isolation is cleaner in presentation than in practice. the 3 major comments →

arxiv 2602.04613 v2 pith:3G7EPLPY submitted 2026-02-04 cs.CL

Translation Heads: Disentangling meaning from language in LLM-based machine translation

Th\'eo Lasnier , Armel Zebaze , Djam\'e Seddah , Rachel Bawden , Beno\^it Sagot This is my paper
classification cs.CL
keywords machine translationmechanistic interpretabilityattention headsactivation patchingsteering vectorstarget language identificationsentence equivalencein-context learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that LLM sentence-level translation is mechanistically localized: a tiny fraction of attention heads (~1%) causally drives the task, and this set divides into two functional groups — language heads that determine the target language and translation heads that maintain meaning equivalence with the source. Using activation patching on contrastive few-shot prompts, the authors identify these heads across three model families and twenty language directions, finding the two groups mostly disjoint (Jaccard index around 0.13). They then build steering vectors from the heads' averaged activations and show that steering the top 1% of each group under a prompt-free zero-shot setting matches or approaches instructed zero-shot translation quality, while ablating them selectively destroys the corresponding function. If true, this means translation capability can be elicited and disabled through a very small, targeted set of components, and that 'which language' and 'what it means' are represented separately inside the model.

Core claim

The central claim is that LLM sentence-level MT decomposes into two causally separable subtasks, each mediated by its own sparse set of attention heads. Language heads — identified by corrupting demonstration targets into a different language while preserving meaning — control the output language; translation heads — identified by replacing targets with random same-language sentences — control sentence equivalence. Repeatedly, only 5–10 heads out of thousands stand out, the same heads recur across translation directions, and the two groups barely overlap. Steering the average activations of just 1% of each group reproduces translation quality comparable to an explicit instruction (e.g., 33.3

What carries the argument

The central object is the attention head set identified by activation patching with contrastive few-shot prompts. For sentence-level outputs, the paper introduces a KL-divergence criterion to pick the target token position where clean and corrupted prompts diverge most, then patches activations at that position. The identified heads are used to build mean-activation steering vectors (scaled by an amplification factor) that are added to the residual stream, turning an instruction-free prompt into a translation prompt.

Load-bearing premise

The load-bearing premise is that the two corrupted prompt families isolate exactly the two intended subtasks — replacing demonstration targets with another language isolates 'which language to output', and replacing them with random same-language sentences isolates 'preserve meaning'; if those corruptions also shift lexical, syntactic, or difficulty properties, the identified head sets may be conflated.

What would settle it

A decisive check: construct a corruption that swaps every content word of the demonstration targets for same-language synonyms (preserving syntax, length, and target language) and see whether activation patching still isolates the same 'translation heads'; if the head set changes drastically, the claimed equivalence subtask is an artifact of the particular corruption. Alternatively, ablate the top 'language heads' on a direction into English, where the paper reports language heads matter less — if target-language accuracy stays at random-ablation levels but BLEU still collapses, the functional

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Translation is mechanistically localized: interventions on ~1% of attention heads are sufficient to elicit or disrupt sentence-level MT, so model editing and steering are viable at this scale.
  • Language identity and meaning equivalence are separately encoded, with mostly disjoint head sets (Jaccard index ≈0.13), refining the picture of how multilingual LLMs organize translation.
  • Instruction-free steering with language and equivalence vectors matches instructed zero-shot prompting (e.g., 33.3 vs 32.6 BLEU for Gemma-3-12B English→X), so prompts may be replaceable by targeted activation control.
  • Ablating language heads collapses target-language accuracy when translating out of English, while ablating translation heads degrades meaning; the asymmetry reflects English-centric pretraining.
  • Equivalence vectors transfer across translation directions with minimal quality loss, suggesting the meaning-preservation component is largely direction-agnostic.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable prediction the paper does not make: if the same decomposition holds in other cross-lingual tasks (paraphrase, summarization, code-switching), 'language' and 'content' may be generally separable at the head level.
  • The paper's own evidence shows first-token selection identifies similar heads to their KL-based method for larger models; a cheaper identification protocol might be possible, but would need validation on smaller models.
  • The contrastive corruptions likely change more than the intended subtask (e.g., lexical statistics, syntactic templates), so the cleanest test of the decomposition would use synthetic corruptions that vary only the target language at constant token distribution; the paper's causal steering partially addresses but does not fully rule out this confound.
  • Because language vectors decode to language-associated tokens while equivalence vectors decode to overlapping high-vocabulary tokens, one could attempt to 'retarget' a model into a language it generates poorly by swapping language vectors alone — though the paper's low-resource results suggest this may hit a generation-quality ceiling.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a mechanistic decomposition of sentence-level machine translation in LLMs into two subtasks: target language identification and sentence equivalence. Using activation patching on two corrupted few-shot prompt families — wrong/mixed target languages, and random same-language target sentences — the authors identify roughly 1% of attention heads as specialized per subtask across Gemma-3, Qwen-3, and Llama-3 models and 20 translation directions. Steering these heads with mean-activation vectors under an instruction-free zero-shot prompt yields translation quality close to instructed zero-shot prompting (e.g., 33.3 vs 32.6 BLEU for Gemma-3-12B English→X), while ablating them produces function-specific failures (language switching vs. loss of meaning). Equivalence vectors transfer across directions. The paper also analyzes the effect of shot count, amplification factor, token position, and compares the identified heads with those from prior word-level MT studies.

Significance. If the causal claims hold, the findings are significant for mechanistic interpretability and MT: they provide evidence that a sparse set of attention heads can control an entire sentence-level NLP task, that the heads split into largely disjoint functional groups, and that steering only these heads suffices to elicit translation without instructions. The breadth — three model families, multiple scales, 20 directions, several automatic metrics — and released code are strengths. The causal validation via steering and ablation is the most compelling part and gives the paper real value. However, the semantic interpretation of the two head classes rests on an unexamined assumption that the two corruptions isolate exactly the intended subtasks; the paper would be materially strengthened by control experiments and an explicit limitations discussion.

major comments (3)
  1. [§3.2, sentence-equivalence corruption c_MT] Replacing each demonstration target with a random same-language sentence removes source–target meaning correspondence, but it also changes target-side lexical/syntactic statistics, discourse coherence, and the ease of inferring a coherent input–output mapping. Activation patching may therefore identify heads that encode 'this prompt is/is not a coherent translation task' rather than 'preserve the source's meaning.' This confound directly underdetermines the central claim that translation heads encode sentence equivalence. The paper provides no control (e.g., permuting Q/A pairs while using genuine translations, or using translations of different source sentences) and no limitations discussion. Please add such controls or explicitly temper the semantic interpretation.
  2. [§3.2, language-identification corruption c_lang] The text says each demonstration target is replaced by a correct translation in a language selected from the set L, while the Figure 1 caption describes the corruption as 'mixing multiple target languages.' Mixing languages removes cross-demonstration target-language consistency in addition to removing the intended target language, conflating target-language identity with mapping consistency. If the goal is to isolate target-language identification, a cleaner corruption would use a single wrong target language for all demonstrations, or at least report both variants. The causal steering results soften this concern but do not fully resolve the identification confound.
  3. [§5, Figs. 3–4] The headline 'comparable' claims rest on small BLEU differences (e.g., 33.3 vs 32.6 for Gemma-3-12B English→X) on a single devtest split with greedy decoding. No confidence intervals, bootstrap resampling, or significance tests are reported. With roughly 1,012 sentences, a 0.7 BLEU difference may be within noise. Please provide uncertainty estimates and, if possible, per-direction paired comparisons. This matters because the steering-sufficiency claim is one of the paper's main conclusions.
minor comments (4)
  1. [General] Typos and small phrasing issues: 'a instruction-free' in the Figure 5 caption; 'ANALISYS' in Appendix B.4.1; 'minimal performance loss performance' in Section 7. The abstract says 'three families' while the setup lists four model families (Gemma-3, Llama-3.2, Llama-2-7B, Qwen-3); clarify the grouping.
  2. [§5] The text says '5 to 10 heads' and gives '5 out of 1,024' as an example; specify the model and total head count corresponding to each number. The heatmaps in Figures 3–4 are dense; increasing font size and labeling the axes more explicitly would help.
  3. [§6 / References] The citation 'Anonymous (2025)' appears in the related work and in the head-classification comparison. In a final version this should be de-anonymized or clearly marked as a self-citation, depending on the submission venue's anonymity policy.
  4. [§3.1] The notation DKL is used without specifying the base (natural log vs log2); the KL values are only used comparatively, but stating the base would be cleaner.

Circularity Check

0 steps flagged

No circular reduction found: head identification and steering are independently evaluated on held-out devtest; §3.2's corruption confound is a validity concern, not a constructional circularity.

full rationale

The derivation chain is self-contained where circularity could arise. Heads are identified in §3.1–3.2 via activation patching between clean and corrupted few-shot prompts, with the two corruptions defined to target language choice and meaning correspondence. No parameter is fitted on the devtest set: §4 states the dev set is used to identify heads and build steering vectors, while the devtest set is used for evaluation, and the main results fix the amplification factor at α=1, with a sensitivity study in §6 rather than per-direction tuning. The steering vectors are simple averaged clean activations multiplied by the output projection; they are not optimized against the target metric. The central claims — steering 1% of heads approaches instructed zero-shot on BLEU/MetricX and ablations cause function-specific failures — are evaluated against external baselines and random-head controls, so the predictions do not reduce to the quantities used to define the heads. The only potentially self-referential citations are the anonymous companion paper and co-authored in-context-learning/low-resource MT references (Zebaze et al. 2025a,b), but these are used as related work and context, not as the justification for the decomposition or the localization result. There is a genuine validity concern, identified by the skeptical read, that the c_MT corruption changes lexical/syntactic statistics and task coherence along with meaning, so the identified 'translation heads' may encode task coherence rather than sentence equivalence; however, that is an empirical underdetermination, not a circular reduction — no equation makes sentence equivalence identical to the corruption, and the steering/ablation failure modes provide independent, non-tautological evidence. The manuscript's lack of a limitations section noting this confound is a transparency issue but does not constitute circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical or architectural entities. Its free parameters are hyperparameters of the steering/identification procedure (α, head fraction, k). The main axiomatic load is the two-subtask decomposition and the causal-transparency assumption of activation patching.

free parameters (3)
  • amplification factor α = 1 (default; sensitivity explored from -2 to 20)
    Scales the steering vector added to the residual stream. Main results use α=1; the ablation shows best performance around 0.5–3, but no per-direction fitting is reported.
  • head fraction for steering/ablation = 1% of language heads and 1% of translation heads
    The headline '1%' is a hand-chosen operating point; experiments also vary 2–5%. It is not tuned per direction, but it is not a fully parameter-free choice.
  • number of in-context demonstrations k = 5 (default)
    Head identification and steering use k=5 demonstrations; the paper studies k=0,1,20 and shows consistency for k>0.
axioms (4)
  • domain assumption Activation patching at attention-head granularity measures a component's causal contribution to the next-token distribution.
    Invoked throughout §3.1; the entire head-identification method rests on this standard MI assumption.
  • ad hoc to paper The two corrupted prompt families isolate exactly target-language identification and sentence equivalence, respectively.
    Introduced in §3.2 as the decomposition of MT; the paper's central claim of distinct head sets depends on this isolation being clean.
  • domain assumption The FLORES-200 reference translations used for teacher forcing and evaluation are adequate gold translations.
    Teacher forcing conditions on ground-truth target tokens (§3.1), and BLEU/MetricX evaluation presupposes reference quality.
  • domain assumption BLEU, MetricX-24, chrF++, and XCOMET scores are meaningful proxies for translation quality in the comparisons made.
    Used for all steering and ablation comparisons; standard practice, but metric-specific biases are not separately analyzed.

pith-pipeline@v1.3.0-alltime-deepseek · 102363 in / 14011 out tokens · 159200 ms · 2026-08-03T04:30:54.128973+00:00 · methodology

0 comments
read the original abstract

Mechanistic Interpretability (MI) seeks to explain how neural networks implement their capabilities, but the scale of Large Language Models (LLMs) has limited prior MI work in Machine Translation (MT) to word-level analyses. We study sentence-level MT from a mechanistic perspective by analyzing attention heads to understand how LLMs internally encode and distribute translation functions. We decompose MT into two subtasks: producing text in the target language (i.e. target language identification) and preserving the input sentence's meaning (i.e. sentence equivalence). Across three families of open-source models and 20 translation directions, we find that distinct, sparse sets of attention heads specialize in each subtask. Based on this insight, we construct subtask-specific steering vectors and show that modifying just 1% of the relevant heads enables instruction-free MT performance comparable to instruction-based prompting, while ablating these heads selectively disrupts their corresponding translation functions.

Figures

Figures reproduced from arXiv: 2602.04613 by Armel Zebaze, Beno\^it Sagot, Djam\'e Seddah, Rachel Bawden, Th\'eo Lasnier.

Figure 1
Figure 1. Figure 1: We consider two few-shot MT prompts: a clean prompt and a corrupted prompt (obtained in this case by mixing multiple target languages) and both ending with the same query. For each prompt, we apply teacher forcing and compute the token-level output distributions for the ground-truth query. At each target position, we measure the KL divergence between the distributions conditioned on the clean and corrupted… view at source ↗
Figure 2
Figure 2. Figure 2: Average log probability deltas per layer and attention head in GEMMA-3-12B-PT under language (left) and translation (right) corruption (averaged across the 20 language directions) tokens with α = 1 as an amplification factor. 5. Results Identified heads [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: MT performance of GEMMA-3-12B-PT in an instruction￾free zero-shot prompt when steering n% translation heads (x-axis) and m% language heads (y-axis) with the average outputs of these heads under few-shot examples. (a) and (b) report average BLEU scores for all translation pairs with English as source and target language, respectively. (c) and (d) report MetricX-24 scores in the same setup. Above each figure… view at source ↗
Figure 5
Figure 5. Figure 5: MT performance of GEMMA-3-12B-PT in a instruction￾free zero-shot prompt when steering 1% of language heads and translation heads by varying the language pairs used to create the language agnostic sentence equivalence (x-axis) and the target language identity (y-axis) steering vectors. We report average BLEU (left) and MetricX-24 (right) scores with the target language identity as reference. are ablated, bu… view at source ↗
Figure 4
Figure 4. Figure 4: MT performance of GEMMA-3-12B-PT when ablating n% translation heads (x-axis) and m% of language heads (y-axis) in an instructed zero-shot setup. (a) and (b) report average BLEU scores for all translation pairs with English as source and target language, respectively. (c) and (d) report MetricX-24 scores and (e) and (f) report the target language accuracy in the same setup. Results are compared against abla… view at source ↗
Figure 6
Figure 6. Figure 6: Log probability delta per layer and attention head in GEMMA-3-12B-PT under language (left) and translation (right) corruption for English→French, for the 1-shot scenario. tokens in the target language of the corresponding direction (Appendix C.3). Notably, for GEMMA-3-4B-PT, decoding language vectors produces tokens closely associated with the target language (e.g., French, Québec, and France for French; T… view at source ↗
Figure 8
Figure 8. Figure 8: Log probability delta per layer and attention head in GEMMA-3-4B-PT under language corruption for English→French. We report activation patching results (a-d) for fixed positions {0, 2}, random and KL-based (ours), respectively. H0 H1 H2 H3 H4 H5 H6 H7 L0 L3 L6 L9 L12 L15 L18 L21 L24 L27 L30 L33 (a) H0 H1 H2 H3 H4 H5 H6 H7 L0 L3 L6 L9 L12 L15 L18 L21 L24 L27 L30 L33 (b) H0 H1 H2 H3 H4 H5 H6 H7 L0 L3 L6 L9 L… view at source ↗
Figure 9
Figure 9. Figure 9: Log probability delta per layer and attention head in GEMMA-3-4B-PT under translation corruption for English→French. We report activation patching results (a-d) for fixed positions 0, 2, random and KL-based (ours), respectively. 270M 1B 4B 12B 0 10 20 30 40 50 Index 0 Index 2 Index 4 Random KL (Ours) [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Effect of the studied token index on MT performance for English→French when steering with 5% of the language and translation heads. We report BLEU scores under an instruction￾free zero-shot setup across four Gemma-3 models. Each bar cor￾responds to a different token selection strategy: fixed positions (i ∈ {0, 2, 4}), random selection, and our KL-based method. lation heads ( [PITH_FULL_IMAGE:figures/full… view at source ↗
Figure 11
Figure 11. Figure 11: Log probability delta per layer and attention head in GEMMA-3-270M under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H0 H1 H2 H3 L0 L2 L4 L6 L8 L10 L12 L14 L16 (a) H0 H1 H2 H3 L0 L2 L4 L6 L8 L10 L12 L14 L16 (b) H0 H1 H2 H3 L0 L2 … view at source ↗
Figure 12
Figure 12. Figure 12: Log probability delta per layer and attention head in GEMMA-3-270M under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H0 H1 H2 H3 L0 L2 L4 L6 L8 L10 L12 L14 L16 L18 L20 L22 L24 (a) H0 H1 H2 H3 L0 L2 L4 L6 L8 L10 L12 L14 L16 L18 L20 … view at source ↗
Figure 13
Figure 13. Figure 13: Log probability delta per layer and attention head in GEMMA-3-1B-PT under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Log probability delta per layer and attention head in GEMMA-3-1B-PT under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H0 H1 H2 H3 H4 H5 H6 H7 L0 L3 L6 L9 L12 L15 L18 L21 L24 L27 L30 L33 (a) H0 H1 H2 H3 H4 H5 H6 H7 L0 L3 L6 L9 L12 L… view at source ↗
Figure 15
Figure 15. Figure 15: Log probability delta per layer and attention head in GEMMA-3-4B-PT under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H0 H1 H2 H3 H4 H5 H6 H7 L0 L3 L6 L9 L12 L15 L18 L21 L24 L27 L30 L33 (a) H0 H1 H2 H3 H4 H5 H6 H7 L0 L3 L6 L9 L12… view at source ↗
Figure 16
Figure 16. Figure 16: Log probability delta per layer and attention head in GEMMA-3-4B-PT under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic, Russian and Swahili respectively. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Activation Patching result under translation corruption for GEMMA-3-12B-PT 18 [PITH_FULL_IMAGE:figures/full_fig_p018_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Activation Patching result under language corruption for GEMMA-3-12B-PT 19 [PITH_FULL_IMAGE:figures/full_fig_p019_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Log probability delta per layer and attention head in GEMMA-3-27B-PT under translation corruption. (a) represent the mean delta across the translation direction English→French and French→English. While (b) and (c) represent the delta for the translation pair English to French and French to English, respectively. H 0 H 3 H 6 H 9 H 1 2 H 1 5 H 1 8 H 2 1 H 2 4 H 2 7 H 3 0 L0 L7 L14 L21 L28 L35 L42 L49 L56 (a… view at source ↗
Figure 20
Figure 20. Figure 20: Log probability delta per layer and attention head in GEMMA-3-27B-PT under language corruption. (a) represent the mean delta across the translation direction English→French and French→English. While (b) and (c) represent the delta for the translation pair English to French and French to English, respectively. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Log probability delta per layer and attention head in QWEN3-0.6B-BASE under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 L0 L4 L8 L12 L16 L20 L24 (a) H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 L0 L… view at source ↗
Figure 22
Figure 22. Figure 22: Log probability delta per layer and attention head in QWEN3-0.6B-BASE under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 L0 L4 L8 L12 L16 L20 L24 (a) H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 L0 L4 … view at source ↗
Figure 23
Figure 23. Figure 23: Log probability delta per layer and attention head in QWEN3-1.7B-BASE under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Log probability delta per layer and attention head in QWEN3-1.7B-BASE under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H 0 H 3 H 6 H 9 H 1 2 H 1 5 H 1 8 H 2 1 H 2 4 H 2 7 H 3 0 L0 L7 L14 L21 L28 L35 (a) H 0 H 3 H 6 H 9 H 1 2 H 1 5… view at source ↗
Figure 25
Figure 25. Figure 25: Log probability delta per layer and attention head in QWEN3-4B-BASE under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d),(e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H 0 H 3 H 6 H 9 H 1 2 H 1 5 H 1 8 H 2 1 H 2 4 H 2 7 H 3 0 L0 L7 L14 L21 L28 L35 (a) H 0 H 3 H 6 H 9 H 1 2 H 1 5 … view at source ↗
Figure 26
Figure 26. Figure 26: Log probability delta per layer and attention head in QWEN3-4B-BASE under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. B.1.3. LLAMA-3.2 We report activation patching results for the Llama-3.2 model family across two scales. For each… view at source ↗
Figure 27
Figure 27. Figure 27: Log probability delta per layer and attention head in LLAMA-3.2-1B under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H 0 H 3 H 6 H 9 H 1 2 H 1 5 H 1 8 H 2 1 H 2 4 H 2 7 H 3 0 L0 L7 L14 (a) H 0 H 3 H 6 H 9 H 1 2 H 1 5 H 1 8 H 2 1 … view at source ↗
Figure 28
Figure 28. Figure 28: Log probability delta per layer and attention head in LLAMA-3.2-1B under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 H 1 6 H 1 8 H 2 0 H 2 2 L0 L5 L10 L15 L20 L25 (a) H 0 H 2 H 4 H 6 H 8 H 1 0 … view at source ↗
Figure 29
Figure 29. Figure 29: Log probability delta per layer and attention head in LLAMA-3.2-3B under translation corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represent the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 H 1 6 H 1 8 H 2 0 H 2 2 L0 L5 L10 L15 L20 L25 (a) H 0 H 2 H 4 H 6 H 8 H 1 … view at source ↗
Figure 30
Figure 30. Figure 30: Log probability delta per layer and attention head in LLAMA-3.2-3B under language corruption. (a) represent the mean delta across the 20 translations directions, while (b), (c), (d), (e) represents the delta for the translation pair English to French, Chinese, Arabic and Swahili respectively. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in GEMMA-3-270M. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_31.png] view at source ↗
Figure 32
Figure 32. Figure 32: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in GEMMA-3-1B. .13 .12 .11 .13 .11 .12 .13 .10 .09 .05 .11 .12 .13 .10 .13 .12 .12 .12 .06 .11 .06 .07 .05 .07 .06 .05 .06 .04 .05 .04 .05 .06 .06 .05 .06 .06 .06 .05 .04 .05 .07 .07 .06 .07 .07 .06 .07 .05 .05 .05 .06 .06 .06 .06 .07 .06 .06 .06 .05 .05 .07 .06 .… view at source ↗
Figure 33
Figure 33. Figure 33: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in GEMMA-3-4B-PT. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_33.png] view at source ↗
Figure 34
Figure 34. Figure 34: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in GEMMA-3-12B-PT. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_34.png] view at source ↗
Figure 35
Figure 35. Figure 35: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in QWEN3-0.6B-BASE. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_35.png] view at source ↗
Figure 36
Figure 36. Figure 36: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in QWEN3-1.7B-BASE. .06 .05 .05 .03 .04 .05 .04 .05 .04 .03 .04 .05 .06 .05 .06 .05 .05 .05 .03 .05 .04 .04 .04 .04 .03 .04 .03 .04 .03 .03 .04 .04 .05 .05 .04 .04 .04 .04 .03 .04 .04 .04 .04 .04 .03 .04 .04 .04 .04 .04 .04 .04 .05 .05 .05 .04 .04 .04 .04 .05 .04 … view at source ↗
Figure 37
Figure 37. Figure 37: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in QWEN3-4B-BASE. We report 28 [PITH_FULL_IMAGE:figures/full_fig_p028_37.png] view at source ↗
Figure 38
Figure 38. Figure 38: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis) identified through activation patching in LLAMA-3.2-1B. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_38.png] view at source ↗
Figure 39
Figure 39. Figure 39: Jaccard index between the top 5% of language heads (y-axis) and the top 5% of translation heads (x-axis)identified through activation patching in LLAMA-3.2-3B. B.3. Steering Following the experiments in Section 5, we provide detailed steering results across models and translation pairs. We first present a qualitative analysis of failure modes when steering with only language heads or only translation head… view at source ↗
Figure 40
Figure 40. Figure 40: Steering-induced MT performance for GEMMA-3-270M under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g, i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report results … view at source ↗
Figure 41
Figure 41. Figure 41: Steering-induced MT performance for GEMMA-3-1B-PT under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g, i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report results… view at source ↗
Figure 42
Figure 42. Figure 42: Steering-induced MT performance for GEMMA-3-4B-PT under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g, i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report results… view at source ↗
Figure 43
Figure 43. Figure 43: Steering-induced MT performance (BLEU) for GEMMA-3-12B-PT under an instruction-free zero-shot setup across all 20 translation directions. Each heatmap corresponds to a specific translation pair, with the x-axis and y-axis denoting the proportion of translation heads and language heads steered, respectively. The score reported next to each translation pair indicates the performance obtained in an instructe… view at source ↗
Figure 44
Figure 44. Figure 44: Steering-induced MT performance (MetricX-24) for GEMMA-3-12B-PT under an instruction-free zero-shot setup across all 20 translation directions. Each heatmap corresponds to a specific translation pair, with the x-axis and y-axis denoting the proportion of translation heads and language heads steered, respectively. The score reported next to each translation pair indicates the performance obtained in an ins… view at source ↗
Figure 45
Figure 45. Figure 45: Steering-induced MT performance (MetricX-24 QE) for GEMMA-3-12B-PT under an instruction-free zero-shot setup across all 20 translation directions. Each heatmap corresponds to a specific translation pair, with the x-axis and y-axis denoting the proportion of translation heads and language heads steered, respectively. The score reported next to each translation pair indicates the performance obtained in an … view at source ↗
Figure 46
Figure 46. Figure 46: Steering-induced MT performance (chrF++) for GEMMA-3-12B-PT under an instruction-free zero-shot setup across all 20 translation directions. Each heatmap corresponds to a specific translation pair, with the x-axis and y-axis denoting the proportion of translation heads and language heads steered, respectively. The score reported next to each translation pair indicates the performance obtained in an instruc… view at source ↗
Figure 47
Figure 47. Figure 47: Steering-induced MT performance (COMET) for GEMMA-3-12B-PT under an instruction-free zero-shot setup across all 20 translation directions. Each heatmap corresponds to a specific translation pair, with the x-axis and y-axis denoting the proportion of translation heads and language heads steered, respectively. The score reported next to each translation pair indicates the performance obtained in an instruct… view at source ↗
Figure 48
Figure 48. Figure 48: Steering-induced MT performance for QWEN3-0.6B-BASE under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g, i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report resul… view at source ↗
Figure 49
Figure 49. Figure 49: Steering-induced MT performance for QWEN3-1.7B-BASE under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g, i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report resul… view at source ↗
Figure 50
Figure 50. Figure 50: Steering-induced MT performance for QWEN3-4B-BASE under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g, i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report results… view at source ↗
Figure 51
Figure 51. Figure 51: Steering-induced MT performance for LLAMA-3.2-1B under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report results f… view at source ↗
Figure 52
Figure 52. Figure 52: Steering-induced MT performance for LLAMA-3.2-3B under an instruction-free zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h) and XCOMET (i, j) when steering n% of translation heads (x-axis) and m% of language heads (y-axis). Subfigures (a, c, e, g, i) report results for translation pairs with English as the source language, while (b, d, f, h, j) report results … view at source ↗
Figure 53
Figure 53. Figure 53: MT performance under head ablation for GEMMA-3-270M using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source language… view at source ↗
Figure 54
Figure 54. Figure 54: MT performance under head ablation for GEMMA-3-1B-PT using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source languag… view at source ↗
Figure 55
Figure 55. Figure 55: MT performance under head ablation for GEMMA-3-4B-PT using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source languag… view at source ↗
Figure 56
Figure 56. Figure 56: MT performance under head ablation (BLEU) for GEMMA-3-12B-PT across all 20 translation directions using an instructed zero-shot setup. Each heatmap shows performance when ablating n% of Translation Heads (x-axis) and m% of Language Heads (y-axis) for a specific translation pair. Results are compared against ablating j% of randomly selected heads. 48 [PITH_FULL_IMAGE:figures/full_fig_p048_56.png] view at source ↗
Figure 57
Figure 57. Figure 57: MT performance under head ablation (MetricX-24) for GEMMA-3-12B-PT across all 20 translation directions using an instructed zero-shot setup. Each heatmap shows performance when ablating n% of Translation Heads (x-axis) and m% of Language Heads (y-axis) for a specific translation pair. Results are compared against ablating j% of randomly selected heads. 49 [PITH_FULL_IMAGE:figures/full_fig_p049_57.png] view at source ↗
Figure 58
Figure 58. Figure 58: MT performance under head ablation (MetricX-24 QE) for GEMMA-3-12B-PT across all 20 translation directions using an instructed zero-shot setup. Each heatmap shows performance when ablating n% of Translation Heads (x-axis) and m% of Language Heads (y-axis) for a specific translation pair. Results are compared against ablating j% of randomly selected heads. 50 [PITH_FULL_IMAGE:figures/full_fig_p050_58.png] view at source ↗
Figure 59
Figure 59. Figure 59: MT performance under head ablation (CHRF++) for GEMMA-3-12B-PT across all 20 translation directions using an instructed zero-shot setup. Each heatmap shows performance when ablating n% of Translation Heads (x-axis) and m% of Language Heads (y-axis) for a specific translation pair. Results are compared against ablating j% of randomly selected heads. 51 [PITH_FULL_IMAGE:figures/full_fig_p051_59.png] view at source ↗
Figure 60
Figure 60. Figure 60: MT performance under head ablation (COMET) for GEMMA-3-12B-PT across all 20 translation directions using an instructed zero-shot setup. Each heatmap shows performance when ablating n% of Translation Heads (x-axis) and m% of Language Heads (y-axis) for a specific translation pair. Results are compared against ablating j% of randomly selected heads. 52 [PITH_FULL_IMAGE:figures/full_fig_p052_60.png] view at source ↗
Figure 61
Figure 61. Figure 61: Target language accuracy under head ablation for GEMMA-3-12B-PT across all 20 translation directions using an instructed zero-shot setup. Each heatmap shows performance when ablating n% of Translation Heads (x-axis) and m% of Language Heads (y-axis) for a specific translation pair. Results are compared against ablating j% of randomly selected heads. 53 [PITH_FULL_IMAGE:figures/full_fig_p053_61.png] view at source ↗
Figure 62
Figure 62. Figure 62: MT performance under head ablation for QWEN3-0.6B-BASE using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source langu… view at source ↗
Figure 63
Figure 63. Figure 63: MT performance under head ablation for QWEN3-1.7B-BASE using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source langu… view at source ↗
Figure 64
Figure 64. Figure 64: MT performance under head ablation for QWEN3-4B-BASE using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source languag… view at source ↗
Figure 65
Figure 65. Figure 65: MT performance under head ablation for LLAMA-3.2-1B using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source language… view at source ↗
Figure 66
Figure 66. Figure 66: MT performance under head ablation for LLAMA-3.2-3B using an instructed zero-shot setup. We report BLEU (a, b), MetricX-24 (c, d), MetricX-24 QE (e, f), chrF++ (g, h), XCOMET (i, j), and target language accuracy (k, l) when ablating and n% of translation heads (x-axis) and m% of language heads (y-axis) . Subfigures (a, c, e, g, i, k) report results for translation pairs with English as the source language… view at source ↗
Figure 67
Figure 67. Figure 67: Log probability delta per layer and attention head in GEMMA-3-12B-PT under language corruption for the English to French translation pair. We report activation patching results (a-c) for 0, 1, 20 shots, respectively. H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 L0 L4 L8 L12 L16 L20 L24 L28 L32 L36 L40 L44 (a) H 0 H 2 H 4 H 6 H 8 H 1 0 H 1 2 H 1 4 L0 L4 L8 L12 L16 L20 L24 L28 L32 L36 L40 L44 (b) H 0 H 2 H 4 H 6 H… view at source ↗
Figure 68
Figure 68. Figure 68: Log probability delta per layer and attention head in GEMMA-3-12B-PT under translation corruption for the English to French translation pair. We report activation patching results (a-c) for 0, 1, 20 shots, respectively. 58 [PITH_FULL_IMAGE:figures/full_fig_p058_68.png] view at source ↗
Figure 69
Figure 69. Figure 69: Effect of the number of few-shot examples on steering-induced MT performance. We report BLEU scores under an instruction￾free zero-shot setup for GEMMA-3-12B-PT when steering 1% of Language and translation heads for 0, 1, 2, 3, 5, 10, 20 and 50 shots. Results are compared against generation in a n-shot setup. C.2. Transferability of the equivalence vectors Do we know if the representation of the translati… view at source ↗
Figure 70
Figure 70. Figure 70: Cosine similarity of mean head outputs across translation directions for GEMMA-3-12B-PT, computed in a few-shot setup. For each translation direction, we extract the average activation output of the top-1 Language Head (a, b) and top-1 Translation Head (c, d) identified through activation patching, then compute pairwise cosine similarities. The left and right columns correspond to translation pairs with E… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

    cs.CL 2026-04 unverdicted novelty 7.0

    Translation function vectors extracted from English to one target language improve correct token ranking for translations to multiple other unseen target languages in decoder-only multilingual LLMs.

  2. Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

    cs.CL 2026-04 conditional novelty 5.0

    Translation function vectors extracted from one language direction transfer to unseen target languages, indicating a language-agnostic translation signal in multilingual LLMs.

Reference graph

Works this paper leans on

12 extracted references · 2 linked inside Pith · cited by 1 Pith paper

  1. [7]

    Wang, W., Li, Z., Lian, D., Ma, C., Song, L., and Wei, Y

    URL https://openreview.net/forum? id=NpsVSN6o4ul. Wang, W., Li, Z., Lian, D., Ma, C., Song, L., and Wei, Y . Mitigating the language mismatch and repetition issues in LLM-based machine translation via model editing. In Al-Onaizan, Y ., Bansal, M., and Chen, Y .-N. (eds.),Pro- ceedings of the 2024 Conference on Empirical Methods in 13 Disentangling meaning...

    2024

  2. [68]

    findings-naacl.68/

    URL https://aclanthology.org/2025. findings-naacl.68/. Zhang, B., Haddow, B., and Birch, A. Prompting large language model for machine translation: a case study. InProceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023. Zhang, H., Chen, K., Bai, X., Li, X., Xiang, Y ., and Zhang, M. Exploring the translation mechanis...

    2025

  3. [176]

    We will endeavour to cut carbon dioxide emis- sions per unit of GDP by a notable margin by 2020 from the 2005 level,

    URL https://aclanthology.org/2024. findings-naacl.176/. Zhu, W., Liu, H., Dong, Q., Xu, J., Huang, S., Kong, L., Chen, J., and Li, L. Multilingual machine translation with large language models: Empirical results and analysis. In Findings of the association for computational linguistics: NAACL 2024, pp. 2765–2781, 2024b. 14 Disentangling meaning from lang...

    2024

  4. [363]

    emnlp-main.363/

    URL https://aclanthology.org/2020. emnlp-main.363/. Lee, D., Lee, S. C., Yang, C., Baek, Y ., and Choo, J. Explor- ing in-context example generation for machine translation. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.),Findings of the Association for Computational Linguistics: ACL 2025, pp. 26554–26568, Vienna, Aus- tria, July 2025. As...

    doi:10.18653/v1/2025 2020

  5. [624]

    findings-emnlp.624/

    URL https://aclanthology.org/2023. findings-emnlp.624/. Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V ., Gabr, M., Matsushita, H., Kim, Y . J., Afify, M., and Awadalla, H. H. How good are gpt models at machine translation? a comprehensive evaluation, 2023. URL https:// arxiv.org/abs/2302.09210. Jiao, W., Wang, W., tse Huang, J., Wang, X., Shi, S., and ...

    Pith/arXiv arXiv 2023

  6. [879]

    emnlp-main.879/

    URL https://aclanthology.org/2024. emnlp-main.879/. Wendler, C., Veselovsky, V ., Monea, G., and West, R. Do llamas work in English? on the latent language of multi- lingual transformers. In Ku, L.-W., Martins, A., and Sriku- mar, V . (eds.),Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp...

    Pith/arXiv arXiv 2024

  7. [1216]

    findings-emnlp.1216/

    URL https://aclanthology.org/2025. findings-emnlp.1216/. Zebaze, A. R., Sagot, B., and Bawden, R. In-context example selection via similarity search improves low- resource machine translation. In Chiruzzo, L., Ritter, A., and Wang, L. (eds.),Findings of the Association for Computational Linguistics: NAACL 2025, pp. 1222– 1252, Albuquerque, New Mexico, Apr...

    doi:10.18653/v1/2025.findings-naacl 2025

  8. [2020]

    Lauscher, A., Ravishankar, V ., Vuli´c, I., and Glavaš, G

    URL https://openreview.net/forum? id=HJeT3yrtDr. Lauscher, A., Ravishankar, V ., Vuli´c, I., and Glavaš, G. From zero to hero: On the limitations of zero-shot language transfer with multilingual Transformers. In Webber, B., Cohn, T., He, Y ., and Liu, Y . (eds.),Pro- ceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNL...

    doi:10.18653/v1/2020.emnlp-main 2020

  9. [2022]

    Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J

    https://transformer-circuits.pub/2022/in-context- learning-and-induction-heads/index.html. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine translation. In Isabelle, P., Charniak, E., and Lin, D. (eds.),Proceed- ings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–31...

    arXiv 2022

  10. [2023]

    URL https://proceedings.mlr.press/ v202/von-oswald23a.html. Wang, K. R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: a circuit for indi- rect object identification in GPT-2 small. InThe Eleventh International Conference on Learning Representations,

  11. [2024]

    Tillmann, C., V ogel, S., Ney, H., and Zubiaga, A

    URL https://openreview.net/forum? id=tbVWug9f2h. Tillmann, C., V ogel, S., Ney, H., and Zubiaga, A. A DP-based search using monotone alignments in statis- tical translation. In35th Annual Meeting of the As- sociation for Computational Linguistics and 8th Con- ference of the European Chapter of the Association for Computational Linguistics, pp. 289–296, Ma...

    arXiv 1997

  12. [2025]

    Zhao, Y ., Zhang, W., Chen, G., Kawaguchi, K., and Bing, L

    URL https://openreview.net/forum? id=3QjESmXftM. Zhao, Y ., Zhang, W., Chen, G., Kawaguchi, K., and Bing, L. How do large language models handle multilingualism? Advances in Neural Information Processing Systems, 37: 15296–15319, 2024. Zhu, W., Liu, H., Dong, Q., Xu, J., Huang, S., Kong, L., Chen, J., and Li, L. Multilingual machine trans- lation with lar...

    2024