Pith. sign in

REVIEW 29 cited by

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.04675 v4 pith:AGYF4FTE submitted 2023-04-10 cs.CL

Wenhao Zhu , Hongyi Liu , Qingxiu Dong , Jingjing Xu , Shujian Huang , Lingpeng Kong , Jiajun Chen , Lei Li This is my paper · ORCID
classification cs.CL
keywords translationllmsexemplarslanguagelanguageslargeanalysisempirical
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT). In this paper, we systematically investigate the advantages and challenges of LLMs for MMT by answering two questions: 1) How well do LLMs perform in translating massive languages? 2) Which factors affect LLMs' performance in translation? We thoroughly evaluate eight popular LLMs, including ChatGPT and GPT-4. Our empirical results show that translation capabilities of LLMs are continually involving. GPT-4 has beat the strong supervised baseline NLLB in 40.91% of translation directions but still faces a large gap towards the commercial translation system like Google Translate, especially on low-resource languages. Through further analysis, we discover that LLMs exhibit new working patterns when used for MMT. First, LLM can acquire translation ability in a resource-efficient way and generate moderate translation even on zero-resource languages. Second, instruction semantics can surprisingly be ignored when given in-context exemplars. Third, cross-lingual exemplars can provide better task guidance for low-resource translation than exemplars in the same language pairs. Code will be released at: https://github.com/NJUNLP/MMT-LLM.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 29 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMs

    cs.CR 2025-05 conditional novelty 8.0 of 10

    A sequential vocabulary-search attack decodes original prompts from unpermuted and permuted LLM hidden states, compromising PermLLM, STIP, and Centaur.

  2. Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

    cs.AI 2024-12 conditional novelty 7.0 of 10

    A relevance-guided, fully automatic attack using open-source tools extracts most of a RAG system's private knowledge base without any access to the target's internals.

  3. Improving Language Transfer Capability of Decoder-only Architecture in Multilingual Neural Machine Translation

    cs.CL 2024-12 conditional novelty 7.0 of 10

    A two-stage decoder-only architecture with instruction-level contrastive learning improves zero-shot multilingual translation and closes most of the gap to encoder-decoder models.

  4. Mitigating Context Interference for Reliable and Efficient Search Agents

    cs.CL 2026-08 conditional novelty 6.0 of 10

    A distilled context refiner plus refinement during RL training improves reliability and efficiency of multi-turn LLM search agents, with the latest retrieved documents identified as the main source of context interference.

  5. LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Label-aware diagnostic reflection plus two-stage outcome GRPO improves same-backbone IE F1 over SFT, with larger gains under relation-extraction domain shift.

  6. Dutch CrowS-Pairs: Adapting a Challenge Dataset for Measuring Social Biases in Language Models for Dutch

    cs.CL 2025-07 conditional novelty 6.0 of 10

    The paper presents a Dutch adaptation of the CrowS-Pairs bias benchmark and reports bias scores for seven masked and two autoregressive language models across nine demographic categories.

  7. Cascade: Token-Sharded Private LLM Inference

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Cascade performs LLM inference by sharding the token sequence across non-colluding nodes, claiming resistance to vocabulary-matching and learning-based reconstruction attacks while being orders of magnitude faster than SMPC.

  8. WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipedia

    cs.CL 2025-07 conditional novelty 6.0 of 10

    WETBench shows that existing machine-generated text detectors, particularly zero-shot methods, underperform on task-specific Wikipedia editing scenarios, with supervised detectors averaging 78% accuracy and zero-shot ...

  9. Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Framework

    cs.HC 2025-06 conditional novelty 6.0 of 10

    Structured moral prompts, especially first-principles reasoning, improve LLM moral classification accuracy across 12 open models and four benchmarks, and reasoning distillation transfers these gains to a 3B model.

  10. Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management

    cs.AR 2025-04 conditional novelty 6.0 of 10

    A dependency-aware unified caching manager and cost-model swapper reduce Time-To-First-Token for multi-LoRA LLM serving by roughly 50 to 60 percent over vLLM and S-LoRA.

  11. CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation

    cs.CL 2025-01 conditional novelty 6.0 of 10

    A confidence-reward score for selecting preference pairs improves DPO-based machine translation fine-tuning over reward-only selection methods on ALMA-7B and NLLB-1.3B.

  12. FastSwitch: Optimizing Context Switching Efficiency in Fairness-aware Large Language Model Serving

    cs.LG 2024-11 conditional novelty 6.0 of 10

    FastSwitch reduces context-switching overhead in fairness-aware LLM serving with coarse-grained KV cache allocation, asynchronous multithreaded swapping, and KV cache reuse, reporting 1.4-11.2x tail-latency speedups o...

  13. On the Shortcut Learning in Multilingual Neural Machine Translation

    cs.CL 2024-11 conditional novelty 6.0 of 10

    Zero-shot off-target translation in multilingual NMT is caused by a learned shortcut toward the central language, and removing such examples late in training lets models forget it.

  14. Web-Browsing LLMs Can Access Social Media Profiles and Infer User Demographics

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Web-browsing LLMs can retrieve X profile content and infer demographics with above-chance accuracy in some cases, but the study's evidence is partly confounded by training-data memorization and a heavily reduced synth...

  15. Psychology-Driven Enhancement of Humour Translation

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A decomposition-and-recomposition prompt method for humor translation reports large gains on LLM-based metrics, but the evaluation lacks human validation and statistical checks.

  16. Information Loss in LLMs' Multilingual Translation: The Role of Training Data, Language Proximity, and Language Family

    cs.CL 2025-06 reject novelty 5.0 of 10

    Round-trip translation quality in GPT-4 and Llama 2 is jointly shaped by training data volume and language distance from English, with orthographic, phylogenetic, syntactic, and geographic distances as the strongest p...

  17. Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution

    cs.CR 2025-06 conditional novelty 5.0 of 10

    Prompt injection can make LLM agents leak personal data they observed while executing tasks, with measured attack success rates around 15-20 percent and password leakage much rarer.

  18. Mutarjim: Advancing Bidirectional Arabic-English Translation with a Small Language Model

    cs.CL 2025-05 reject novelty 5.0 of 10

    A compact 1.5B Arabic-English model beats GPT-4o mini only on the authors' own Tarjama-25 benchmark, while trailing large models on standard WMT24++ and IWSLT2017 tests.

  19. Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately

    cs.LG 2025-05 conditional novelty 5.0 of 10

    SART reduces LLM serving latency by combining redundant branch sampling with early stopping and PRM-based dynamic branch pruning, while keeping answer accuracy comparable to baseline methods.

  20. Identifying Legal Holdings with LLMs: A Systematic Study of Performance, Scale, and Memorization

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Zero-shot GPT-4o and other LLMs match fine-tuned legal models on CaseHOLD, performance scales with model size, and results persist after case names are anonymized.

  21. Towards Multilingual LLM Evaluation for Baltic and Nordic languages: A study on Lithuanian History

    cs.CL 2025-01 conditional novelty 5.0 of 10

    On a translated Lithuanian history multiple-choice benchmark, GPT-4o beats all tested open and Nordic-tuned models, and Nordic-language fine-tuning does not improve accuracy.

  22. Self-Evolution Knowledge Distillation for LLM-based Machine Translation

    cs.CL 2024-12 conditional novelty 5.0 of 10

    A token-adaptive distillation method that mixes teacher and ground-truth targets only for hard tokens yields consistent BLEU gains in LLM translation.

  23. Hallucinations in medical devices

    eess.IV 2025-08 conditional novelty 4.0 of 10

    AI hallucinations in medical devices are defined as plausible errors, either impactful or benign, to guide device evaluation.

  24. A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis

    cs.CL 2025-06 conditional novelty 4.0 of 10

    An LLM agent that reframes beam analysis as OpenSeesPy code generation reaches over 99 percent reliability on a small benchmark, but chiefly because the prompt contains a near-identical solved example.

  25. Rethinking the Outlier Distribution in Large Language Models: An In-depth Study

    cs.CL 2025-05 reject novelty 4.0 of 10

    The paper claims that most massive activations in LLMs are residual-propagated 'fake' outliers that can be removed without accuracy loss.

  26. End-to-End Bangla AI for Solving Math Olympiad Problem Benchmark: Leveraging Large Language Model Using Integrated Approach

    cs.CL 2025-01 conditional novelty 4.0 of 10

    Fine-tuning Qwen2.5-7B with translated math datasets plus retrieval and tool-integrated reasoning yields 71/100 on a Bangla math olympiad test set, versus 77/100 for a larger base model.

  27. BhashaVerse : Translation Ecosystem for Indian Subcontinent Languages

    cs.CL 2024-12 reject novelty 4.0 of 10

    A claimed 2B-parameter multi-task translation model for 36 Indian languages, built from pivoted and synthetic corpora, evaluated without baselines and with inconsistent reported numbers.

  28. BanglaDialecto: An End-to-End AI-Powered Regional Speech Standardization

    cs.CL 2024-11 conditional novelty 4.0 of 10

    An ASR + MT + TTS pipeline converts Noakhali dialect speech to standard Bangla, with Whisper-large V2 achieving 0.8% CER and BanglaT5 a 41.6 BLEU on the authors' NDD dataset.

  29. LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures

    cs.CR 2025-05 conditional novelty 3.0 of 10

    This survey categorizes attacks on large language models by lifecycle phase and maps them to prevention and detection defenses, concluding that only a few defenses are highly effective.

Pith tools