Pith. sign in

REVIEW 3 major objections 3 minor 56 references

Evaluating LLMs on Chinese Idiom Translation

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The best translation systems tested, including GPT-4, mistranslate Chinese idioms in 28% of cases, and standard automatic metrics cannot detect these failures.

desk verdict Useful first benchmark for Chinese idiom translation, but every headline number is provisional until the body confirms annotation reliability and a held-out detector. read the letter →

arxiv 2508.10421 v1 pith:Z2QHUTM7 submitted 2025-08-14 cs.CL

Cai Yang , Yao Dou , David Heineman , Xiaofeng Wu , Wei Xu This is my paper
classification cs.CL
keywords ChineseidiomsmachinetranslationlargelanguagemodelsevaluationerrortaxonomyhumanannotationfigurativeIdiomEval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that Chinese idiom translation is a distinct, measurable failure mode of machine translation that current quality checks do not capture. It builds IdiomEval, a set of 900 human-annotated translation pairs from nine systems (including GPT-4 and Google Translate) across web, news, Wikipedia, and social media, and labels each output with one of four error types: incorrect, literal, partial, or missing. The headline result is that the best system tested, GPT-4, still gets 28% of these idioms wrong, while standard automatic metrics correlate with human idiom ratings at Pearson below 0.48—so a human-looking score can hide a mistranslated idiom. A reader should care because, if this holds, idiom quality is an invisible but common gap in every LLM-based translation pipeline, and fixing it requires a dedicated evaluation instrument rather than better generic metrics.

What carries the argument

IdiomEval is the central instrument: 900 human-annotated translation pairs from nine systems across four domains, organized by a four-way error taxonomy (incorrect, literal, partial, missing) that turns an anecdotal sense that 'idioms are translated badly' into countable categories. The taxonomy does the work: it produces the 28% error rate, provides the human ratings against which standard metrics correlate below 0.48, and supplies labeled training data for the $F_1=0.68$ error detector.

What would settle it

Ask a second set of annotators to re-label the same 900 translation pairs with the same four-category taxonomy and compute inter-annotator agreement; if agreement is low, or if the 28% error rate for GPT-4 moves materially, the headline results are annotation-dependent rather than a stable property of the systems.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that Chinese idioms expose a blind spot in modern translation systems. Across 900 hand-annotated translation pairs produced by nine systems, every system examined makes errors that fall into four categories—incorrect, literal, partial, and missing—and the best-performing system, GPT-4, errs in 28% of cases. Existing automatic evaluation metrics track human idiom-quality judgments poorly, with Pearson correlation below 0.48, so these failures would not show up in routine evaluations. The paper additionally shows that a detector trained on the annotation data reaches $F_1=0.68$, indicating that automatic error detection is possible but far from perfec

Load-bearing premise

The measurement stands or falls on the human annotations: the abstract reports no inter-annotator agreement, no sampling procedure for the idioms or domains, and no definition of 'correct' for idioms with several acceptable renderings, so if raters disagree, the 28% error rate, the correlation figures, and the detector's $F_1$ all shift.

Editorial extensions

If this is right

  • If the 28% figure holds, GPT-4 and comparable systems silently mistranslate more than one in four Chinese idioms in ordinary text.
  • Because standard metrics correlate below 0.48 with human idiom ratings, systems optimized on those metrics can appear high-quality while regressing on idioms.
  • The $F_1=0.68$ detector offers a concrete way to flag suspect idiom translations for human review even when overall translation scores look good.
  • The four error types imply different fixes: literal translations need meaning recovery, partial translations need completeness, missing translations need detection, and incorrect translations need replacement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper samples four written domains, so the 28% rate is not shown to extend to spoken, literary, or domain-specialized Chinese; if it did, idiom errors would be an even more widespread everyday problem than the paper demonstrates.
  • A natural extension the paper leaves implicit is to test whether the same blind spot appears for idioms in other languages, which would turn a Chinese-specific result into a general property of how large language models handle figurative language.
  • One testable consequence: a translation system prompted or trained to check for the four error types before output should reduce the 28% error rate; that experiment is not in the paper.
  • The low metric correlations imply idiom quality should become its own reporting dimension in translation evaluations, rather than being folded into a single overall score.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper introduces IdiomEval, a framework with an error taxonomy for Chinese idiom translation, and reports a human-annotated dataset of 900 translation pairs produced by nine MT systems, including GPT-4o and Google Translate, across four domains (web, news, Wikipedia, social media). The headline empirical claims are that contemporary systems mistranslate Chinese idioms at a high rate (the best system, GPT-4, errs in 28% of cases), standard automatic metrics correlate poorly with human judgments (Pearson < 0.48), and an error-detection model achieves F1 = 0.68. As received, however, the full text is corrupted mojibake, so only the abstract and a few recoverable fragments can be evaluated. The validity of every reported number depends on the annotation methodology, which cannot be inspected in the supplied text.

Significance. If the claims hold, this paper addresses a real and under-studied failure mode in LLM-based MT: idiom translation quality is largely invisible to standard metrics. The proposed resource (900 annotated pairs, nine systems, four domains) and error taxonomy (incorrect, literal, partial, missing) would be useful to the community. The paper also makes a practical claim that idiom errors can be detected automatically at F1 = 0.68. However, the significance is currently conditional: the headline error rates and correlations are only as reliable as the human annotations, and the detector's score is only meaningful if evaluated on held-out data. The manuscript does not provide the evidence needed to assess either point.

major comments (3)
  1. [Abstract] The central empirical claims (28% error rate, Pearson < 0.48, F1 = 0.68) are all outputs of the human annotation process described only as 'We annotate 900 translation pairs ... across four domains.' The abstract reports no inter-annotator agreement, no sampling procedure for selecting idioms or source texts, and no definition of what counts as an acceptable translation for idioms that admit multiple correct renderings. Without these, the boundaries between 'literal,' 'partial,' and 'incorrect' are subjective, and the reported rates and correlations are not stable measurements. Please report agreement statistics and the annotation guidelines, including how multiple acceptable translations were handled.
  2. [Abstract / F1 detector] The statement 'we thus develop improved models that achieve F1 scores of 0.68 for detecting idiom translation errors' does not specify whether the detector was evaluated on a held-out set disjoint from the 900 training/annotation instances. If the F1 is computed on the same annotations used to train the detector, it is a self-evaluation and does not support the implied generalization. Please provide the train/test split, the number of instances, and the precision/recall/F1 for each error category.
  3. [Full text (all sections)] The supplied full text is corrupted mojibake; essentially none of the methodology, experiments, tables, or equations can be read. Consequently, the annotation procedure, system list, metric computation, and taxonomy definitions cannot be verified. This is load-bearing because the paper's claims are empirical and depend entirely on details that are not visible. A clean, readable manuscript must be provided before the work can be evaluated.
minor comments (3)
  1. [Abstract] The abstract mentions 'GPT-4o' among the evaluated systems but then says 'The best-performing system, GPT-4.' Please clarify whether GPT-4 and GPT-4o are distinct entries and list all nine systems in the main text.
  2. [Abstract] The 28% error rate for GPT-4 is reported without a confidence interval or the number of items it is based on; given the centrality of this number, a simple CI would help assess stability.
  3. [Abstract] 'Improved models' is plural but only one F1 value is reported; specify the model architecture(s) and whether F1 is macro-averaged or computed on pooled categories.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: headline measurements are anchored to external human annotations, not to the authors' own definitions or fitted predictions.

full rationale

The paper's central claims are empirical measurements: 900 translation pairs from nine systems are human-annotated, then error rates and metric correlations are computed against those annotations. That is an external benchmark, not a derivation that feeds its own conclusion back in. The abstract introduces an error taxonomy, but the taxonomy is a labeling scheme, not an equation that forces the 28% error rate or the Pearson < 0.48 figures by construction. The improved detection model achieving F1 = 0.68 is the only place where the authors' own labels are reused, but the abstract does not state that it is trained and evaluated on the same set, and no specific reduction to a fitted value or to a self-citation chain can be exhibited from the available text. The supplied full text is corrupted, so no internal equations or citations are inspectable; on the abstract alone, none of the enumerated circularity patterns is present. The absence of inter-annotator agreement and sampling details is a validity threat, not a circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are identifiable from the abstract: the headline numbers come from human annotation and system outputs, not from fitting constants. The F1=0.68 detector is a trained model, but its fitted weights are the deliverable, not a parameter the central claim depends on. No invented physical or formal entities are postulated; IdiomEval is a benchmark and taxonomy, not a theorized entity with independent evidence requirements.

assumptions (3)
  • domain assumption Chinese idiom translation adequacy is best judged by human raters using the authors' error taxonomy (incorrect, literal, partial, missing).
    The entire evaluation treats the taxonomy and human ratings as ground truth; the abstract reports no inter-annotator agreement, so rating reliability is assumed.
  • domain assumption The four sampled domains (web, news, Wikipedia, social media) and the chosen idiom set represent the difficulty of Chinese idiom translation generally.
    Headline error rates are presented as general statements about system performance, but the abstract does not describe how idioms or sources were sampled.
  • domain assumption Standard MT metrics, implied but not named in the abstract, are the appropriate comparison for diagnosing idiom-quality measurement.
    The metric-correlation finding (Pearson below 0.48) presumes that existing metrics are the right baselines; the specific metric list and human-rating scale are not visible in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating LLMs on Chinese Idiom Translation." pith.science (2026). https://pith.science/paper/Z2QHUTM7

@misc{pith2026250810421,
  author       = {Pith},
  title        = {Pith review of: Evaluating LLMs on Chinese Idiom Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z2QHUTM7}},
  note         = {Machine review of arXiv:2508.10421}
}
abstract

Idioms, whose figurative meanings usually differ from their literal interpretations, are common in everyday language, especially in Chinese, where they often contain historical references and follow specific structural patterns. Despite recent progress in machine translation with large language models, little is known about Chinese idiom translation. In this work, we introduce IdiomEval, a framework with a comprehensive error taxonomy for Chinese idiom translation. We annotate 900 translation pairs from nine modern systems, including GPT-4o and Google Translate, across four domains: web, news, Wikipedia, and social media. We find these systems fail at idiom translation, producing incorrect, literal, partial, or even missing translations. The best-performing system, GPT-4, makes errors in 28% of cases. We also find that existing evaluation metrics measure idiom quality poorly with Pearson correlation below 0.48 with human ratings. We thus develop improved models that achieve F$_1$ scores of 0.68 for detecting idiom translation errors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 34 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  4. [4]

    Automatic evaluation and analysis of idioms in neural machine translation

    Christos Baziotis, Prashant Mathur, and Eva Hasler. Automatic evaluation and analysis of idioms in neural machine translation. In Andreas Vlachos and Isabelle Augenstein (eds.), Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp.\ 3682--3700, Dubrovnik, Croatia, May 2023. Association for Computa...

    work page doi:10.18653/v1/2023.eacl-main.267 2023

  5. [5]

    Findings of the wmt 2023 shared task on quality estimation

    Frederic Blain, Chrysoula Zerva, Ricardo Rei, Nuno M Guerreiro, Diptesh Kanojia, Jos \'e GC de Souza, Beatriz Silva, T \^a nia Vaz, Yan Jingxuan, Fatemeh Azadi, et al. Findings of the wmt 2023 shared task on quality estimation. In Proceedings of the Eighth Conference on Machine Translation, pp.\ 629--653, 2023

    work page 2023

  6. [6]

    Fillmore, Ralph Grishman, Nancy Ide, Alessandro Lenci, Catherine Macleod, and Antonio Zampolli

    Nicoletta Calzolari, Charles J. Fillmore, Ralph Grishman, Nancy Ide, Alessandro Lenci, Catherine Macleod, and Antonio Zampolli. Towards best practice for multiword expressions in computational lexicons. In International Conference on Language Resources and Evaluation, 2002

    work page 2002

  7. [7]

    Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation

    Verna Dankers, Christopher Lucas, and Ivan Titov. Can transformer be too compositional? analysing idiom processing in neural machine translation. ArXiv, abs/2205.15301, 2022 a

    work page Pith review arXiv 2022

  8. [8]

    Can transformer be too compositional? analysing idiom processing in neural machine translation

    Verna Dankers, Christopher Lucas, and Ivan Titov. Can transformer be too compositional? analysing idiom processing in neural machine translation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 3608--3626, Dublin, Irela...

    doi:10.18653/v1/2022.acl-long.252 2022

Show all 56 references
  1. [9]

    Smith, and Yejin Choi

    Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah A. Smith, and Yejin Choi. Is GPT -3 text indistinguishable from human text? scarecrow: A framework for scrutinizing machine text. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th An...

    2022 doi

  2. [10]

    Examining the tip of the iceberg: A data set for idiom translation

    Marzieh Fadaee, Arianna Bisazza, and Christof Monz. Examining the tip of the iceberg: A data set for idiom translation. ArXiv, abs/1802.04681, 2018

    2018 arXiv

  3. [11]

    Results of wmt23 metrics shared task: Metrics might be guilty but references are not innocent

    Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Fr \'e d \'e ric Blain, Daniel Deutsch, Craig Stewart, et al. Results of wmt23 metrics shared task: Metrics might be guilty but references are not innocent. In Proceedings...

    2023

  4. [12]

    Are llms breaking mt metrics? results of the wmt24 metrics shared task

    Markus Freitag, Nitika Mathur, Daniel Deutsch, Chi-Kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Fr \'e d \'e ric Blain, Tom Kocmi, Jiayi Wang, et al. Are llms breaking mt metrics? results of the wmt24 metrics shared task. In Proceedings of the Ninth Conference o...

    2024

  5. [13]

    CHENGYU - BENCH : Benchmarking large language models for chinese idiom understanding and use

    Yicheng Fu, Zhemin Huang, Liuxin Yang, Yumeng Lu, and Zhongdongming Dai. CHENGYU - BENCH : Benchmarking large language models for chinese idiom understanding and use. In ACL 2025 Student Research Workshop, 2025. URL https://openreview.net/forum?id=Ku0p1ZXkWA

    2025

  6. [14]

    Unsupervised compositional translation of multiword expressions

    Pablo Gamallo and Marcos Garcia. Unsupervised compositional translation of multiword expressions. In MWE-WN@ACL, 2019

    2019

  7. [15]

    xcomet: Transparent machine translation evaluation through fine-grained error detection

    Nuno M Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and Andr \'e FT Martins. xcomet: Transparent machine translation evaluation through fine-grained error detection. arXiv preprint arXiv:2310.10482, 2023

    2023 arXiv

  8. [16]

    Thresh: A unified, customizable and deployable platform for fine-grained text evaluation

    David Heineman, Yao Dou, and Wei Xu. Thresh: A unified, customizable and deployable platform for fine-grained text evaluation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.\ 336--345, 2023

    2023

  9. [17]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1 0 (2): 0 3, 2022

    2022

  10. [18]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  11. [19]

    M etric X -23: The G oogle submission to the WMT 2023 metrics shared task

    Juraj Juraska, Mara Finkelstein, Daniel Deutsch, Aditya Siddhant, Mehdi Mirzazadeh, and Markus Freitag. M etric X -23: The G oogle submission to the WMT 2023 metrics shared task. In Philipp Koehn, Barry Haddow, Tom Kocmi, and Christof Monz (eds.), Proceedings of the Eighth Con...

    2023 doi

  12. [20]

    M etric X -24: The G oogle submission to the WMT 2024 metrics shared task

    Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. M etric X -24: The G oogle submission to the WMT 2024 metrics shared task. In Barry Haddow, Tom Kocmi, Philipp Koehn, and Christof Monz (eds.), Proceedings of the Ninth Conference on Machine Translation, pp.\...

    2024

  13. [21]

    DEMETR : Diagnosing evaluation metrics for translation

    Marzena Karpinska, Nishant Raj, Katherine Thai, Yixiao Song, Ankita Gupta, and Mohit Iyyer. DEMETR : Diagnosing evaluation metrics for translation. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Lan...

    2022 doi

  14. [22]

    The languages of the world

    Kenneth Katzner and Kirk Miller. The languages of the world. Routledge, 2002

    2002

  15. [23]

    Memorization vs

    Aochong Oliver Li and Tanya Goyal. Memorization vs. reasoning: Updating LLM s with new knowledge. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025, pp.\ 25853--25874, Vien...

    2025 doi

  16. [24]

    Translate meanings, not just words: Idiomkb’s role in optimizing idiomatic translation with language models

    Shuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu, Hao Yang, Shimin Tao, and Yanghua Xiao. Translate meanings, not just words: Idiomkb’s role in optimizing idiomatic translation with language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\...

    2024

  17. [25]

    Text polishing with chinese idiom: Task, datasets and pre-trained baselines

    Junwei Liao and Shuai Cheng. Text polishing with chinese idiom: Task, datasets and pre-trained baselines. ACM Transactions on Asian and Low-Resource Language Information Processing, 2023

    2023

  18. [26]

    O pen E val: Benchmarking C hinese LLM s across capability, alignment and safety

    Chuang Liu, Linhao Yu, Jiaxuan Li, Renren Jin, Yufei Huang, Ling Shi, Junhui Zhang, Xinmeng Ji, Tingting Cui, Liutao Liutao, Jinwang Song, Hongying Zan, Sun Li, and Deyi Xiong. O pen E val: Benchmarking C hinese LLM s across capability, alignment and safety. In Yixin Cao, Yang...

    2024

  19. [27]

    Crossing the threshold: Idiomatic machine translation through retrieval augmentation and loss weighting

    Emmy Liu, Aditi Chaudhary, and Graham Neubig. Crossing the threshold: Idiomatic machine translation through retrieval augmentation and loss weighting. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Langua...

    2023 doi

  20. [28]

    Crosslingual generalization through multitask finetuning

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786, 2022

    2022 arXiv

  21. [29]

    Hello gpt-4o, May 2024

    OpenAI. Hello gpt-4o, May 2024. URL https://openai.com/index/hello-gpt-4o/

    2024

  22. [30]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp.\ 311--318, 2002

    2002

  23. [31]

    Ricardo Rei, Jos \'e G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and Andr \'e F. T. Martins. COMET -22: Unbabel- IST 2022 submission for the metrics shared task. In Philipp Koehn, Lo \" c Barrault, Ond r ej Bojar, ...

    2022

  24. [32]

    Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G

    Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and Andr \'e F. T. Martins. C omet K iwi: IST -unbabel 2022 submission for the quality estimatio...

    2022

  25. [33]

    Large language models for P ersian- E nglish idiom translation

    Sara Rezaeimanesh, Faezeh Hosseini, and Yadollah Yaghoobzadeh. Large language models for P ersian- E nglish idiom translation. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Co...

    2025 doi

  26. [34]

    Sag, Timothy Baldwin, Francis Bond, Ann A

    Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann A. Copestake, and Dan Flickinger. Multiword expressions: A pain in the neck for nlp. In Conference on Intelligent Text Processing and Computational Linguistics, 2002

    2002

  27. [35]

    Evaluating machine translation performance on C hinese idioms with a blacklist method

    Yutong Shao, Rico Sennrich, Bonnie Webber, and Federico Fancellu. Evaluating machine translation performance on C hinese idioms with a blacklist method. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018) , Miyazaki, Japan,...

    2018

  28. [36]

    O’Donnell

    Michaela Socolof, Jackie Chi Kit Cheung, Michael Wagner, and Timothy J. O’Donnell. Characterizing idioms: Conventionality and contingency. ArXiv, abs/2104.08664, 2021

    2021 arXiv

  29. [37]

    Learning and evaluating chinese idiom embeddings

    Minghuan Tan and Jing Jiang. Learning and evaluating chinese idiom embeddings. In Recent Advances in Natural Language Processing, 2021

    2021

  30. [38]

    A bert-based two-stage model for chinese chengyu recommendation

    Minghuan Tan, Jing Jiang, and Bingtian Dai. A bert-based two-stage model for chinese chengyu recommendation. ACM Trans. Asian Low Resour. Lang. Inf. Process., 20: 0 92:1--92:18, 2021

    2021

  31. [39]

    Petci: A parallel english translation dataset of chinese idioms

    Kenan Tang. Petci: A parallel english translation dataset of chinese idioms. ArXiv, abs/2202.09509, 2022

    2022 arXiv

  32. [40]

    Creative and context-aware translation of east asian idioms with gpt-4

    Kenan Tang, Peiyang Song, Yao Qin, and Xifeng Yan. Creative and context-aware translation of east asian idioms with gpt-4. arXiv preprint arXiv:2410.00988, 2024

    2024 arXiv

  33. [41]

    Id10m: Idiom identification in 10 languages

    Simone Tedeschi, Federico Martelli, and Roberto Navigli. Id10m: Idiom identification in 10 languages. In NAACL-HLT, 2022

    2022

  34. [42]

    Construction of C hinese idiom knowledge-base and its applications

    Lei Wang and Shiwen Yu. Construction of C hinese idiom knowledge-base and its applications. In Proceedings of the 2010 Workshop on Multiword Expressions: from Theory to Applications, pp.\ 11--18, Beijing, China, August 2010. Coling 2010 Organizing Committee. URL https://aclant...

    2010

  35. [43]

    The cultural differences between english and chinese idioms and translation

    Ran Wang. The cultural differences between english and chinese idioms and translation. In 2022 International Conference on Creative Industry and Knowledge Economy (CIKE 2022), pp.\ 610--614. Atlantis Press, 2022

    2022

  36. [44]

    A contrastive research on the definitions and categories of chinese and english idioms

    You Wang. A contrastive research on the definitions and categories of chinese and english idioms. In 7th International Conference on Humanities and Social Science Research (ICHSSR 2021), pp.\ 423--426. Atlantis Press, 2021

    2021

  37. [45]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  38. [46]

    A paradigm shift in machine translation: Boosting translation performance of large language models

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. A paradigm shift in machine translation: Boosting translation performance of large language models. In ICLR, 2024

    2024

  39. [47]

    mt5: A massively multilingual pre-trained text-to-text transformer

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934, 2020

    2010 arXiv

  40. [48]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng...

    2024 arXiv

  41. [49]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  42. [50]

    Multiword expression aware neural machine translation

    Andrea Zaninello and Alexandra Birch. Multiword expression aware neural machine translation. In International Conference on Language Resources and Evaluation, 2020

    2020

  43. [51]

    Ziheng Zeng and S. Bhat. Getting bart to ride the idiomatic train: Learning to represent idiomatic expressions. Transactions of the Association for Computational Linguistics, 10: 0 1120--1137, 2022

    2022

  44. [52]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  45. [53]

    Chid: A large-scale chinese idiom dataset for cloze test

    Chujie Zheng, Minlie Huang, and Aixin Sun. Chid: A large-scale chinese idiom dataset for cloze test. ArXiv, abs/1906.01265, 2019

    1906 arXiv

  46. [54]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  47. [55]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  48. [56]

    Idiom translation

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.