Pith. sign in

REVIEW 5 major objections 6 minor 34 references

Do LLMs exhibit the same commonsense capabilities across languages?

T0 review · 5 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that open-source LLMs generate commonsense-consistent sentences more reliably in English than in Spanish, Dutch, or Valencian, and presents MULTICOM, a four-language benchmark that makes the gap measurable.

desk verdict A genuinely useful multilingual commonsense benchmark, but the headline English-superiority result rests on machine-translated references and English-centric metrics that may inflate the gap. read the letter →

arxiv 2509.06401 v1 pith:UIT7T7N2 submitted 2025-09-08 cs.CL

Ivan Martínez-Murillo , Elena Lloret , Paloma Moreda , Albert Gatt This is my paper
classification cs.CL
keywords multilingualcommonsensegenerationconstrainedtextLLMevaluationEnglishbiaslow-resourcelanguagesbenchmarkreasoningLLM-as-a-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models show the same commonsense reasoning ability when generating text in different languages. To test this, the authors build MULTICOM, a benchmark in English, Spanish, Dutch, and Valencian, in which a model must produce a plausible everyday sentence containing three given words. Evaluating five open-source model families with automatic metrics, LLM judges, and human raters, they find a consistent pattern: outputs are judged more commonsensical in English, with notably lower scores in the less-resourced languages. The paper's central claim is that this English advantage is real across evaluation methods, while adding a supporting context has mixed but often helpful effects for Dutch and Valencian.

What carries the argument

The engine of the argument is the contrast between reference and counterfactual sentences in MULTICOM. For each triplet of keywords, the dataset provides a commonsense-consistent target sentence and a fluent but nonsensical counterfactual; a model's commonsense quality is measured by the gap between its similarity to the target and its similarity to the counterfactual. A wider gap means the model can tell everyday logic from nonsense. Building on that, the benchmark combines semantic metrics (BERTScore, Universal Sentence Encoder cosine similarity), syntactic-dependency comparisons (via Stanza and Levenshtein distance or vector cosine), plus two LLM-as-judge scorers and human ratings, so the

What would settle it

Native-speaker rewrites of the MULTICOM reference sentences and LLM judges prompted in each target language, with humans rating the same outputs: if English no longer ranks first under both changes, the claim of an inherent English advantage fails.

Watch

Extended reading notes

Core claim

The central claim is that open-source LLMs' commonsense generation is not language-uniform: the same models that reliably produce plausible sentences in English produce weaker, less commonsensical sentences in Spanish, Dutch, and Valencian. The evidence comes from MULTICOM, which extends the Spanish COCOTEROS corpus with counterfactual and unrelated references and translates it into four languages. Models are scored by how much closer their outputs sit to a commonsense reference than to a deliberately nonsensical counterfactual, and by LLM judges and human annotators. All three evaluation strands agree on the ranking, with English first and Valencian last; the authors interpret this as a con

Load-bearing premise

The translated non-English reference sentences are natural, idiomatic gold standards; if machine translation produced awkward or non-typical sentences, models score worse against them regardless of their true commonsense quality, inflating the English advantage.

Editorial extensions

If this is right

  • If the claim holds, English-only commonsense evaluations overstate the general commonsense competence of current open LLMs.
  • Adding a supporting context sentence is a cheap intervention that tends to improve generation in Dutch and Valencian, even though it helps little or hurts in English and Spanish.
  • The target-versus-counterfactual gap is a usable, reference-contrast measure of commonsense that can be ported to other languages.
  • Model families whose pretraining emphasized language diversity show a flatter language profile, suggesting data curation can narrow, but not remove, the English gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the LLM judges were prompted in English with English rubrics, part of the measured gap could be judge bias rather than purely generation quality; re-running the judges in each target language would isolate this.
  • The counterfactual-contrast design could be reused as a lightweight probe: since only the contrast sentences need translating, the benchmark could be extended to many more low-resource languages cheaply.
  • If the English edge comes mainly from pretraining-data volume, fine-tuning on a modest amount of target-language commonsense text might shrink the gap faster than scaling model size, which the paper found to have only modest effects.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. Martínez-Murillo et al. present MULTICOM, a four-language extension (English, Spanish, Dutch, Valencian) of the Spanish COCOTEROS corpus, and use it to evaluate open-weight LLMs from five families (LLaMA, Qwen, Gemma, EuroLLM, Salamandra) on constrained commonsense generation from a keyword triple, with and without a supporting context. The evaluation triangulates automatic metrics (BERTScore, Universal Sentence Encoder cosine similarity, dependency/Levenshtein and dependency/vector cosine similarity), two LLM-as-a-judge models (Prometheus-2, JudgeLM), and a 20-instance human annotation study of one model. The paper reports that the reference-vs-counterfactual gap is consistently largest in English, that performance is lower in Spanish, Dutch, and Valencian, that context has mixed but generally positive effects for lower-resourced languages, and that human and LLM judgments broadly agree. The dataset is released publicly.

Significance. If the empirical conclusion holds, the paper provides a useful multilingual benchmark and a broad comparison of open-source LLMs; the public release of MULTICOM and the multi-method evaluation are strengths. The claim is not circular: the reference/counterfactual gap is an external, fixed measurement. However, the central comparison rests on the assumption that the translated reference sentences are equally natural, idiomatic, and commonsense-valid gold standards in all four languages, and that the automatic metrics are language-invariant. The manuscript does not currently validate that assumption, and several design choices—English-only prompts and rubrics, English-centric encoders, no statistical testing—are acknowledged or self-evident confounds. Thus the paper is a potentially valuable contribution, but it needs substantial supporting evidence before the cross-lingual claim is established.

major comments (5)
  1. [§3.1 and Appendix A] The load-bearing assumption is that the non-English references are equally valid gold standards. The translation pipeline uses OPUS-MT and Salt, with only a Grok-based keyword-presence check. Appendix A's alignment prompt explicitly allows the model to replace a missing keyword with 'another word from the sentence', so some non-English instances may no longer instantiate the original concept triple; this changes the task across languages. No evaluation of grammaticality, idiomaticity, or commonsense plausibility of the translated references is reported. Because the gap score is computed relative to these references, translationese or unnatural references would mechanically reduce the gap in non-English languages, independent of model quality. The paper needs either manual validation of a sample of translations or a robustness analysis restricted to high-confidence translations.
  2. [§3.3 and §4.1] The automatic metrics are not shown to be language-fair. The USE model is not specified; the SpaCy models used for dependency/vector similarity have different coverage and training distributions; the Levenshtein match threshold and cosine-similarity threshold are free parameters but are not reported; and BERTScore with bert-base-multilingual-cased may not be equally reliable for Valencian. The Limitations section acknowledges the general encoder/parser issue, but no sensitivity analysis or per-language calibration is provided. Given that the automatic metrics are the primary quantitative evidence, the paper should report model versions, thresholds, and results of a language-appropriateness check (e.g., comparing metric decisions against human judgments on the same items).
  3. [Abstract and §4.1] The abstract states 'significantly lower performance' in less-resourced languages, but no statistical tests, confidence intervals, or effect-size measures are reported. Table 1 gives only point estimates for one model, and the appendix figures are descriptive. Given that outputs for the same model across languages are correlated and that the metrics may have different ranges or floor/ceiling effects, the 'significant' language differences are unsupported. The authors should report per-language score distributions, paired comparisons across models, or at least bootstrap intervals for the reported gaps.
  4. [§4.3] The human evaluation is limited to 20 instances, one model, no-context condition, and three annotators per language; Valencian was judged by Catalan speakers. The reported majority agreement (0.75–0.95) is a weak reliability measure and does not address the reference-equivalence problem: human raters scored generated sentences without being asked whether the reference translations are equally natural. This small sample cannot validate the broad cross-lingual claim, nor can it disambiguate model quality from reference quality. A stratified human evaluation that also rates reference naturalness across languages would be needed.
  5. [§3.3 and §4.2] Prometheus and JudgeLM are given English prompts and rubrics even for non-English outputs, as shown in Appendices C and D. The manuscript acknowledges this as a limitation, but because the cross-lingual comparison is the central claim, it is load-bearing. A judge with weaker target-language competence may under-score non-English outputs. The paper should include a per-language calibration or prompt-language invariance check, e.g., judge scores against human scores on a stratified sample per language, to rule out an LLM-judge bias as the source of the English advantage.
minor comments (6)
  1. [§2] Typographical issues: 'several several' and 'commonense' appear in the Related Work section; 'Gemmma' appears in §3.2. These should be corrected.
  2. [Table 1] The column header order 'ES EN CA-VA NL' is inconsistent with the enumeration in the text ('English (EN), Spanish (ES), Valencian (CA-VA), Dutch (NL)'). Align the header order with the text or vice versa for readability.
  3. [Appendix A] The alignment prompt references 'keywords_es' even when the source language for Dutch was English. Clarify how the post-processing prompt was adapted for each translation direction, and how the final keywords were validated after replacement.
  4. [References] The Grok citation is incomplete; the Miller et al. reference contains a stray 'Damerau?Levenshtein'; the Prolific reference lacks a date/access detail. These should be cleaned up.
  5. [§4.1] The main text presents only LLaMA results; comprehensive results are in appendices as figures, which are difficult to read in PDF form. Consider providing numeric tables or higher-resolution figures so that the reported cross-model patterns can be checked.
  6. [Figure 3] The caption says 'the tested models' but only one model (LLaMA-3.2-3B-Instruct) is evaluated. Correct the caption to reflect the single-model evaluation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the English-superiority finding is an empirical comparison against fixed references/counterfactuals; the only author-overlap citation (COCOTEROS) is a dataset resource, not a load-bearing theorem.

full rationale

The claimed result is that open-source LLMs generate commonsense sentences more reliably in English than in Spanish, Dutch, or Valencian. The derivation chain is: (1) MULTICOM is built from COCOTEROS and translated to four languages (Section 3.1); (2) models generate sentences from keyword triples, with or without context; (3) automatic metrics compute similarity to a target reference and to a counterfactual, and the gap is interpreted as commonsense quality; (4) LLM-as-judge and human ratings are correlated with this gap. No step fits inputs to outputs: the references and counterfactuals are fixed prior to model evaluation, are not derived from the evaluated models, and no parameter is fitted to the test data and then reported as a prediction. The overlap with prior work is the COCOTEROS citation; this is a corpus used as test material, not a uniqueness theorem or an ansatz, so it does not make the argument circular. The paper's own Limitations section acknowledges that English-centric encoders, English-language rubrics, and Catalan speakers for Valencian could bias comparisons. These are validity threats and could explain part of the English advantage, but they are measurement concerns, not definitional reductions. Likewise, the possibility that machine-translated non-English references are less idiomatic (Section 3.1) is a benchmark-quality concern; it does not make the English-superiority claim equivalent to the benchmark's construction. Because no equation or fitted value is renamed as a prediction, the analysis is self-contained and not circular.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the translated benchmark as a cross-lingual gold standard, the validity of the gap-based automatic metric, the quality of the source dataset, and the neutrality of English prompts. These are domain assumptions, not standard math axioms. The two threshold parameters are the only free numeric inputs, and both are undisclosed.

free parameters (2)
  • Levenshtein dependency match threshold = not stated
    Section 3.3 says a dependency triplet counts as a match when its average Levenshtein distance falls below 'a predefined threshold', but the value and selection procedure are never reported.
  • Dependency vector cosine similarity threshold = not stated
    Section 3.3 says a match is recorded when cosine similarity 'exceeds a predefined threshold', with no numeric value or tuning details given.
assumptions (4)
  • domain assumption Machine-translated references are valid and equivalent gold standards across languages.
    All non-Spanish data in MULTICOM is produced by machine translation (OPUS-MT for English and Dutch, Salt for Valencian) with Grok-based keyword repair. Section 3.1. If translations are unnatural, language comparisons are confounded.
  • domain assumption Similarity to a reference versus a counterfactual is a valid proxy for commonsense generation quality.
    The main automatic metric uses the gap in similarity between generated vs target and generated vs counterfactual, assuming the similarity model captures the distinction between commonsense and non-commonsense (Section 3.3).
  • domain assumption COCOTEROS is a valid source of commonsense generation instances.
    The benchmark extends COCOTEROS without independent validation of the Spanish source's commonsense quality (Section 3.1).
  • domain assumption English-language prompts and rubrics are appropriate for all target languages.
    Generation prompts and judge rubrics are written in English for all languages based on unshown preliminary analysis (Appendices D and E).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do LLMs exhibit the same commonsense capabilities across languages?." pith.science (2026). https://pith.science/paper/UIT7T7N2

@misc{pith2026250906401,
  author       = {Pith},
  title        = {Pith review of: Do LLMs exhibit the same commonsense capabilities across languages?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UIT7T7N2}},
  note         = {Machine review of arXiv:2509.06401}
}
read the original abstract

This paper explores the multilingual commonsense generation abilities of Large Language Models (LLMs). To facilitate this investigation, we introduce MULTICOM, a novel benchmark that extends the COCOTEROS dataset to four languages: English, Spanish, Dutch, and Valencian. The task involves generating a commonsensical sentence that includes a given triplet of words. We evaluate a range of open-source LLMs, including LLaMA, Qwen, Gemma, EuroLLM, and Salamandra, on this benchmark. Our evaluation combines automatic metrics, LLM-as-a-judge approaches (using Prometheus and JudgeLM), and human annotations. Results consistently show superior performance in English, with significantly lower performance in less-resourced languages. While contextual support yields mixed results, it tends to benefit underrepresented languages. These findings underscore the current limitations of LLMs in multilingual commonsense generation. The dataset is publicly available at https://huggingface.co/datasets/gplsi/MULTICOM.

Figures

Figures reproduced from arXiv: 2509.06401 by the authors.

Figure 1
Figure 1. LLM-as-Judge results with Prometheus-2 [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. LLM-as-Judge results with JudgeLM, normalised to a 1-5 scale [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Results obtained for the tested models with [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Results obtained for the LLaMA models with the automatic metrics. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Results obtained for the Qwen models with the automatic metrics. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Results obtained for the EuroLLM models with the automatic metrics. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Results obtained for the Salamandra models with the automatic metrics. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Results obtained for the Gemma models with the automatic metrics. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 17 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Zeinab Aghahadi and Alireza Talebpour. 2022. Avicenna: a challenge dataset for natural language generation toward commonsense syllogistic reasoning. Journal of Applied Non-Classical Logics, 32(1):55--71

    work page 2022

  4. [4]

    Prajjwal Bhargava and Vincent Ng. 2022. Commonsense knowledge reasoning and generation with pre-trained language models: A survey. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12317--12325

    work page 2022

  5. [5]

    Damian Blasi, Antonios Anastasopoulos, and Graham Neubig. 2022. https://doi.org/10.18653/v1/2022.acl-long.376 Systematic Inequalities in Language Technology Performance across the World 's Languages . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( Volume 1: Long Papers ) , pages 5486--5505, Dublin, Ireland. As...

    doi:10.18653/v1/2022.acl-long.376 2022

  6. [6]

    Fredrik Carlsson, Joey \"O hman, Fangyu Liu, Severine Verlinden, Joakim Nivre, and Magnus Sahlgren. 2022. Fine-grained controllable text generation using non-residual prompting. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6837--6857

    work page 2022

  7. [7]

    Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, and 1 others. 2018. Universal sentence encoder. arXiv preprint arXiv:1803.11175

    arXiv 2018

  8. [8]

    Do, Junze Li, Tung-Duong Vuong, Zhaowei Wang, Yangqiu Song, and Xiaojuan Ma

    Quyet V. Do, Junze Li, Tung-Duong Vuong, Zhaowei Wang, Yangqiu Song, and Xiaojuan Ma. 2024. https://doi.org/10.48550/arXiv.2411.03964 What really is commonsense knowledge? CoRR, abs/2411.03964

    arXiv doi:10.48550/arxiv.2411.03964 2024

Show all 34 references
  1. [9]

    Aitor Gonzalez-Agirre, Marc Pàmies, Joan Llop, Irene Baucells, Severino Da Dalt, Daniel Tamayo, José Javier Saiz, Ferran Espuña, Jaume Prats, Javier Aula-Blasco, Mario Mina, Iñigo Pikabea, Adrián Rubio, Alexander Shvets, Anna Sallés, Iñaki Lacunza, Jorge Palomar, Júlia Falcão,...

    2025 arXiv

  2. [10]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  3. [11]

    Grok. 2025. Grok 3 beta — the age of reasoning agents. Technical report, x.ai. [Accessed 01-07-2025]

    2025

  4. [12]

    Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing (2nd Edition). Prentice-Hall, Inc., USA

    2009

  5. [13]

    Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.248 Prometheus 2: An open source language model specialized in evaluating other la...

    2024 doi

  6. [14]

    Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The Winograd Schema Challenge . In Proceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning , pages 552--561

    2012

  7. [15]

    Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.165 C ommon G en: A constrained text generation challenge for generative commonsense reasoning . In Findings of the Ass...

    2020 doi

  8. [16]

    Mar \' a Mir \'o Maestre, Iv \'a n Mart \' nez-Murillo, Elena Lloret, Paloma Moreda, and Armando Su \'a rez Cueto. 2024. Cocoteros: A spanish corpus with contextual knowledge for natural language generation. In 40th Annual Conference of the Spanish Association for Natural Lang...

    2024

  9. [17]

    Guerreiro, Ricardo Rei, Duarte M

    Pedro Henrique Martins, Patrick Fernandes, João Alves, Nuno M. Guerreiro, Ricardo Rei, Duarte M. Alves, José Pombal, Amin Farajian, Manuel Faysse, Mateusz Klimaszewski, Pierre Colombo, Barry Haddow, José G.C. de Souza , Alexandra Birch, and André F.T. Martins. 2025. https://do...

    2025 doi

  10. [18]

    Miller, Agnes F

    Frederic P. Miller, Agnes F. Vandome, and John McBrewster. 2009. Levenshtein Distance: Information theory, Computer science, String (computer science), String metric, Damerau?Levenshtein distance, Spell checker, Hamming distance. Alpha Press

    2009

  11. [19]

    Prolific. 2025. [Accessed 09-07-2025]. https://www.prolific.com/ [link]

    2025

  12. [20]

    Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. https://nlp.stanford.edu/pubs/qi2020stanza.pdf Stanza: A Python natural language processing toolkit for many human languages . In Proceedings of the 58th Annual Meeting of the Association for Co...

    2020

  13. [21]

    Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S. Yu. 2025. https://doi.org/10.1016/j.patter.2024.101118 A survey of multilingual large language models . Patterns, 6(1). Publisher: Elsevier

    2025

  14. [22]

    Harald Steck, Chaitanya Ekanadham, and Nathan Kallus. 2024. Is cosine-similarity of embeddings really about similarity? In Companion Proceedings of the ACM Web Conference 2024, pages 887--890

    2024

  15. [23]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan...

    2024 arXiv

  16. [24]

    Qwen Team. 2025. https://arxiv.org/abs/2505.09388 Qwen3 technical report . Preprint, arXiv:2505.09388

    2025 arXiv

  17. [25]

    o hler, Ren \'e J \

    Klaudia Thellmann, Bernhard Stadler, Michael Fromm, Jasper Schulze Buschhoff, Alex Jude, Fabio Barth, Johannes Leveling, Nicolas Flores-Herr, Joachim K \"o hler, Ren \'e J \"a kel, and 1 others. 2024. Towards multilingual llm evaluation for european languages. arXiv preprint a...

    2024 arXiv

  18. [26]

    J \"o rg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT — B uilding open translation services for the W orld. In Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT), Lisbon, Portugal

    2020

  19. [27]

    Generalitat Valenciana. 2025. S alt.usu. https://salt.gva.es/es/traductor. [Accessed 01-07-2025]

    2025

  20. [28]

    Han Wang, Yang Liu, Chenguang Zhu, Linjun Shou, Ming Gong, Yichong Xu, and Michael Zeng. 2021. https://doi.org/10.18653/v1/2021.findings-acl.269 Retrieval enhanced model for commonsense generation . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021,...

    2021 doi

  21. [29]

    Yuemei Xu, Ling Hu, Jiayi Zhao, Zihan Qiu, Yuqi Ye, and Hanwen Gu. 2024. https://doi.org/10.48550/arXiv.2404.00929 A Survey on Multilingual Large Language Models : Corpora , Alignment , and Bias . arXiv preprint. ArXiv:2404.00929 version: 1

    2024 doi arXiv

  22. [30]

    Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Yun Xing, Junjue Wang, Huitao Li, Xin Li, Kunyu Yu, Nan Liu, Qingyu Chen, Douglas Teodoro, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, and Irene Li. 2025. https://doi.org/10.48550/arXiv.2503.1049...

    2025 doi arXiv

  23. [31]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with bert . In International Conference on Learning Representations

    2020

  24. [32]

    Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.491 Don ' t trust C hat GPT when your question is not in E nglish: A study of multilingual abilities and types of LLM s . In Proceedings of the 2023 Conferen...

    2023 doi

  25. [33]

    Yunxiang Zhang and Xiaojun Wan. 2024. Situated G en: Incorporating geographical and temporal contexts into generative commonsense reasoning. Advances in Neural Information Processing Systems, 36

    2024

  26. [34]

    Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2025. https://openreview.net/forum?id=xsELpEPn4A Judge LM : Fine-tuned large language models are scalable judges . In The Thirteenth International Conference on Learning Representations

    2025

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.