Pith. sign in

REVIEW 4 major objections 6 minor 277 references

Hidden Language Consistency Phenomena in Reasoning LLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Rising task difficulty can make reasoning models abruptly drop the requested language, and the drop can inflate accuracy at harder levels.

desk verdict The difficulty-resolved consistency analysis is a real step beyond PolyMath, but the dramatic breakdown numbers rest on an unvalidated language detector and should be treated as provisional. read the letter →

arxiv 2608.08447 v1 pith:KNULEB4U submitted 2026-08-09 cs.CL cs.AI

Muhammad Ali Shafique , Kelly Marchisio This is my paper · ORCID
classification cs.CLcs.AI
keywords multilingualreasoninglanguageconsistencybreakdownchain-of-thoughtquantizationLLMevaluationPolyMathlow-resourcelanguages
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that multilingual reasoning cannot be judged by final-answer accuracy alone: a model can solve a problem while thinking or answering in the wrong language, and this drift grows as problems get harder. Using PolyMath across eight languages and four difficulty levels, the authors identify four difficulty-dependent consistency behaviours and name the abrupt one the language consistency breakdown effect, where output-language consistency drops by 30 or more points between adjacent difficulty levels. Their central result is that this breakdown can break the usual monotonic accuracy–difficulty relationship, with accuracy preserved or even improved at a harder level as the model shifts to a dominant internal language. They also show that quantization changes language consistency independently of accuracy, so compressed models need separate consistency checks.

What carries the argument

The load-bearing machinery is a pair of per-difficulty metrics: thinking-language consistency (TC) and answer-language consistency (AC), each defined as the fraction of model outputs in which an automatic language detector identifies the reasoning trace or final answer as the requested language. Forced-target-language prompting supplies the reference language, and a breakdown is defined as a drop of at least $\Delta_{\mathrm{break}}=30$ percentage points in TC or AC between two adjacent difficulty levels. PolyMath supplies eight languages and four difficulty levels so the same benchmark can track accuracy and consistency jointly; tolerance-based voting with $\epsilon=1.0$ lets the quantization comparison treat small TC differences as ties.

What would settle it

Take the same eight languages and four difficulty levels, have human annotators label the language of a sample of thinking traces and final answers, and compare TC and AC with the script-based scores; if the abrupt drops in languages like Bengali, Swahili, or Telugu vanish under human labels, the breakdown effect is an artefact of detection rather than model behaviour.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that language consistency is a distinct, measurable axis of multilingual reasoning behaviour that moves with task difficulty. Thinking-language consistency (TC) and answer-language consistency (AC) are defined as the fraction of reasoning traces and final answers, respectively, whose detected language matches the requested language. Across eight languages and four difficulty levels, consistency either stays aligned with the input, stays misaligned, degrades gradually, or collapses abruptly; the abrupt case is formalised as a drop of at least 30 percentage points between consecutive difficulty levels and called the language consistency breakdown effect. The effect is concentrated in less strongly represented and non-Latin-script languages, and it can make accuracy rise at a harder level, as in Bengali, Swahili, and Telugu for DeepSeek-R1-Distill-Qwen-7B. Quantization can improve or degrade TC independent of accuracy, with GPTQ and AWQ often beating AutoRound on thinking-language consistency even though AutoRound best preserves accuracy.

Load-bearing premise

The load-bearing premise is that the benchmark's language-detection scripts correctly identify the language of thinking traces and answers in all eight languages; if they misread short, mixed, or code-switched outputs, the reported breakdowns and the four-case taxonomy shift.

Editorial extensions

If this is right

  • Accuracy-only leaderboards can misrank multilingual reasoning models, because a harder level can look 'better' while language consistency collapses.
  • A model's accuracy gain at higher difficulty should be treated as a possible language shift rather than improved reasoning.
  • Compressed-model deployments should measure thinking- and answer-language consistency alongside accuracy, since the method that best preserves accuracy does not best preserve language.
  • Reasoning models are more susceptible to answer-language breakdowns than the non-reasoning models tested, so reasoning-specific evaluation suites need language checks.
  • Relaxing the instruction to reason in a preferred language does not guarantee the final answer stays in the requested language at high difficulty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the breakdown is driven by the model's internal dominant language, a testable mitigation is to insert periodic language anchors or per-step target-language verification during decoding; the paper does not test this.
  • Because TC and AC inherit the accuracy of automatic language detection, manually annotating a sample of short, code-switched, or low-resource outputs would confirm whether the reported breakdowns are linguistic behaviour rather than detector artefacts.
  • The same difficulty-consistency analysis could be ported to non-mathematical reasoning domains, such as science or code generation, where longer generations may show earlier or sharper breakdowns.
  • The four-case taxonomy suggests that averaging consistency across languages hides the most important behaviour; reporting per-language trajectories should become the default for multilingual evaluation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies whether multilingual reasoning models preserve the requested language in their thinking traces and final answers as task difficulty increases. Using the PolyMath benchmark in eight languages and four difficulty levels, with five reasoning and three non-reasoning models under forced-target-language prompting, it defines thinking-language consistency (TC) and answer-language consistency (AC) as binary whole-sequence indicators. The authors report four difficulty-dependent consistency cases, a 'language consistency breakdown effect' defined by a 30-point drop between consecutive difficulty levels, and a claim that this breakdown can break the expected monotonic accuracy–difficulty relationship. They additionally compare W4A16 quantization methods (GPTQ, AWQ, AutoRound) for their effects on accuracy and TC, concluding that multilingual evaluation should jointly track accuracy, TC, AC, and difficulty.

Significance. If established, the core claim is practically important: accuracy-only evaluation of multilingual reasoning models would be incomplete, and language consistency would need to be tracked jointly with difficulty. The paper covers a broad set of models and languages, uses three seeds, and includes useful sensitivity analyses for the breakdown threshold and the tolerance parameter. The central finding, however, rests on a measurement instrument that is not validated, on comparisons across disjoint problem sets, and on point estimates without uncertainty quantification. The descriptive patterns in Figures 3 and 4 are visible and plausible, but the paper's stronger inferences—the breakdown effect and its causal role in non-monotonic accuracy—require additional evidence. The claimed contribution is therefore conditional on the validity of the language detector and the comparability of difficulty levels.

major comments (4)
  1. [§3, Eqs. (1)–(2); §4.2, Eqs. (3)–(4)] TC and AC are computed exclusively from whole-sequence hard language labels produced by the official PolyMath detection scripts, but the paper provides no validation of these detectors on these eight models or languages. A long mixed-language thinking trace can receive a hard label that flips abruptly at a critical ratio of language tokens, so the 'breakdown' in Case 4 and the numbers in Section 4.2 (e.g., Bengali TC dropping from 97.9% to 1.6%) may be a threshold artifact of gradual code-mixing rather than a sudden behavioral change. The authors should report detector confidence or token-level language proportions, validate the detector per language (especially Bengali, Swahili, and Telugu), and show that the breakdown persists under token-level or soft-label consistency metrics. This validation is load-bearing because Eqs. (1)–(2) are the basis for all four findings.
  2. [§4.3, Figure 3] The claim that language-consistency breakdown causes accuracy to 'preserve or improve' at a harder difficulty level compares accuracy on disjoint problem sets: low, medium, high, and top are different PolyMath items. The reported increases (Bengali 47.5% to 50.9%, Swahili 4.5% to 35.7%, Telugu 22.9% to 41.9%) could reflect difficulty calibration or item selection rather than a language-driven shift. To support the causal statement, the paper needs a control that holds problem content fixed while varying difficulty, or an item-level analysis showing that accuracy gains concentrate in generations where the language switched. Without this, the monotonicity-breaking claim in Section 4.3 is not established.
  3. [§3; Tables 2 and 3] All quantitative claims are point estimates without confidence intervals or significance tests, despite the statement that three seeds are used. The quantization ranking in Table 2 relies on average differences of a few accuracy points (e.g., AutoRound −0.80 vs. AWQ −2.53 for DeepSeek-R1-Distill-Qwen-7B), and Table 3 reports co-winner counts without any measure of seed-level variance. The authors should report the spread across seeds, bootstrap confidence intervals, or a paired significance test for the headline comparisons; otherwise findings 3 and 4, which hinge on small differences, are not robustly supported.
  4. [§4.1, Figure 2 and §4.2, Eq. (3)] The four-case taxonomy is presented with hand-selected examples and no operational rule for assigning a model–language pair to a case, and it is derived from the same data used to illustrate it. The breakdown definition in Eq. (3) depends on a user-chosen threshold Δbreak = 30, and Appendix C.2 shows that counts change materially with the threshold (e.g., OLMo-3-7B-Think has 0/8 TC breakdowns at all thresholds, while its AC count varies from 6/8 to 2/8). The paper should specify a reproducible classification procedure for the four cases and report the number of model–language pairs in each case; otherwise the taxonomy is a descriptive organizing device rather than an empirical result.
minor comments (6)
  1. [Abstract and §1] The phrase 'thelan guage' appears in the first line of the introduction, and Section 1 contains an incomplete sentence listing 'identify various language consistency behaviors,language-consistency breakdown effect, prompt control'; these should be fixed.
  2. [Figure 2] The header row of the table in Figure 2 (a) contains 'Model Accuracy Input Consistency Output Consistency' with empty cells under 'Accuracy' and 'Input Consistency'; the figure caption does not explain these columns, and the four case names in Figure 2 (b) are not defined until Section 4.1.
  3. [Tables 2, 6, 7, and 8] The column header 'W-A-KV Bits' is not defined anywhere in the text; the paper should state that this denotes weight-activation-key/value bit widths.
  4. [Table 8 caption] The caption says 'Table 3 reports tolerance-based voting results for ϵ = 1.0,' but Table 3 covers ϵ ∈ {0.5, 1.0, 2.0}; the reference should be corrected.
  5. [§5.1] The phrase 'as showed in Table 2' is ungrammatical, and the sentence 'This gives the overall accuracy-preservation order' should read 'This gives the overall accuracy-preservation ordering'.
  6. [§4.3 and §7] The Limitations section lists benchmark, language, model, and prompt coverage but does not mention the two main threats identified above: detector validity and the lack of matched difficulty-level items; adding these limitations would make the scope of the claims clearer.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: TC/AC are measured with external PolyMath scripts and the breakdown threshold is an explicit, sensitivity-checked operationalization.

full rationale

The paper's load-bearing measurements are not circular. TC and AC are computed with the official PolyMath language-detection scripts, which are external to this paper and not fitted here, so the central consistency scores rest on an independent benchmark. The language-consistency breakdown effect is explicitly defined by a user-set threshold (Equations 3 and 4), with sensitivity analysis over thresholds in Appendix C.2; it is a transparent labeling of observed drops rather than a hidden fit or a prediction derived from the same quantity. The four-case taxonomy is a descriptive classification of measured traces, not an independent claim derived from the taxonomy itself. The accuracy-increase observation at higher difficulty is an empirical co-occurrence reported with concrete numbers (e.g., Bengali 47.5% to 50.9% while TC falls 97.9% to 1.6%); the phrase 'due to' is an interpretive statement, not a mathematical equivalence. The quantization conclusions use a defined tolerance-based voting rule with epsilon sensitivity reported, so no fitted parameter is renamed as a prediction. The only self-citation, Marchisio et al. 2024, appears in related-work context and is not load-bearing for any conclusion. No uniqueness theorem, ansatz, or known result is imported by self-citation to force the paper's choices. Correctness concerns about language-detector accuracy or threshold-induced abruptness are measurement-validity risks, not circularity, and are explicitly acknowledged in the Limitations section.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims are empirical and do not rest on mathematical axioms. The main unpaid assumptions are the reliability of the language detector, the split of reasoning versus answer text, and the comparability of difficulty levels across languages. The analysis thresholds are hand-chosen; sensitivity checks are reported but the choices remain.

free parameters (3)
  • Breakdown threshold Delta break = 30 percentage points
    User-defined drop threshold in Eqs. 3 and 4; sensitivity for 50 and 70 percentage points is reported in Table 4.
  • Consistency threshold tau_cons = 90%
    Used in Tables 1 and 5 to count languages below threshold at Top difficulty; chosen by hand, not fitted.
  • Tolerance epsilon for TC voting = 1.0 percentage point
    Used in tolerance-based voting to rank quantization methods by TC; sensitivity for 0.5, 1.0, and 2.0 is reported in Table 3.
assumptions (3)
  • domain assumption PolyMath's official language-detection scripts identify the language of reasoning traces and final answers correctly in all eight languages.
    TC and AC are computed as detected language equals requested language; the paper provides no validation of the detector on these models or outputs.
  • domain assumption Thinking and answer portions can be reliably split using the </think> delimiter when available.
    Section 3 post-processing splits responses on </think>; non-reasoning models have no thinking trace, and malformed outputs would affect TC.
  • domain assumption The four PolyMath difficulty levels are comparable across the eight languages, so accuracy differences between levels reflect model behavior rather than translation or sampling artifacts.
    The non-monotonic accuracy finding in Section 4.3 compares accuracy on disjoint problem sets at different difficulty levels; benchmark composition could explain some rises.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hidden Language Consistency Phenomena in Reasoning LLMs." pith.science (2026). https://pith.science/paper/KNULEB4U

@misc{pith2026260808447,
  author       = {Pith},
  title        = {Pith review of: Hidden Language Consistency Phenomena in Reasoning LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KNULEB4U}},
  note         = {Machine review of arXiv:2608.08447}
}
read the original abstract

Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended language while reasoning and responding. This omission conceals important multilingual behaviors that emerge as tasks become harder. In this paper, we study task difficulty, task accuracy, thinking-language consistency (TC), and answer-language consistency (AC) across reasoning models using PolyMath benchmark in eight languages and four difficulty levels. We uncover four findings: (1) language consistency exhibits four difficulty-dependent behaviors: output-language consistency remains aligned with input, remains misaligned, degrades gradually, or collapses abruptly. (2) We identify the language consistency breakdown effect, where increasing difficulty can cause a sudden drop in output-language consistency, especially in less strongly represented and non-Latin-script languages. (3) Due to this breakdown effect, accuracy can be preserved or even improved at a harder difficulty level as the model shifts to its internal dominant language. (4) Quantization can improve or degrade output-language consistency independently of its effect on accuracy, with GPTQ and AWQ often outperforming AutoRound under tolerance-based voting with {\epsilon} = 1.0. These results show that multilingual capability cannot be characterized by accuracy alone; reliable evaluation should jointly consider task accuracy, language consistency, and task difficulty for multilingual benchmarks.

Figures

Figures reproduced from arXiv: 2608.08447 by the authors.

Figure 1
Figure 1. Difficulty-wise accuracy, thinking-language [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration and taxonomy of input–output language consistency in thinking traces. (a) An example of [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Difficulty-wise final-answer accuracy, thinking-language consistency (TC), and answer-language con [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Analogous plot to Figure [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Aggregated thinking-language consistency of Baseline and W4A16-quantized DeepSeek-R1-Distill-Qwen [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Difficulty-wise final-answer accuracy, thinking-language consistency (TC), and answer-language consis [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Difficulty-wise final-answer accuracy, thinking-language consistency (TC), and answer-language consis [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Difficulty-wise final-answer accuracy, thinking-language consistency (TC), and answer-language consis [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Difficulty-wise final-answer accuracy, thinking-language consistency (TC), and answer-language consis [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Difficulty-wise final-answer accuracy, thinking-language consistency (TC), and answer-language [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Difficulty-wise final-answer accuracy, thinking-language consistency (TC), and answer-language [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Aggregated thinking-language consistency of Baseline and W4A16-quantized OLMo-3-7B-Think across [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Example of output-language consistency: the model receives an Chinese input, reasons and answers in [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Example of output-language inconsistency: the model receives an Arabic input, reasons in English, and [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

277 extracted references · 6 canonical work pages

  1. [1]

    Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, Piero Kauffmann, Yash Lara, Caio César Teodoro Mendes, Arindam Mitra, Besmira Nushi, Dimitris Papailiopoulos, Olli Saarikivi, Shital Shah, Vaishnavi Shrivastava, and 4 others. 2025. https://arxi...

    arXiv 2025

  2. [2]

    Orevaoghene Ahia, Julia Kreutzer, and Sara Hooker. 2021. The low-resource double bind: An empirical study of pruning for low-resource machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3316--3333

    2021

  3. [4]

    Pinzhen Chen, Shaoxiong Ji, Nikolay Bogoychev, Andrey Kutuzov, Barry Haddow, and Kenneth Heafield. 2024 a . https://aclanthology.org/2024.findings-eacl.90 Monolingual or multilingual instruction tuning: Which makes a better alpaca . In Findings of the Association for Computational Linguistics: EACL 2024, pages 1347--1356, St. Julian ' s, Malta. Associatio...

    2024

  4. [6]

    Wenhua Cheng, Weiwei Zhang, Haihao Shen, Yiyang Cai, Xin He, Lv Kaokao, and Yi Liu. 2024. Optimize weight rounding via signed gradient descent for the quantization of llms. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11332--11350

    2024

  5. [7]

    Nadezhda Chirkova and Vassilina Nikoulina. 2024. https://aclanthology.org/2024.naacl-long.401 Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers...

    2024

  6. [10]

    Fahim Faisal and Antonios Anastasopoulos. 2023. https://doi.org/10.18653/v1/2023.mrl-1.12 Geographic and geopolitical biases of language models . In Proceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL), pages 139--163, Singapore. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.mrl-1.12 2023

  7. [14]

    Carolin Holtermann, Paul R \"o ttger, Timm Dill, and Anne Lauscher. 2024. Evaluating the elementary multilingual capabilities of large language models with multiq. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4476--4494

    2024

  8. [16]

    Tannon Kew, Florian Schottmann, and Rico Sennrich. 2024. Turning english-centric llms into polyglots: How much multilinguality is needed? In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13097--13124

    2024

Show all 277 references
  1. [18]

    Md Tawkat Islam Khondaker, Abdul Waheed, Muhammad Abdul-Mageed, and 1 others. 2023. Gptaraeval: A comprehensive evaluation of chatgpt on arabic nlp. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 220--247

    2023

  2. [19]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Princip...

    2023

  3. [21]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let's verify step by step. In International Conference on Learning Representations, volume 2024, pages 39578--39601

    2024

  4. [22]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of machine learning and systems, 6:87--100

    2024

  5. [26]

    Team Olmo, :, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, and...

    2026 arXiv

  6. [28]

    Luiza Pozzobon, Patrick Lewis, Sara Hooker, and Beyza Ermis. 2024. https://arxiv.org/abs/2403.03893 From one to many: Expanding the scope of toxicity mitigation in language models . Preprint, arXiv:2403.03893

    2024 arXiv

  7. [30]

    Rico Sennrich, Jannis Vamvas, and Alireza Mohammadshahi. 2024. https://aclanthology.org/2024.eacl-short.4 Mitigating hallucinations and off-target machine translation with source-contrastive and language-contrastive decoding . In Proceedings of the 18th Conference of the Europ...

    2024

  8. [31]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, ...

    2025 arXiv

  9. [32]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Feng Tang, Flood Sung, Guangda Wei, Guokun Lai, and 75 others. 2025 b . https://api.s...

    2025 arXiv

  10. [33]

    NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prang...

    2022 arXiv

  11. [35]

    Qwen Team. 2025 b . https://qwenlm.github.io/blog/qwq-32b/ Qwq-32b: Embracing the power of reinforcement learning

    2025

  12. [37]

    Yiming Wang, Pei Zhang, Jialong Tang, Hao-Ran Wei, Baosong Yang, Rui Wang, Chenshu Sun, Feitong Sun, Jiran Zhang, Junxuan Wu, and 1 others. 2026. Polymath: Evaluating mathematical reasoning in multilingual contexts. Advances in Neural Information Processing Systems, 38

    2026

  13. [38]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  14. [41]

    Zheng Xin Yong, Ruochen Zhang, Jessica Forde, Skyler Wang, Arjun Subramonian, Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Lintang Sutawika, Jan Christian Blaise Cruz, Yin Lin Tan, Long Phan, Long Phan, Rowena Garcia, Thamar Solorio, and Alham Aji. 2023. https://doi.org/10....

    2023 doi

  15. [43]

    QwQ-32B: Embracing the Power of Reinforcement Learning , url =

    Qwen Team , month =. QwQ-32B: Embracing the Power of Reinforcement Learning , url =

  16. [44]

    Understanding and Mitigating Language Confusion in LLM s

    Marchisio, Kelly and Ko, Wei-Yin and Berard, Alexandre and Dehaze, Th \'e o and Ruder, Sebastian. Understanding and Mitigating Language Confusion in LLM s. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.380

    2024 doi

  17. [45]

    arXiv preprint arXiv:2412.15115 , year=

    Qwen2.5 Technical Report , author=. arXiv preprint arXiv:2412.15115 , year=

    arXiv

  18. [46]

    2025 , howpublished =

    FP8 W8A8 Quantization , author =. 2025 , howpublished =

    2025

  19. [47]

    arXiv preprint arXiv:2509.25149 , year=

    Pretraining large language models with nvfp4 , author=. arXiv preprint arXiv:2509.25149 , year=

  20. [48]

    arXiv preprint arXiv:2601.18306 , year=

    Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLM , author=. arXiv preprint arXiv:2601.18306 , year=

  21. [49]

    2024 , eprint=

    How Does Quantization Affect Multilingual LLMs? , author=. 2024 , eprint=

    2024

  22. [50]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    Optimize weight rounding via signed gradient descent for the quantization of llms , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  23. [51]

    2026 , eprint=

    Olmo 3 , author=. 2026 , eprint=

    2026

  24. [52]

    arXiv preprint arXiv:2504.04823 , year=

    Quantization hurts reasoning? an empirical study on quantized reasoning models , author=. arXiv preprint arXiv:2504.04823 , year=

    arXiv

  25. [53]

    arXiv preprint arXiv:2402.06619 , year=

    Aya dataset: An open-access collection for multilingual instruction tuning , author=. arXiv preprint arXiv:2402.06619 , year=

    arXiv

  26. [54]

    arXiv preprint arXiv:2009.03300 , year=

    Measuring massive multitask language understanding , author=. arXiv preprint arXiv:2009.03300 , year=

    2009 arXiv

  27. [55]

    arXiv preprint arXiv:2210.03057 , year=

    Language models are multilingual chain-of-thought reasoners , author=. arXiv preprint arXiv:2210.03057 , year=

    arXiv

  28. [56]

    2022 , eprint=

    No Language Left Behind: Scaling Human-Centered Machine Translation , author=. 2022 , eprint=

    2022

  29. [57]

    The 61st Annual Meeting Of The Association For Computational Linguistics , year=

    Crosslingual Generalization through Multitask Finetuning , author=. The 61st Annual Meeting Of The Association For Computational Linguistics , year=

  30. [58]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , publisher =

    XCOPA: A multilingual dataset for causal commonsense reasoning , author =. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , publisher =. doi:10.18653/v1/2020.emnlp-main.185 , url =

    2020 doi

  31. [59]

    Few-shot Learning with Multilingual Generative Language Models

    Lin, Xi Victoria and Mihaylov, Todor and Artetxe, Mikel and Wang, Tianlu and Chen, Shuohui and Simig, Daniel and Ott, Myle and Goyal, Naman and Bhosale, Shruti and Du, Jingfei and Pasunuru, Ramakanth and Shleifer, Sam and Koura, Punit Singh and Chaudhary, Vishrav and O ' Horo,...

    2022

  32. [60]

    doi:10.5281/zenodo.10256836 , url =

    A framework for few-shot language model evaluation , author =. doi:10.5281/zenodo.10256836 , url =

    doi

  33. [61]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    XNLI: Evaluating Cross-lingual Sentence Representations , author=. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    2018

  34. [62]

    Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study

    Liu, Peiyu and Liu, Zikang and Gao, Ze-Feng and Gao, Dawei and Zhao, Wayne Xin and Li, Yaliang and Ding, Bolin and Wen, Ji-Rong. Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study. Proceedings of the 2024 Joint International Conference on Comput...

    2024

  35. [63]

    arXiv preprint arXiv:2404.14294 , year=

    A survey on efficient inference for large language models , author=. arXiv preprint arXiv:2404.14294 , year=

    arXiv

  36. [65]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  37. [66]

    Advances in Neural Information Processing Systems , volume=

    Quantizable transformers: Removing outliers by helping attention heads do nothing , author=. Advances in Neural Information Processing Systems , volume=

  38. [67]

    Too Brittle to Touch: Comparing the Stability of Quantization and Distillation towards Developing Low-Resource MT Models

    Diddee, Harshita and Dandapat, Sandipan and Choudhury, Monojit and Ganu, Tanuja and Bali, Kalika. Too Brittle to Touch: Comparing the Stability of Quantization and Distillation towards Developing Low-Resource MT Models. Proceedings of the Seventh Conference on Machine Translat...

    2022

  39. [68]

    A Comparative Study on the Impact of Model Compression Techniques on Fairness in Language Models

    Ramesh, Krithika and Chavan, Arnav and Pandit, Shrey and Sitaram, Sunayana. A Comparative Study on the Impact of Model Compression Techniques on Fairness in Language Models. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...

    2023 doi

  40. [69]

    ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Distill-Quantize-Tune-Leveraging Large Teachers for Low-Footprint Efficient Multilingual NLU on Edge , author=. ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2023 , organization=

    2023

  41. [70]

    Intriguing Properties of Compression on Multilingual Models

    Ogueji, Kelechi and Ahia, Orevaoghene and Onilude, Gbemileke and Gehrmann, Sebastian and Hooker, Sara and Kreutzer, Julia. Intriguing Properties of Compression on Multilingual Models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022....

    2022 doi

  42. [71]

    Findings of the Association for Computational Linguistics: EMNLP 2021 , pages=

    The low-resource double bind: An empirical study of pruning for low-resource machine translation , author=. Findings of the Association for Computational Linguistics: EMNLP 2021 , pages=

    2021

  43. [72]

    arXiv: Learning , year=

    What Do Compressed Deep Neural Networks Forget , author=. arXiv: Learning , year=

  44. [73]

    Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback

    Lai, Viet and Nguyen, Chien and Ngo, Nghia and Nguyen, Thuat and Dernoncourt, Franck and Rossi, Ryan and Nguyen, Thien. Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback. Proceedings of the 2023 Conference on E...

    2023 doi

  45. [74]

    A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies

    Do. A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pape...

    2021 doi

  46. [75]

    GLUEC o S : An Evaluation Benchmark for Code-Switched NLP

    Khanuja, Simran and Dandapat, Sandipan and Srinivasan, Anirudh and Sitaram, Sunayana and Choudhury, Monojit. GLUEC o S : An Evaluation Benchmark for Code-Switched NLP. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v...

    2020 doi

  47. [76]

    Multilingual Large Language Models Are Not (Yet) Code-Switchers

    Zhang, Ruochen and Cahyawijaya, Samuel and Cruz, Jan Christian Blaise and Winata, Genta and Aji, Alham. Multilingual Large Language Models Are Not (Yet) Code-Switchers. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v...

    2023 doi

  48. [77]

    The Decades Progress on Code-Switching Research in NLP : A Systematic Survey on Trends and Challenges

    Winata, Genta and Aji, Alham Fikri and Yong, Zheng Xin and Solorio, Thamar. The Decades Progress on Code-Switching Research in NLP : A Systematic Survey on Trends and Challenges. Findings of the Association for Computational Linguistics: ACL 2023. 2023. doi:10.18653/v1/2023.fi...

    2023 doi

  49. [78]

    Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South E ast A sian Languages

    Yong, Zheng Xin and Zhang, Ruochen and Forde, Jessica and Wang, Skyler and Subramonian, Arjun and Lovenia, Holy and Cahyawijaya, Samuel and Winata, Genta and Sutawika, Lintang and Cruz, Jan Christian Blaise and Tan, Yin Lin and Phan, Long and Phan, Long and Garcia, Rowena and ...

    2023

  50. [79]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    arXiv

  51. [80]

    International Conference on Learning Representations , year=

    The Curious Case of Neural Text Degeneration , author=. International Conference on Learning Representations , year=

  52. [81]

    arXiv preprint arXiv:2107.03374 , year=

    Evaluating large language models trained on code , author=. arXiv preprint arXiv:2107.03374 , year=

    arXiv

  53. [82]

    arXiv preprint arXiv:2307.10169 , year=

    Challenges and applications of large language models , author=. arXiv preprint arXiv:2307.10169 , year=

    arXiv

  54. [83]

    The State and Fate of Linguistic Diversity and Inclusion in the NLP World

    Joshi, Pratik and Santy, Sebastin and Budhiraja, Amar and Bali, Kalika and Choudhury, Monojit. The State and Fate of Linguistic Diversity and Inclusion in the NLP World. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653...

    2020 doi

  55. [84]

    International Conference on Machine Learning , pages=

    Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation , author=. International Conference on Machine Learning , pages=. 2020 , organization=

    2020

  56. [85]

    Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

    Ahia, Orevaoghene and Kumar, Sachin and Gonen, Hila and Kasai, Jungo and Mortensen, David and Smith, Noah and Tsvetkov, Yulia. Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models. Proceedings of the 2023 Conference on Empirical Methods in Natu...

    2023 doi

  57. [86]

    Utility is in the Eye of the User: A Critique of NLP Leaderboards

    Ethayarajh, Kawin and Jurafsky, Dan. Utility is in the Eye of the User: A Critique of NLP Leaderboards. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp-main.393

    2020 doi

  58. [87]

    arXiv preprint arXiv:2305.14857 , year=

    Buffet: Benchmarking large language models for few-shot cross-lingual transfer , author=. arXiv preprint arXiv:2305.14857 , year=

    arXiv

  59. [88]

    Wilson & Sperber (Eds.), Meaning and Relevance , pages=

    Linguistic form and relevance , author=. Wilson & Sperber (Eds.), Meaning and Relevance , pages=

  60. [89]

    and Xu, Yan and Fung, Pascale

    Bang, Yejin and Cahyawijaya, Samuel and Lee, Nayeon and Dai, Wenliang and Su, Dan and Wilie, Bryan and Lovenia, Holy and Ji, Ziwei and Yu, Tiezheng and Chung, Willy and Do, Quyet V. and Xu, Yan and Fung, Pascale. A Multitask, Multilingual, Multimodal Evaluation of C hat GPT on...

    2023

  61. [90]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Towards mitigating LLM hallucination via self reflection , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  62. [91]

    Proceedings of ICLR 2024 , year=

    Human Feedback is not Gold Standard , author=. Proceedings of ICLR 2024 , year=

    2024

  63. [92]

    Speech acts , pages=

    Logic and conversation , author=. Speech acts , pages=. 1975 , publisher=

    1975

  64. [93]

    Proceedings of ICLR 2024 , year=

    The False Promise of Imitating Proprietary Language Models , author=. Proceedings of ICLR 2024 , year=

    2024

  65. [94]

    Internet in

    Kantar and IAMAI , year =. Internet in

  66. [95]

    Company Blog of Databricks , url=

    Free dolly: Introducing the world’s first truly open instruction-tuned llm , author=. Company Blog of Databricks , url=

  67. [96]

    Stanford Center for Research on Foundation Models , url=

    Alpaca: A strong, replicable instruction-following model , author=. Stanford Center for Research on Foundation Models , url=

  68. [97]

    arXiv preprint arXiv:1607.01759 , year=

    Bag of tricks for efficient text classification , author=. arXiv preprint arXiv:1607.01759 , year=

    arXiv

  69. [98]

    mm T 5: Modular Multilingual Pre-Training Solves Source Language Hallucinations

    Pfeiffer, Jonas and Piccinno, Francesco and Nicosia, Massimo and Wang, Xinyi and Reid, Machel and Ruder, Sebastian. mm T 5: Modular Multilingual Pre-Training Solves Source Language Hallucinations. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023. doi...

    2023 doi

  70. [99]

    Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation

    Vu, Tu and Barua, Aditya and Lester, Brian and Cer, Daniel and Iyyer, Mohit and Constant, Noah. Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/...

    2022 doi

  71. [100]

    Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution

    Li, Tianjian and Murray, Kenton. Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution. Findings of the Association for Computational Linguistics: ACL 2023. 2023. doi:10.18653/v1/2023.findings-acl.789

    2023 doi

  72. [101]

    2024 , eprint=

    Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model , author=. 2024 , eprint=

    2024

  73. [102]

    Advances in neural information processing systems , volume=

    Qlora: Efficient finetuning of quantized llms , author=. Advances in neural information processing systems , volume=

  74. [103]

    The Eleventh International Conference on Learning Representations , year=

    Language models are multilingual chain-of-thought reasoners , author=. The Eleventh International Conference on Learning Representations , year=

  75. [104]

    2024 , journal=

    Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning , author=. 2024 , journal=

    2024

  76. [105]

    2023 , eprint=

    Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models , author=. 2023 , eprint=

    2023

  77. [106]

    2024 , eprint=

    Gemini: A Family of Highly Capable Multimodal Models , author=. 2024 , eprint=

    2024

  78. [107]

    Gemma: Open Models Based on Gemini Research and Technology , author=

  79. [108]

    2023 , eprint=

    Mistral 7B , author=. 2023 , eprint=

    2023

  80. [109]

    2023 , eprint=

    The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants , author=. 2023 , eprint=

    2023

  81. [110]

    arXiv preprint arXiv:2403.08295 , year=

    Gemma: Open models based on gemini research and technology , author=. arXiv preprint arXiv:2403.08295 , year=

    arXiv

  82. [111]

    2021 , eprint=

    MuRIL: Multilingual Representations for Indian Languages , author=. 2021 , eprint=

    2021

  83. [112]

    Better Quality Pre-training Data and T5 Models for A frican Languages

    Oladipo, Akintunde and Adeyemi, Mofetoluwa and Ahia, Orevaoghene and Owodunni, Abraham and Ogundepo, Odunayo and Adelani, David and Lin, Jimmy. Better Quality Pre-training Data and T5 Models for A frican Languages. Proceedings of the 2023 Conference on Empirical Methods in Nat...

    2023

  84. [113]

    and Hassid, Michael and Heafield, Kenneth and Hooker, Sara and Raffel, Colin and Martins, Pedro H

    Treviso, Marcos and Lee, Ji-Ung and Ji, Tianchu and Aken, Betty van and Cao, Qingqing and Ciosici, Manuel R. and Hassid, Michael and Heafield, Kenneth and Hooker, Sara and Raffel, Colin and Martins, Pedro H. and Martins, André F. T. and Forde, Jessica Zosa and Milder, Peter an...

    2023

  85. [114]

    2019 , eprint=

    The State of Sparsity in Deep Neural Networks , author=. 2019 , eprint=

    2019

  86. [115]

    Three pathways to better recognize the expertise of Global South researchers , journal =

    Nakamura, Gabriel and Soares, Bruno and Pillar, Valério and Diniz-Filho, José and Duarte, Leandro , year =. Three pathways to better recognize the expertise of Global South researchers , journal =

  87. [116]

    Nature , year=

    Papers and patents are becoming less disruptive over time , author=. Nature , year=

  88. [117]

    PanGu- : Large-scale autoregressive pretrained Chinese language models with auto-parallel computation , author =

  89. [118]

    arXiv preprint arXiv:2206.04615 , year=

    Beyond the imitation game: Quantifying and extrapolating the capabilities of language models , author=. arXiv preprint arXiv:2206.04615 , year=

    arXiv

  90. [119]

    2023 , eprint=

    Deep Learning on a Data Diet: Finding Important Examples Early in Training , author=. 2023 , eprint=

    2023

  91. [120]

    2019 , eprint=

    CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data , author=. 2019 , eprint=

    2019

  92. [121]

    2024 , eprint=

    Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca , author=. 2024 , eprint=

    2024

  93. [122]

    arXiv preprint arXiv:2212.04960 , year=

    BigScience: A case study in the social construction of a multilingual large language model , author=. arXiv preprint arXiv:2212.04960 , year=

    arXiv

  94. [123]

    Advances in Neural Information Processing Systems , volume=

    The bigscience roots corpus: A 1.6 tb composite multilingual dataset , author=. Advances in Neural Information Processing Systems , volume=

  95. [124]

    arXiv preprint arXiv:2211.05100 , year=

    Bloom: A 176b-parameter open-access multilingual language model , author=. arXiv preprint arXiv:2211.05100 , year=

    arXiv

  96. [125]

    and Uszkoreit, Jakob and Le, Quoc and Petrov, Slav

    Kwiatkowski, Tom and Palomaki, Jennimaria and Redfield, Olivia and Collins, Michael and Parikh, Ankur and Alberti, Chris and Epstein, Danielle and Polosukhin, Illia and Devlin, Jacob and Lee, Kenton and Toutanova, Kristina and Jones, Llion and Kelcey, Matthew and Chang, Ming-W...

    2019 doi

  97. [126]

    Elias Frantar and Dan Alistarh , year=

  98. [127]

    arXiv preprint arXiv:2210.17323 , year=

    Gptq: Accurate post-training quantization for generative pre-trained transformers , author=. arXiv preprint arXiv:2210.17323 , year=

    arXiv

  99. [128]

    arXiv preprint arXiv:1907.05019 , year=

    Massively multilingual neural machine translation in the wild: Findings and challenges , author=. arXiv preprint arXiv:1907.05019 , year=

    1907 arXiv

  100. [129]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Intriguing Properties of Quantization at Scale , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  101. [130]

    int8 (): 8-bit matrix multiplication for transformers at scale , author=

    Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale , author=. Advances in neural information processing systems , volume=

  102. [131]

    arXiv preprint arXiv:2104.00772 , year=

    Low-resource language modelling of South African languages , author=. arXiv preprint arXiv:2104.00772 , year=

    arXiv

  103. [132]

    Hugging Face Repository , year=

    Merak-7B: The LLM for Bahasa Indonesia , author=. Hugging Face Repository , year=

  104. [133]

    2024 , eprint=

    MaLLaM -- Malaysia Large Language Model , author=. 2024 , eprint=

    2024

  105. [134]

    Dat Quoc Nguyen and Linh The Nguyen and Chi Tran and Dung Ngoc Nguyen and Dinh Phung and Hung Bui , journal =

  106. [135]

    2023 , howpublished=

    SEA-LION (Southeast Asian Languages In One Network): A Family of Large Language Models for Southeast Asia , author=. 2023 , howpublished=

    2023

  107. [136]

    2023 , publisher =

    Edward Beeching and Clémentine Fourrier and Nathan Habib and Sheon Han and Nathan Lambert and Nazneen Rajani and Omar Sanseviero and Lewis Tunstall and Thomas Wolf , title =. 2023 , publisher =

    2023

  108. [138]

    Ethnologue , howpublished =

  109. [139]

    CoRR , volume =

    Noam Shazeer , title =. CoRR , volume =. 2020 , url =. 2002.05202 , timestamp =

    2020 arXiv

  110. [140]

    2022 , eprint=

    Scalable Training of Language Models using JAX pjit and TPUv4 , author=. 2022 , eprint=

    2022

  111. [141]

    2020 , eprint=

    Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism , author=. 2020 , eprint=

    2020

  112. [142]

    Smith and Mike Lewis , title =

    Ofir Press and Noah A. Smith and Mike Lewis , title =. CoRR , volume =. 2021 , url =. 2108.12409 , timestamp =

    2021 arXiv

  113. [143]

    CoRR , volume =

    Jianlin Su and Yu Lu and Shengfeng Pan and Bo Wen and Yunfeng Liu , title =. CoRR , volume =. 2021 , url =. 2104.09864 , timestamp =

    2021 arXiv

  114. [144]

    2024 , eprint=

    Aya 23: Open Weight Releases to Further Multilingual Progress , author=. 2024 , eprint=

    2024

  115. [145]

    2023 , eprint=

    TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings , author=. 2023 , eprint=

    2023

  116. [146]

    2023 , eprint=

    GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints , author=. 2023 , eprint=

    2023

  117. [147]

    2024 , eprint=

    From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models , author=. 2024 , eprint=

    2024

  118. [148]

    2020 , eprint=

    Characterising Bias in Compressed Models , author=. 2020 , eprint=

    2020

  119. [149]

    arXiv preprint arXiv:2401.04088 , year=

    Mixtral of experts , author=. arXiv preprint arXiv:2401.04088 , year=

    arXiv

  120. [150]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  121. [151]

    Intriguing properties of quantization at scale , year =

    Ahmadian, Arash and Dash, Saurabh and Chen, Hongyu and Venkitesh, Bharat and Gou, Stephen and Blunsom, Phil and \". Intriguing properties of quantization at scale , year =. Proceedings of the 37th International Conference on Neural Information Processing Systems , articleno =

  122. [152]

    2024 , eprint=

    Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? , author=. 2024 , eprint=

    2024

  123. [153]

    Elias Frantar and Saleh Ashkboos and Torsten Hoefler and Dan Alistarh , year=

  124. [154]

    Proceedings of machine learning and systems , volume=

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration , author=. Proceedings of machine learning and systems , volume=

  125. [155]

    arXiv e-prints , pages=

    Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback , author=. arXiv e-prints , pages=

  126. [156]

    2023 , eprint=

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model , author=. 2023 , eprint=

    2023

  127. [157]

    Advances in Neural Information Processing Systems , volume=

    Alpacafarm: A simulation framework for methods that learn from human feedback , author=. Advances in Neural Information Processing Systems , volume=

  128. [158]

    Hashimoto , title =

    Xuechen Li and Tianyi Zhang and Yann Dubois and Rohan Taori and Ishaan Gulrajani and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto , title =. GitHub repository , howpublished =. 2023 , publisher =

    2023

  129. [159]

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric and others , journal=

  130. [160]

    Proceedings of The ACM Collective Intelligence Conference , url =

    Gender bias and stereotypes in Large Language Models , author =. Proceedings of The ACM Collective Intelligence Conference , url =

  131. [161]

    ArXiv , volume =

    Casteist but Not Racist? Quantifying Disparities in Large Language Model Bias between India and the West , author =. ArXiv , volume =

  132. [162]

    On Evaluating and Mitigating Gender Biases in Multilingual Settings , author =

  133. [163]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    GPTAraEval: A comprehensive evaluation of ChatGPT on Arabic NLP , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  134. [164]

    Bach , year = 2023, volume =

    Zheng-Xin Yong and Cristina Menghini and Stephen H. Bach , year = 2023, volume =. Low-Resource Languages Jailbreak

    2023

  135. [165]

    Scalable Extraction of Training Data from (Production) Language Models , author =

  136. [166]

    ArXiv , volume =

    Privacy in Large Language Models: Attacks, Defenses and Future Directions , author =. ArXiv , volume =

  137. [167]

    2023 IEEE Symposium on Security and Privacy (SP) , pages =

    Analyzing Leakage of Personally Identifiable Information in Language Models , author =. 2023 IEEE Symposium on Security and Privacy (SP) , pages =

    2023

  138. [168]

    Multilingual jailbreak challenges in large language models , author =

  139. [169]

    A Material Lens on Coloniality in NLP , author =

  140. [170]

    Towards Measuring the Representation of Subjective Global Opinions in Language Models , author =

  141. [171]

    Lost in Translation: Large Language Models in Non-English Content Analysis , author =

  142. [172]

    How good are Large Language Models on African Languages? , author =

  143. [173]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    Measuring Massive Multitask Language Understanding , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

  144. [174]

    2021 , eprint=

    Randomness In Neural Network Training: Characterizing The Impact of Tooling , author=. 2021 , eprint=

    2021

  145. [175]

    2023 , eprint=

    On The Fairness Impacts of Hardware Selection in Machine Learning , author=. 2023 , eprint=

    2023

  146. [176]

    2023 , eprint=

    FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling , author=. 2023 , eprint=

    2023

  147. [177]

    2019 , eprint=

    Differential Privacy Has Disparate Impact on Model Accuracy , author=. 2019 , eprint=

    2019

  148. [178]

    2022 , eprint=

    Robust Distillation for Worst-class Performance , author=. 2022 , eprint=

    2022

  149. [179]

    A Call for Clarity in Reporting BLEU Scores

    Post, Matt. A Call for Clarity in Reporting BLEU Scores. Proceedings of the Third Conference on Machine Translation: Research Papers. 2018. doi:10.18653/v1/W18-6319

    2018 doi

  150. [180]

    The F lores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation

    Goyal, Naman and Gao, Cynthia and Chaudhary, Vishrav and Chen, Peng-Jen and Wenzek, Guillaume and Ju, Da and Krishnan, Sanjana and Ranzato, Marc ' Aurelio and Guzm \'a n, Francisco and Fan, Angela. The F lores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine ...

    2022 doi

  151. [181]

    B leu: a Method for Automatic Evaluation of Machine Translation

    Papineni, Kishore and Roukos, Salim and Ward, Todd and Zhu, Wei-Jing. B leu: a Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 2002. doi:10.3115/1073083.1073135

    2002

  152. [182]

    2024 , eprint=

    Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs , author=. 2024 , eprint=

    2024

  153. [183]

    Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM , author =

  154. [184]

    The Twelfth International Conference on Learning Representations , year=

    Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models , author=. The Twelfth International Conference on Learning Representations , year=

  155. [185]

    2024 , eprint=

    Understanding and Mitigating Language Confusion in LLMs , author=. 2024 , eprint=

    2024

  156. [186]

    m T 5: A Massively Multilingual Pre-trained Text-to-Text Transformer

    Xue, Linting and Constant, Noah and Roberts, Adam and Kale, Mihir and Al-Rfou, Rami and Siddhant, Aditya and Barua, Aditya and Raffel, Colin. m T 5: A Massively Multilingual Pre-trained Text-to-Text Transformer. Proceedings of the 2021 Conference of the North American Chapter ...

    2021 doi

  157. [187]

    arXiv preprint arXiv:2405.03146 , year=

    Quantifying the Capabilities of LLMs across Scale and Precision , author=. arXiv preprint arXiv:2405.03146 , year=

    arXiv

  158. [188]

    International Conference on Machine Learning , pages=

    The case for 4-bit precision: k-bit inference scaling laws , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  159. [189]

    2024 , eprint=

    An Empirical Study of LLaMA3 Quantization: From LLMs to MLLMs , author=. 2024 , eprint=

    2024

  160. [190]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  161. [191]

    Publications Manual , year = "1983", publisher =

    1983

  162. [192]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    1981

  163. [193]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  164. [194]

    Dan Gusfield , title =. 1997

    1997

  165. [195]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    s1: Simple test-time scaling , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  166. [196]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  167. [197]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  168. [198]

    International conference on machine learning , pages=

    Smoothquant: Accurate and efficient post-training quantization for large language models , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  169. [199]

    Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling , author=. 2023

    2023

  170. [200]

    arXiv preprint arXiv:2308.13137 , year=

    Omniquant: Omnidirectionally calibrated quantization for large language models , author=. arXiv preprint arXiv:2308.13137 , year=

    arXiv

  171. [201]

    arXiv preprint arXiv:2404.00456 , year=

    Quarot: Outlier-free 4-bit inference in rotated llms , author=. arXiv preprint arXiv:2404.00456 , year=

    arXiv

  172. [202]

    arXiv preprint arXiv:2405.04532 , year=

    QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving , author=. arXiv preprint arXiv:2405.04532 , year=

    arXiv

  173. [203]

    arXiv preprint arXiv:2405.16406 , year=

    SpinQuant--LLM quantization with learned rotations , author=. arXiv preprint arXiv:2405.16406 , year=

    arXiv

  174. [204]

    arXiv preprint arXiv:2401.15024 , year=

    Slicegpt: Compress large language models by deleting rows and columns , author=. arXiv preprint arXiv:2401.15024 , year=

    arXiv

  175. [205]

    2024 , eprint=

    Transformers need glasses! Information over-squashing in language tasks , author=. 2024 , eprint=

    2024

  176. [206]

    arXiv preprint arXiv:2302.13971 , year=

    Llama: Open and efficient foundation language models , author=. arXiv preprint arXiv:2302.13971 , year=

    arXiv

  177. [207]

    2024 , url=

    Llama 3 Model Card , author=. 2024 , url=

    2024

  178. [208]

    International Conference on Learning Representations , year=

    Pointer Sentinel Mixture Models , author=. International Conference on Learning Representations , year=

  179. [209]

    The Journal of Machine Learning Research , volume=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. The Journal of Machine Learning Research , volume=. 2020 , publisher=

    2020

  180. [210]

    arXiv preprint arXiv:2402.17762 , year=

    Massive Activations in Large Language Models , author=. arXiv preprint arXiv:2402.17762 , year=

    arXiv

  181. [211]

    arXiv preprint arXiv:2403.01241 , year=

    Intactkv: Improving large language model quantization by keeping pivot tokens intact , author=. arXiv preprint arXiv:2403.01241 , year=

    arXiv

  182. [212]

    arXiv preprint arXiv:2404.14047 , year=

    How Good Are Low-bit Quantized LLaMA3 Models? An Empirical Study , author=. arXiv preprint arXiv:2404.14047 , year=

    arXiv

  183. [213]

    Advances in Neural Information Processing Systems , volume=

    Training transformers with 4-bit integers , author=. Advances in Neural Information Processing Systems , volume=

  184. [214]

    arXiv preprint arXiv:2404.03605 , year=

    Mitigating the impact of outlier channels for language model quantization with activation regularization , author=. arXiv preprint arXiv:2404.03605 , year=

    arXiv

  185. [215]

    Advances in neural information processing systems , volume=

    Robust quantization: One model to rule them all , author=. Advances in neural information processing systems , volume=

  186. [216]

    arXiv preprint arXiv:2002.01113 , year=

    Efficient riemannian optimization on the stiefel manifold via the cayley transform , author=. arXiv preprint arXiv:2002.01113 , year=

    2002 arXiv

  187. [217]

    Yale University, New Haven , year=

    Notes on optimization on stiefel manifolds , author=. Yale University, New Haven , year=

  188. [218]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Piqa: Reasoning about physical commonsense in natural language , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  189. [219]

    arXiv preprint arXiv:1803.05457 , year=

    Think you have solved question answering? try arc, the ai2 reasoning challenge , author=. arXiv preprint arXiv:1803.05457 , year=

    arXiv

  190. [220]

    Communications of the ACM , volume=

    Winogrande: An adversarial winograd schema challenge at scale , author=. Communications of the ACM , volume=. 2021 , publisher=

    2021

  191. [221]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    HellaSwag: Can a Machine Really Finish Your Sentence? , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

  192. [222]

    Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics , pages=

    The LAMBADA dataset: Word prediction requiring a broad discourse context , author=. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics , pages=

  193. [223]

    Proceedings of Machine Learning and Systems , volume=

    Atom: Low-bit quantization for efficient and accurate llm serving , author=. Proceedings of Machine Learning and Systems , volume=

  194. [224]

    arXiv preprint arXiv:2305.17888 , year=

    Llm-qat: Data-free quantization aware training for large language models , author=. arXiv preprint arXiv:2305.17888 , year=

    arXiv

  195. [225]

    arXiv preprint arXiv:2310.09259 , year=

    Towards end-to-end 4-bit inference on generative large language models , author=. arXiv preprint arXiv:2310.09259 , year=

    arXiv

  196. [226]

    arXiv preprint arXiv:2401.18079 , year=

    Kvquant: Towards 10 million context length llm inference with kv cache quantization , author=. arXiv preprint arXiv:2401.18079 , year=

    arXiv

  197. [228]

    arXiv preprint arXiv:2403.12544 , year=

    Affinequant: Affine transformation quantization for large language models , author=. arXiv preprint arXiv:2403.12544 , year=

    arXiv

  198. [229]

    int8 (): 8-bit matrix multiplication for transformers at scale , author=

    Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale , author=. Advances in Neural Information Processing Systems , volume=

  199. [230]

    2023 , url =

    Nvidia cutlass library , author=. 2023 , url =

    2023

  200. [231]

    2023 , url =

    FlashInfer: Kernel Library for LLM Serving , author=. 2023 , url =

    2023

  201. [232]

    arXiv preprint arXiv:2402.18158 , year=

    Evaluating quantized large language models , author=. arXiv preprint arXiv:2402.18158 , year=

    arXiv

  202. [233]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Quantization and training of neural networks for efficient integer-arithmetic-only inference , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  203. [234]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  204. [235]

    arXiv preprint arXiv:2407.10671 , year=

    Qwen2 technical report , author=. arXiv preprint arXiv:2407.10671 , year=

    arXiv

  205. [236]

    arXiv preprint arXiv:2310.06825 , year=

    Mistral 7B , author=. arXiv preprint arXiv:2310.06825 , year=

    arXiv

  206. [237]

    Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages , pages=

    Triton: an intermediate language and compiler for tiled neural network computations , author=. Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages , pages=

  207. [238]

    arXiv preprint arXiv:2402.02750 , year=

    Kivi: A tuning-free asymmetric 2bit quantization for kv cache , author=. arXiv preprint arXiv:2402.02750 , year=

    arXiv

  208. [239]

    Version v0

    A framework for few-shot language model evaluation , author=. Version v0. 0.1. Sept , volume=

  209. [240]

    arXiv preprint arXiv:1910.03771 , year=

    Huggingface's transformers: State-of-the-art natural language processing , author=. arXiv preprint arXiv:1910.03771 , year=

    1910 arXiv

  210. [241]

    Advances in neural information processing systems , volume=

    Pytorch: An imperative style, high-performance deep learning library , author=. Advances in neural information processing systems , volume=

  211. [242]

    International Conference on Machine Learning , pages=

    Up or down? adaptive rounding for post-training quantization , author=. International Conference on Machine Learning , pages=. 2020 , organization=

    2020

  212. [243]

    arXiv preprint arXiv:2102.05426 , year=

    Brecq: Pushing the limit of post-training quantization by block reconstruction , author=. arXiv preprint arXiv:2102.05426 , year=

    arXiv

  213. [244]

    Advances in Neural Information Processing Systems , volume=

    Outlier suppression: Pushing the limit of low-bit transformer language models , author=. Advances in Neural Information Processing Systems , volume=

  214. [245]

    Advances in Neural Information Processing Systems , volume=

    Quip: 2-bit quantization of large language models with guarantees , author=. Advances in Neural Information Processing Systems , volume=

  215. [246]

    arXiv preprint arXiv:2402.04396 , year=

    Quip\#: Even better LLM quantization with hadamard incoherence and lattice codebooks , author=. arXiv preprint arXiv:2402.04396 , year=

    arXiv

  216. [247]

    arXiv preprint arXiv:2405.06219 , year=

    SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models , author=. arXiv preprint arXiv:2405.06219 , year=

    arXiv

  217. [248]

    arXiv preprint arXiv:2410.09426 , year=

    Flatquant: Flatness matters for llm quantization , author=. arXiv preprint arXiv:2410.09426 , year=

    arXiv

  218. [249]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    arXiv

  219. [250]

    ArXiv , year=

    Kimi k1.5: Scaling Reinforcement Learning with LLMs , author=. ArXiv , year=

  220. [251]

    arXiv preprint arXiv:2412.19437 , year=

    Deepseek-v3 technical report , author=. arXiv preprint arXiv:2412.19437 , year=

    arXiv

  221. [252]

    arXiv preprint arXiv:2412.16720 , year=

    Openai o1 system card , author=. arXiv preprint arXiv:2412.16720 , year=

    arXiv

  222. [253]

    International Conference on Learning Representations , volume=

    Let's verify step by step , author=. International Conference on Learning Representations , volume=

  223. [254]

    arXiv preprint arXiv:2311.12022 , year=

    Gpqa: A graduate-level google-proof q&a benchmark , author=. arXiv preprint arXiv:2311.12022 , year=

    arXiv

  224. [255]

    Maxwell-Jia , title =

  225. [256]

    arXiv preprint arXiv:2403.07974 , year=

    Livecodebench: Holistic and contamination free evaluation of large language models for code , author=. arXiv preprint arXiv:2403.07974 , year=

    arXiv

  226. [257]

    arXiv preprint arXiv:2110.14168 , year=

    Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  227. [258]

    2023 , version =

    Fourrier, Clémentine and Habib, Nathan and Kydlíček, Hynek and Wolf, Thomas and Tunstall, Lewis , title =. 2023 , version =

    2023

  228. [259]

    Proceedings of the 29th Symposium on Operating Systems Principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th Symposium on Operating Systems Principles , pages=

  229. [260]

    arXiv preprint arXiv:2311.09755 , year=

    On the impact of calibration data in post-training quantization and pruning , author=. arXiv preprint arXiv:2311.09755 , year=

    arXiv

  230. [261]

    2023 , eprint=

    Microscaling Data Formats for Deep Learning , author=. 2023 , eprint=

    2023

  231. [262]

    , author=

    Visualizing data using t-SNE. , author=. Journal of machine learning research , volume=

  232. [263]

    arXiv preprint arXiv:2412.21187 , year=

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms , author=. arXiv preprint arXiv:2412.21187 , year=

    arXiv

  233. [264]

    The Twelfth International Conference on Learning Representations , year=

    Plug-and-play: An efficient post-training pruning method for large language models , author=. The Twelfth International Conference on Learning Representations , year=

  234. [265]

    International Conference on Machine Learning , pages=

    Sparsegpt: Massive language models can be accurately pruned in one-shot , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  235. [266]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  236. [267]

    Advances in Neural Information Processing Systems , volume=

    Compact language models via pruning and knowledge distillation , author=. Advances in Neural Information Processing Systems , volume=

  237. [268]

    Neurocomputing , volume=

    Roformer: Enhanced transformer with rotary position embedding , author=. Neurocomputing , volume=. 2024 , publisher=

    2024

  238. [269]

    arXiv preprint arXiv:2310.01382 , year=

    Compressing llms: The truth is rarely pure and never simple , author=. arXiv preprint arXiv:2310.01382 , year=

    arXiv

  239. [270]

    Hugging Face repository , howpublished =

    Jia LI and Edward Beeching and Lewis Tunstall and Ben Lipkin and Roman Soletskyi and Shengyi Costa Huang and Kashif Rasul and Longhui Yu and Albert Jiang and Ziju Shen and Zihan Qin and Bin Dong and Li Zhou and Yann Fleureau and Guillaume Lample and Stanislas Polu , title =. H...

    2024

  240. [271]

    arXiv preprint arXiv:2309.16609 , year=

    Qwen technical report , author=. arXiv preprint arXiv:2309.16609 , year=

    arXiv

  241. [272]

    arXiv preprint arXiv:2101.00027 , year=

    The pile: An 800gb dataset of diverse text for language modeling , author=. arXiv preprint arXiv:2101.00027 , year=

    arXiv

  242. [273]

    arXiv preprint arXiv:2503.04697 , year=

    L1: Controlling how long a reasoning model thinks with reinforcement learning , author=. arXiv preprint arXiv:2503.04697 , year=

    arXiv

  243. [274]

    2025 , eprint=

    LIMO: Less is More for Reasoning , author=. 2025 , eprint=

    2025

  244. [275]

    arXiv preprint arXiv:2306.08543 , year=

    MiniLLM: Knowledge distillation of large language models , author=. arXiv preprint arXiv:2306.08543 , year=

    arXiv

  245. [276]

    arXiv preprint arXiv:2502.03373 , year=

    Demystifying Long Chain-of-Thought Reasoning in LLMs , author=. arXiv preprint arXiv:2502.03373 , year=

    arXiv

  246. [277]

    arXiv preprint arXiv:2408.03314 , year=

    Scaling llm test-time compute optimally can be more effective than scaling model parameters , author=. arXiv preprint arXiv:2408.03314 , year=

    arXiv

  247. [278]

    arXiv preprint arXiv:2502.06703 , year=

    Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling , author=. arXiv preprint arXiv:2502.06703 , year=

    arXiv

  248. [279]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Token-budget-aware llm reasoning , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  249. [280]

    arXiv preprint arXiv:2501.12570 , year=

    O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning , author=. arXiv preprint arXiv:2501.12570 , year=

    arXiv

  250. [281]

    arXiv preprint arXiv:2501.03035 , year=

    Quantization meets reasoning: Exploring llm low-bit quantization degradation for mathematical reasoning , author=. arXiv preprint arXiv:2501.03035 , year=

    arXiv

  251. [282]

    A Simple and Effective Pruning Approach for Large Language Models , author=

  252. [283]

    2025 , note =

    Deployment-ready reasoning with quantized DeepSeek-R1 models , author =. 2025 , note =

    2025

  253. [284]

    Binarybert: Pushing the limit of bert quantization , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages=

  254. [285]

    arXiv preprint arXiv:2505.19634 , year=

    Faster and Better LLMs via Latency-Aware Test-Time Scaling , author=. arXiv preprint arXiv:2505.19634 , year=

    arXiv

  255. [286]

    arXiv preprint arXiv:2504.15466 , year=

    Learning adaptive parallel reasoning with language models , author=. arXiv preprint arXiv:2504.15466 , year=

    arXiv

  256. [287]

    arXiv preprint arXiv:2504.06261 , year=

    Hogwild! inference: Parallel llm generation via concurrent attention , author=. arXiv preprint arXiv:2504.06261 , year=

  257. [288]

    arXiv preprint arXiv:2505.24680 , year=

    A Simple Linear Patch Revives Layer-Pruned Large Language Models , author=. arXiv preprint arXiv:2505.24680 , year=

  258. [289]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    arXiv

  259. [290]

    Advances in Neural Information Processing Systems , volume=

    Polymath: Evaluating mathematical reasoning in multilingual contexts , author=. Advances in Neural Information Processing Systems , volume=

  260. [291]

    arXiv preprint arXiv:2311.08391 , year=

    A Material Lens on Coloniality in NLP , author=. arXiv preprint arXiv:2311.08391 , year=

    arXiv

  261. [292]

    Transactions of the Association for Computational Linguistics , volume=

    Hallucinations in large multilingual translation models , author=. Transactions of the Association for Computational Linguistics , volume=. 2023 , publisher=

    2023

  262. [293]

    arXiv preprint arXiv:2405.15032 , year=

    Aya 23: Open Weight Releases to Further Multilingual Progress , author=. arXiv preprint arXiv:2405.15032 , year=

    arXiv

  263. [294]

    The Bell system technical journal , volume=

    A mathematical theory of communication , author=. The Bell system technical journal , volume=. 1948 , publisher=

    1948

  264. [295]

    arXiv preprint arXiv:2402.07827 , year=

    Aya model: An instruction finetuned open-access multilingual language model , author=. arXiv preprint arXiv:2402.07827 , year=

    arXiv

  265. [296]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  266. [297]

    On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation

    Chen, Liang and Ma, Shuming and Zhang, Dongdong and Wei, Furu and Chang, Baobao. On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation. Findings of the Association for Computational Linguistics: ACL 2023. 2023. doi:10.18653/v1/2023.findings-acl.608

    2023 doi

  267. [298]

    Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding

    Sennrich, Rico and Vamvas, Jannis and Mohammadshahi, Alireza. Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding. Proceedings of the 18th Conference of the European Chapter of the Association for Computational...

    2024

  268. [299]

    Language Contamination Helps Explains the Cross-lingual Capabilities of E nglish Pretrained Models

    Blevins, Terra and Zettlemoyer, Luke. Language Contamination Helps Explains the Cross-lingual Capabilities of E nglish Pretrained Models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.233

    2022 doi

  269. [300]

    2024 , eprint=

    3D-Properties: Identifying Challenges in DPO and Charting a Path Forward , author=. 2024 , eprint=

    2024

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.