Pith. sign in

REVIEW 5 major objections 7 minor 89 references

The paper claims that GSM-PLUS-BN, a human-verified Bengali translation of the GSM-Plus perturbation benchmark, is a valid tool for testing whether large language models genuinely reason about Bengali math problems or just pattern-match.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 05:43 UTC pith:2RBLMLBY

load-bearing objection The Bengali GSM-Plus translation is a genuinely useful resource, but the paper's own claims about being first and about the English performance gap don't survive contact with its text, and the Critical Thinking category is internally inconsistent with the answer-preservation guarantee. the 5 major comments →

arxiv 2607.13248 v1 pith:2RBLMLBY submitted 2026-07-14 cs.CL

GSM-Plus-BN: A Perturbation-Based Benchmark for Bangla Mathematical Reasoning in Large Language Models

Bidyarthi Paul , Nahida Jannat Mayouree , Md. Asif Karim , Sagar Chandra Nath , Swastika Kundu This is my paper
classification cs.CL
keywords Bengali mathematical reasoningperturbation benchmarkLLM robustnesschain-of-thought promptingGSM-Pluslow-resource languageexact-match evaluationcritical thinking
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces GSM-PLUS-BN, the first perturbation-based mathematical reasoning benchmark for Bengali, built by translating the English GSM-Plus dataset and having six expert humans verify each item. It then evaluates six open-source language models under both standard prompting and chain-of-thought prompting to measure robustness across eight perturbation types. The central claim is that the dataset preserves the underlying math and correct answers while changing surface form, so the reported accuracy differences reveal genuine model fragility rather than translation artifacts. If correct, this gives the research community a reusable resource to measure Bengali math reasoning and a baseline showing that models lose 20 to 40 percentage points when problems are perturbed, with chain-of-thought helping some models but not others.

Core claim

The authors claim that GSM-PLUS-BN is a valid Bengali adaptation of GSM-Plus, comprising 10,544 instances from 1,318 seed questions across eight perturbation types: numerical substitution, digit expansion, integer-decimal-fraction conversion, adding operation, reversing operation, problem understanding, distraction insertion, and critical thinking. They report that GPT-OSS-20B reaches 96.08% accuracy on seed questions under standard prompting, that larger models such as Llama-3.3-70B and GPT-OSS-120B are more robust across perturbation types, and that chain-of-thought prompting substantially improves most models while slightly hurting others. They also show that critical thinking items, whic

What carries the argument

The central object is the GSM-PLUS-BN dataset itself: a Bengali translation of the GSM-Plus perturbation taxonomy, expanded from 1,318 seeds into 10,544 items. The load-bearing machinery is the hybrid translation pipeline (AI translation followed by six human experts applying rules to preserve reasoning chains, localize names, currency, and numerals) plus the paired evaluation protocol that compares seed questions against their eight perturbed variants under standard and zero-shot chain-of-thought prompts, scored by exact-match accuracy.

Load-bearing premise

The Bengali translations of all eight perturbation types preserve the original seed answer and the full mathematical meaning in every case, including the critical-thinking items where 'the correct answer' is a judgment about missing information.

What would settle it

Have independent Bengali-speaking mathematicians solve a random sample of GSM-PLUS-BN items without seeing the English originals, then compare their answers against the provided labels; if disagreement rates rival the reported model accuracies, the benchmark's answer-preservation claim fails. Alternatively, a back-translation audit of the Bengali items to English that changes the math would falsify the translation fidelity claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If valid, GSM-PLUS-BN provides the first standard benchmark for measuring reasoning robustness of LLMs in Bengali, enabling direct cross-lingual comparisons with the English GSM-Plus.
  • Chain-of-thought prompting helps weaker models like Qwen3-32B dramatically (average +23.85 points) but degrades Llama-3.1-8B and slightly hurts GPT-OSS-120B, so model-specific prompting selection matters in low-resource settings.
  • Critical thinking perturbations remain a universal failure mode below 34% accuracy, indicating current LLMs cannot reliably detect under-determined Bengali math problems.
  • The 20-to-40-point seed-versus-perturbed gap under standard prompting quantifies the robustness deficit that future Bengali math reasoning models should target.
  • Arithmetic perturbations like adding operation and integer-decimal-fraction conversion stay below 80% even with CoT, suggesting reliance on surface patterns rather than genuine arithmetic understanding.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the dataset truly preserves answers across all eight perturbation types, then a model that scores high on seeds but collapses on perturbations is pattern-matching; GSM-PLUS-BN could therefore serve as a direct test of 'genuine understanding' in Bengali, extending a similar argument made for English.
  • The persistent English-to-Bengali robustness gap suggests that robustness improvements learned in English do not transfer automatically; adapting models to Bengali math may require explicit Bengali training data or language-specific prompting strategies.
  • A concrete next step would be to back-translate the Bengali perturbed items to English and re-run the same models: if English accuracy recovers while Bengali accuracy drops, the gap likely reflects translation quality or language-model multilingual gaps rather than reasoning ability.
  • The authors' evaluation subset of 1,000 seeds is not described as randomly sampled; a formal random-sampling audit or inter-annotator agreement check on the translation step would substantiate the baseline numbers reported here.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper introduces GSM-PLUS-BN, a Bengali perturbed-math benchmark derived from the English GSM-Plus dataset, and evaluates six open-weight LLMs under standard prompting and zero-shot chain-of-thought prompting. The dataset comprises 10,544 instances from 1,318 GSM8K/GSM-Plus seed questions across eight perturbation categories, translated by a hybrid AI-plus-human process. The authors report exact-match accuracies for seed questions and for each perturbation type, and draw conclusions about model robustness, the difficulty of Critical Thinking and arithmetic perturbations, and a performance gap relative to English benchmarks. The dataset is publicly released on Mendeley Data, and the paper positions it as the first systematic robustness benchmark for Bengali mathematical reasoning.

Significance. If the dataset construction and the reported baselines are valid, this is a useful contribution. GSM-PLUS-BN fills a clear gap: no previous Bengali benchmark systematically applies perturbation-based robustness evaluation to mathematical reasoning. The public release on Mendeley Data, the eight-category perturbation taxonomy aligned with GSM-Plus, the two prompting conditions, and the spread of model sizes and architectures (8B–120B, dense and MoE) make the resource immediately usable for low-resource reasoning research. The human-translation effort is a strength in principle, though its reliability is not quantified. No machine-checked proofs or parameter-free derivations are involved; the value is empirical. However, the central claims currently rest on several unstated assumptions about answer preservation, answer extraction, sampling, and the English comparison, which must be resolved before the baselines can be taken as reliable.

major comments (5)
  1. [§3.1.5 vs. §3.4 and §3.5] Critical Thinking is defined as removing essential information so that the model 'should recognize that the available information is insufficient and explicitly identify the missing information.' Yet §3.4 states that the variants 'preserve the correct numerical answer of the original seed question,' and §3.5 states that seed_answer 'remains identical across all eight adversarial variants.' These statements are incompatible. If a Critical Thinking variant is genuinely underdetermined, the seed's numeric answer is not entailed by the question text; scoring by exact match to that numeric answer would penalize exactly the behavior the category is designed to elicit. Tables 5 and 6 report Critical Thinking accuracies of 6.60–31.80% (Standard) and 11.00–33.00% (CoT), but without specifying what the gold label is for this category and how non-numeric insufficiency responses are mapped, these nu
  2. [§3.1.2 vs. §3.4 and §3.5] The unconditional answer-preservation guarantee is also contradicted by the Arithmetic Variation examples. The Adding Operation example changes the asked quantity from total bolts to total cost, so the seed answer (3 bolts) is not the answer to the variant; the Reversing Operation example asks for blue fiber and has answer 2, not 3. Thus the statement 'These variants preserve the correct numerical answer of the original seed question' (§3.4) is false as written. The paper should clarify whether the dataset stores a per-variant answer separately from seed_answer, and whether Tables 5–6 are computed against the variant-specific answer. If they are, the text of §3.4 and §3.5 needs correction; if they are not, the arithmetic-variation baselines are invalid.
  3. [Abstract and §5] The abstract claims 'a notable performance gap persists across all models relative to their English benchmarks,' but no English control experiment is reported anywhere. Tables 4–7 contain Bengali-only accuracies. Citing the original GSM-Plus result (GPT-3.5-Turbo ≈61%, §2.4) does not establish an English baseline for the six evaluated models under the same prompts, decoding settings, and extraction rules. This cross-lingual claim should be removed, or the authors should run the identical protocol on the English GSM-Plus (even a matched subset) and report per-model English accuracies.
  4. [§5, Tables 4–7] All reported numbers are single-run exact-match accuracies with no decoding details, no answer-extraction specification, no variance estimates, and no significance tests. The problem is acute given the large observed swings (e.g., Qwen3-32B seed accuracy 13.98% under Standard vs. 76.25% under CoT; Llama-3.1-8B dropping 23.27 points under CoT). Without temperature, top-p, max-tokens, the exact prompt templates, and a deterministic rule for extracting numeric answers from Bengali free text (and for handling non-numeric Critical Thinking responses), the results are not reproducible and rankings may be within run-to-run noise. The authors should provide the full inference protocol and at least report standard errors or confidence intervals from multiple runs.
  5. [§5 opening] The evaluation uses 9,000 samples comprising '1000 samples from seed questions and the other 8000' variants, but the paper never states how the 1,000 seeds were selected from the dataset's 1,318 seeds. If the subset is not random—for example, chosen for translatability or answerability—Tables 4–7 may not be representative of GSM-PLUS-BN as a whole. The sampling procedure should be stated explicitly, and preferably accompanied by a comparison of the sampled and unsampled seed populations.
minor comments (7)
  1. [Throughout] 'Perturbated' is used repeatedly (title, abstract, §§4–5) where 'perturbed' is the standard term.
  2. [§3.1.4, Figure 4, Tables 5–6] The category is inconsistently named 'Distractor Insertion', 'Distraction Insertion', and 'Distraction Insertion'. Use one spelling.
  3. [§3.5] The data-structure description lists seed_question and seed_solution but does not make clear whether solution and answer are per-variant or per-seed. Define this precisely.
  4. [§5.3] After Table 6, several paragraphs begin with bare parenthetical phrases such as '(Numerical Substitution, Digit Expansion, ...' and '(Reversing Operation, Problem Understanding)' with no main clause. These should be rewritten as complete sentences.
  5. [References] References [56] and [57] appear to be the same survey by Wang et al.; one duplicate should be removed.
  6. [Figure 1 and Figure 4] The Bengali example sentences have no English glosses, making it difficult for a non-Bengali reader to verify the perturbation transformations. Adding word-aligned glosses or English back-translations would improve accessibility and auditability.
  7. [§4.2] The prompt templates are illustrated in figures but the exact system and user strings are not given in a machine-readable appendix. For reproducibility, the full prompts should be included.

Circularity Check

1 steps flagged

Critical Thinking ground truth is defined as the seed's numeric answer even though the CT definition requires an insufficiency judgment; the low CT accuracies and the 'missing logical reasoning' conclusion are built into the answer key by construction.

specific steps
  1. self definitional [§3.1.5, §3.4, §3.5; results in §5.6, Tables 5–6; metric in §6]
    "Critical Thinking ... removes essential information ... models are expected to recognize that the available information is insufficient and explicitly identify the missing information [§3.1.5]. seed_answer: The final answer of the original seed question, which remains identical across all eight adversarial variants [§3.5]. ... uniformly poor performance on Critical Thinking perturbations ... reveals that current LLMs fundamentally lack complex, multi-step logical reasoning capabilities [§5.6]."

    With exact-match scoring (§6), a model that follows §3.1.5 and answers 'insufficient information' cannot equal the numeric seed_answer that §3.5 declares invariant, so CT rows are keyed to the seed's number by construction. Low CT accuracy is therefore partly an artifact of the answer key, and the §5.6 inference that models lack the capability is definitionally forced. If instead the CT answer key were an insufficiency response, §3.5's preservation guarantee would be false—either way the reported CT behavior is not an independent measurement.

full rationale

This is an empirical benchmark paper, not a derivation: the dataset is an external translation artifact, and the model accuracies are measurements, so the default assessment is mostly non-circular. There are no fitted parameters being relabeled as predictions, and no uniqueness or correctness theorem is imported from the authors' prior work; the SOMADHAN/PatiGonit self-citations in §2 and §4.2.1 are contextual and do not carry the results. The one definitional circularity is the Critical Thinking category. §3.1.5 defines the correct response as identifying missing information, but §3.4 and §3.5 define every variant's seed_answer as the original numeric answer, and §6 confirms exact-match scoring. Thus the CT accuracy columns in Tables 5–6 and the conclusion that models 'fundamentally lack complex logical reasoning' (§5.6) are in part a construction of the answer key: a model obeying §3.1.5 is scored wrong by §3.5. This does not vitiate the other seven perturbation types, which have no such definitional entanglement, but it does infect the headline 'Critical Thinking is hardest' result. I therefore score the partial circularity at 6 rather than treating the whole paper as circular.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

No numerical parameters are fitted; the paper makes no predictions from fitted constants, so the free-parameter ledger is empty. The central claim rests instead on domain assumptions: the GSM-Plus perturbation property carries over to translation, human verification preserves answers, exact-match is an adequate metric, and the Groq API models correspond to the named checkpoints. None of these are verified with statistics in the paper. No new entities (particles, forces, conserved quantities) are introduced.

axioms (5)
  • domain assumption GSM-Plus perturbed variants preserve the seed question's correct answer by construction.
    Invoked in §3.4-3.6; the paper transfers this property to Bengali without re-verifying it for the translated text.
  • domain assumption The English-to-Bengali human translation preserves mathematical semantics and the correct answer for all 10,544 items.
    §3.2-3.3; asserted, with no automated back-translation checks, inter-annotator agreement, or error-rate audit.
  • domain assumption Exact-match accuracy on the final answer is an adequate correctness metric.
    §6 acknowledges this metric ignores partial correctness; used throughout §5.
  • domain assumption The Groq API endpoints faithfully serve the six named open-source models in fixed versions.
    §4.1; model versions are aliases (e.g., Llama-3.3-70B-Versatile) and API snapshots are not pinned.
  • domain assumption The eight perturbation categories are independent and jointly measure reasoning robustness.
    §3.1; used to draw conclusions about which perturbation types are 'hardest' in §5.2-5.6.

pith-pipeline@v1.3.0-alltime-deepseek · 28764 in / 11903 out tokens · 115575 ms · 2026-08-02T05:43:16.268752+00:00 · methodology

0 comments
read the original abstract

The evaluation of mathematical reasoning in large language models (LLMs) has predominantly focused on high-resource languages like English. This has created a significant barrier to the equitable development and deployment of AI in linguistically diverse regions such as Bangladesh, where over 230 million people speak Bengali. Despite this global significance, there has been minimal prior work on mathematical reasoning in Bengali and no existing research that systematically benchmarks a perturbated Bengali mathematical dataset, leaving a critical void in assessing model robustness and true comprehension beyond pattern recognition. This study addresses this gap by introducing GSM-Plus-BN, a novel perturbated Bengali mathematical dataset derived from the English GSM-Plus benchmark and verified by human translators. We evaluate six open-source LLMs Qwen3-32B, Llama-3.1-8B-Instant, Llama-3.3-70B-Versatile, Llama-4-Scout-17B-16E-Instruct, GPT-OSS-120B, and GPT-OSS-20B using a benchmark of 9,000 evaluation samples comprising 1,000 seed questions and 8,000 perturbed variants under both Standard Prompting and Chain-of-Thought (CoT) Prompting. Experimental results show that GPT-OSS-20B achieves the highest seed question accuracy of 96.08% under Standard Prompting, while larger models such as Llama-3.3-70B and GPT-OSS-120B demonstrate superior robustness across perturbation types. Furthermore, CoT prompting substantially improves reasoning for most models compared to Standard Prompting, yet a notable performance gap persists across all models relative to their English benchmarks, underscoring the inherent difficulty of perturbed Bengali text. This research makes a foundational contribution by providing GSM-PLUS-BN as a new resource and baseline for future Bengali mathematical reasoning research.

Figures

Figures reproduced from arXiv: 2607.13248 by Bidyarthi Paul, Md. Asif Karim, Nahida Jannat Mayouree, Sagar Chandra Nath, Swastika Kundu.

Figure 1
Figure 1. Figure 1: A seed example and its corresponding variants generated using three perturbation types. Despite the substantial progress in English-language mathematical reasoning evaluation, the research landscape for low-resource languages remains severely underexplored [29, 30]. This disparity is not merely a matter of academic interest but has profound implications for the equitable development and deployment of AI te… view at source ↗
Figure 2
Figure 2. Figure 2: provides a comprehensive overview of the development of our proposed dataset. GSM-PLUS-BN Data Collection Data Translation Data Preprocessing GSM-PLUS DATASET [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Summary of all perturbation types with Bengali examples 3.6 Summary of Perturbation Examples Both GSM-Plus and GSM-Plus-BN employ the same adversarial perturbation framework to evaluate the robustness of large language models (LLMs) in mathematical reasoning. Since GSM-Plus-BN is a direct Bengali translation of GSM￾Plus, the perturbation structure and reasoning logic remain identical across both datasets, … view at source ↗
Figure 5
Figure 5. Figure 5: Schematic diagram of our proposed methodology. 4.1.1 Qwen3-32B Qwen3-32B is an open-source large language model developed by Alibaba’s Qwen team and released under the Apache 2.0 license [82]. The model is designed for multilingual instruction following, logical reasoning, and complex problem solving, demonstrating strong performance across a wide range of language understanding and mathematical reasoning … view at source ↗
Figure 6
Figure 6. Figure 6: Illustration of the Standard prompting strategy This strategy serves a critical methodological function as the baseline condition in our comparative analysis. By establishing a performance benchmark under standard prompting, we create a control against which the efficacy of more sophisticated prompting techniques such as Chain-of-Thought can be quantified [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Illustration of the Chain-of-Thought prompting strategy, demonstrating the step-by-step reasoning process generated by the LLM. computation and symbolic manipulation which represents a critical bottleneck in standard prompting scenarios. This approach is particularly valuable in our experimental design, as it allows us to assess whether the LLM’s performance on complex reasoning tasks can be improved throu… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

89 extracted references · 27 linked inside Pith

  1. [1]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  2. [2]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 , 2021

    Pith/arXiv arXiv 2021

  3. [3]

    Language models are few-shot learners

    Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems , 33:1877–1901, 2020

    1901

  4. [4]

    Palm: Scaling language modeling with pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022

    Pith/arXiv arXiv 2022

  5. [5]

    Scaling language models: Methods, analysis & insights from training gopher

    Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Y oung, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021

    Pith/arXiv arXiv 2021

  6. [6]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837, 2022

    2022

  7. [7]

    Lampinen, Ishita Dasgupta, Stephanie C

    Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y . Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James L. McClelland, Jane X. Wang, and Felix Hill. Can language models learn from explanations in context?, 2022

    2022

  8. [8]

    Evaluating mathematical reasoning of large language models: A focus on error identification and correction

    Xiaoyuan Li, Wenjie Wang, Moxin Li, Junrong Guo, Y ang Zhang, and Fuli Feng. Evaluating mathematical reasoning of large language models: A focus on error identification and correction. In Findings of the Association for Computational Linguistics: ACL 2024 , pages 11316–11360, 2024

    2024

  9. [9]

    Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on machine learning research, 2023

    2023

  10. [10]

    Mathify: Evaluating large language models on mathematical problem solving tasks, 2024

    Avinash Anand, Mohit Gupta, Kritarth Prasad, Navya Singla, Sanjana Sanjeev, Jatin Kumar, Adarsh Raj Shivam, and Rajiv Ratn Shah. Mathify: Evaluating large language models on mathematical problem solving tasks, 2024

    2024

  11. [11]

    Challenging big-bench tasks and whether chain-of-thought can solve them

    Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022

    Pith/arXiv arXiv 2022

  12. [12]

    Benchmarking in-context learning strategies of large lan- guage models for math reasoning tasks

    Y ao Rong, Kathrin Seßler, Enkelejda Kasneci, et al. Benchmarking in-context learning strategies of large lan- guage models for math reasoning tasks. IEEE Transactions on Learning Technologies, 2025

    2025

  13. [13]

    Omni-math: A universal olympiad level mathematic benchmark for large language models

    Bofei Gao, Feifan Song, Zhe Y ang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Zhengyang Tang, et al. Omni-math: A universal olympiad level mathematic benchmark for large language models. In International Conference on Learning Representations , volume 2025, pages 100540–100569, 2025

    2025

  14. [14]

    ozl¨"ukl¨

    Kathrin Seßler, Y ao Rong, Emek G¨"ozl¨"ukl¨"u, and Enkelejda Kasneci. Benchmarking large language models for math reasoning tasks. arXiv preprint arXiv:2408.10839, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    Lila: A unified benchmark for mathematical reasoning

    Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, et al. Lila: A unified benchmark for mathematical reasoning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 5807–5832, 2022

    2022

  16. [16]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations , volume 2024, pages 23439–23554, 2024

    2024

  17. [17]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are nlp models really able to solve simple math word prob- lems? In Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies , pages 2080–2094, 2021

    2021

  18. [18]

    Frontiermath: A benchmark for evaluating advanced mathematical reasoning in ai, 2025

    Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Ols- son, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, Olli Järviniemi, Matthew Barnett, Robert 23 A PREPRINT - J ULY 16, 2026 Sandler, Matej Vrzala, Jaime Sevilla, Qiuyu Ren, Elizabeth Pratt, Lionel Levine, Grant Barkley, Natalie Stew- art,...

    2026

  19. [19]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    Pith/arXiv arXiv 2009

  20. [20]

    Math-perturb: Benchmarking llms’ math reasoning abilities against hard perturbations, 2025

    Kaixuan Huang, Jiacheng Guo, Zihao Li, Xiang Ji, Jiawei Ge, Wenzhe Li, Yingqing Guo, Tianle Cai, Hui Y uan, Runzhe Wang, Y ue Wu, Ming Yin, Shange Tang, Y angsibo Huang, Chi Jin, Xinyun Chen, Chiyuan Zhang, and Mengdi Wang. Math-perturb: Benchmarking llms’ math reasoning abilities against hard perturbations, 2025

    2025

  21. [21]

    Dynamath: A dynamic vi- sual benchmark for evaluating mathematical reasoning robustness of vision language models

    Chengke Zou, Xingang Guo, Rui Y ang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic vi- sual benchmark for evaluating mathematical reasoning robustness of vision language models. In International Conference on Learning Representations, volume 2025, pages 48337–48383, 2025

    2025

  22. [22]

    Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers

    Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers. arXiv preprint arXiv:2402.19255, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Evaluating robustness of llms to numerical variations in mathematical reasoning

    Y uli Y ang, Hiroaki Y amada, and Takenobu Tokunaga. Evaluating robustness of llms to numerical variations in mathematical reasoning. In The Sixth Workshop on Insights from Negative Results in NLP , pages 171–180, 2025

    2025

  24. [24]

    An investigation of robustness of llms in mathematical reasoning: Benchmarking with mathematically-equivalent transformation of advanced mathematical problems, 2025

    Y uren Hao, Xiang Wan, and ChengXiang Zhai. An investigation of robustness of llms in mathematical reasoning: Benchmarking with mathematically-equivalent transformation of advanced mathematical problems, 2025

    2025

  25. [25]

    Robustness in large language models: A survey of mitigation strategies and evaluation metrics

    Pankaj Kumar and Subhankar Mishra. Robustness in large language models: A survey of mitigation strategies and evaluation metrics. arXiv preprint arXiv:2505.18658, 2025

    arXiv 2025

  26. [26]

    A novel metric for measuring the ro- bustness of large language models in non-adversarial scenarios

    Samuel Ackerman, Ella Rabinovich, Eitan Farchi, and Ateret Anaby Tavor. A novel metric for measuring the ro- bustness of large language models in non-adversarial scenarios. In Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 2794–2802, 2024

    2024

  27. [27]

    Do large language models understand their knowledge? AIChE Journal , 71(3):e18661, 2025

    V enkat V enkatasubramanian. Do large language models understand their knowledge? AIChE Journal , 71(3):e18661, 2025

    2025

  28. [28]

    Is your model really a good math reasoner? evaluating mathematical reasoning with checklist

    Zihao Zhou, Shudong Liu, Maizhen Ning, Wei Liu, Jindong Wang, Derek Wong, Xiaowei Huang, Qiufeng Wang, and Kaizhu Huang. Is your model really a good math reasoner? evaluating mathematical reasoning with checklist. In International Conference on Learning Representations , volume 2025, pages 34238–34281, 2025

    2025

  29. [29]

    Bhaasha, bhas

    Sampoorna Poria and Xiaolei Huang. Bhaasha, bhas. a, zaban: A survey for low-resourced languages in south asia–current stage and challenges. arXiv preprint arXiv:2509.11570, 2025

    arXiv 2025

  30. [30]

    Llms for low resource lan- guages in multilingual, multimodal and dialectal settings

    Firoj Alam, Shammur Absar Chowdhury, Sabri Boughorbel, and Maram Hasanain. Llms for low resource lan- guages in multilingual, multimodal and dialectal settings. In Proceedings of the 18th conference of the European chapter of the association for computational linguistics: tutorial abstracts , pages 27–33, 2024

    2024

  31. [31]

    Natural language processing applications for low-resource languages

    Partha Pakray, Alexander Gelbukh, and Sivaji Bandyopadhyay. Natural language processing applications for low-resource languages. Natural Language Processing, 31:183–197, 2025

    2025

  32. [32]

    Linguistic nepotism: Trading-off quality for language preference in multilingual rag

    Dayeon Ki, Marine Carpuat, Paul McNamee, Daniel Khashabi, Eugene Y ang, Dawn Lawrie, and Kevin Duh. Linguistic nepotism: Trading-off quality for language preference in multilingual rag. arXiv preprint arXiv:2509.13930, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Bengali language — Wikipedia, The Free Encyclopedia

    Wikipedia contributors. Bengali language — Wikipedia, The Free Encyclopedia. https://en.wikipedia. org/wiki/Bengali_language, 2026. Accessed: 2026-06-29

    2026

  34. [34]

    Geospatial and temporal trends in urban transportation: A study of nyc taxis and pathao food deliveries

    Bidyarthi Paul, Fariha Tasnim Chowdhury, Dipta Biswas, and Meherin Sultana. Geospatial and temporal trends in urban transportation: A study of nyc taxis and pathao food deliveries. arXiv preprint arXiv:2505.03816, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Banglamath: A bangla benchmark dataset for testing llm mathematical reasoning at grades 6, 7, and 8

    Tabia Tanzin Prama, Christopher M Danforth, and Peter Dodds. Banglamath: A bangla benchmark dataset for testing llm mathematical reasoning at grades 6, 7, and 8. In Proceedings of The 3rd Workshop on Mathematical Natural Language Processing (MathNLP 2025) , pages 134–149, 2025

    2025

  36. [36]

    Leveraging large language models for bengali math word problem solving with chain of thought reasoning

    Bidyarthi Paul, Jalisha Jashim Era, Mirazur Rahman Zim, Tahmid Sattar Aothoi, and Faisal Muhammad Shah. Leveraging large language models for bengali math word problem solving with chain of thought reasoning. arXiv preprint arXiv:2505.21354, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Ganitllm: Difficulty-aware bengali mathematical reasoning through curriculum-grpo

    Shubhashis Roy Dipta, Khairul Mahbub, and Nadia Najjar. Ganitllm: Difficulty-aware bengali mathematical reasoning through curriculum-grpo. In Findings of the Association for Computational Linguistics: ACL 2026 , 2026

    2026

  38. [38]

    BanglaBERT: A large-scale language model for bangla

    Abhik Bhattacharjee, Tahmid Hasan, SM Mahmudur Rahman, and Ratul R Shahid. BanglaBERT: A large-scale language model for bangla. In ACL, 2022. 24 A PREPRINT - J ULY 16, 2026

    2022

  39. [39]

    BanglaBERT-Base: A smaller model for bangla NLP

    Subhankar Saha, Abhik Bhattacharjee, and SM Mahmudur Rahman. BanglaBERT-Base: A smaller model for bangla NLP. In EMNLP, 2023

    2023

  40. [40]

    Xtreme: A mas- sively multilingual multi-task benchmark for evaluating cross-lingual generalisation

    Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. Xtreme: A mas- sively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In International conference on machine learning, pages 4411–4421. PMLR, 2020

    2020

  41. [41]

    XGLUE: A new benchmark dataset for cross-lingual knowledge graph construction

    Y aobo Liang, Nan Duan, Y eyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, et al. XGLUE: A new benchmark dataset for cross-lingual knowledge graph construction. In EMNLP, 2020

    2020

  42. [42]

    Breaking language barriers in mul- tilingual mathematical reasoning: Insights and observations

    Nuo Chen, Zinan Zheng, Ning Wu, Ming Gong, Dongmei Zhang, and Jia Li. Breaking language barriers in mul- tilingual mathematical reasoning: Insights and observations. In Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 7001–7016, 2024

    2024

  43. [43]

    BanglaNER: A named entity recognition dataset for bangla

    Md Shamsul Islam, Md Sajjad Rahman, and Md Ariful Hossain. BanglaNER: A named entity recognition dataset for bangla. In LREC, 2020

    2020

  44. [44]

    Bnmmlu: Measuring massive multitask language understanding in bengali

    Saman Sarker Joy and Swakkhar Shatabda. Bnmmlu: Measuring massive multitask language understanding in bengali. In Findings of the Association for Computational Linguistics: ACL 2026 , pages 12211–12230, 2026

    2026

  45. [45]

    MathQA: A dataset for mathematical question answering

    Aida Amini, Saadia Gabriel, Shan Lin, Rik Koncel-Kedziorski, Y ejin Choi, and Hannaneh Hajishirzi. MathQA: A dataset for mathematical question answering. In NeurIPS, 2019

    2019

  46. [46]

    Qwen3 technical report

    An Y ang, Anfeng Li, Baosong Y ang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Y u, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  47. [47]

    Qwen technical report

    Jinze Bai, Shuai Bai, Y unfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Y ang Fan, Wenbin Ge, Y u Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    Pith/arXiv arXiv 2023

  48. [48]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Y asmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  49. [49]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex V aughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  50. [50]

    Evolution of meta’s llama models and parameter- efficient fine-tuning of large language models: a survey

    Abdulhady Abas Abdullah, Arkaitz Zubiaga, Seyedali Mirjalili, Amir H Gandomi, Fatemeh Daneshfar, Moham- madsadra Amini, Alan Salam Mohammed, and Hadi V eisi. Evolution of meta’s llama models and parameter- efficient fine-tuning of large language models: a survey. arXiv preprint arXiv:2510.12178, 2025

    arXiv 2025

  51. [51]

    gpt-oss-120b & gpt-oss-20b model card

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Y u Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025

    Pith/arXiv arXiv 2025

  52. [52]

    Large language models are zero-shot reasoners, 2023

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Y utaka Matsuo, and Y usuke Iwasawa. Large language models are zero-shot reasoners, 2023

    2023

  53. [53]

    Towards understanding chain-of-thought prompting: An empirical study of what matters, 2023

    Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, Y ou Wu, Luke Zettlemoyer, and Huan Sun. Towards understanding chain-of-thought prompting: An empirical study of what matters, 2023

    2023

  54. [54]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Y utaka Matsuo, and Y usuke Iwasawa. Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems , 35:22199–22213, 2022

    2022

  55. [55]

    Mathprompter: Mathematical reasoning using large language models

    Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.05398, 2023

    Pith/arXiv arXiv 2023

  56. [56]

    A survey on large language models for mathematical reasoning

    Peng-Y uan Wang, Tian-Shuo Liu, Chenyang Wang, Ziniu Li, Yidi Wang, Shu Y an, Chengxing Jia, Xu-Hui Liu, Xinwei Chen, Jiacheng Xu, et al. A survey on large language models for mathematical reasoning. ACM Computing Surveys, 2025

    2025

  57. [57]

    A survey on large language models for mathematical reasoning

    Peng-Y uan Wang, Tian-Shuo Liu, Chenyang Wang, Ziniu Li, Yidi Wang, Shu Y an, Chengxing Jia, Xu-Hui Liu, Xinwei Chen, Jiacheng Xu, et al. A survey on large language models for mathematical reasoning. ACM Computing Surveys, 58(8):1–35, 2026

    2026

  58. [58]

    Bennumeval: A benchmark to assess llms numerical reasoning capabilities in bengali

    Kawsar Ahmed, Md Osama, Omar Sharif, Eftekhar Hossain, and Mohammed Moshiul Hoque. Bennumeval: A benchmark to assess llms numerical reasoning capabilities in bengali. In Findings of the Association for Computational Linguistics: ACL 2025 , pages 17782–17799, 2025. 25 A PREPRINT - J ULY 16, 2026

    2025

  59. [59]

    Empowering bengali education with ai: Solving bengali math word problems through transformer models

    Jalisha Jashim Era, Bidyarthi Paul, Tahmid Sattar Aothoi, Mirazur Rahman Zim, and Faisal Muhammad Shah. Empowering bengali education with ai: Solving bengali math word problems through transformer models. In 2024 27th International Conference on Computer and Information Technology (ICCIT) , pages 909–914. IEEE, 2024

    2024

  60. [60]

    Bmwp: the first bengali math word problems dataset for operation prediction and solving

    Sanchita Mondal, Debnarayan Khatua, Sourav Mandal, Dilip K Prasad, and Arif Ahmed Sekh. Bmwp: the first bengali math word problems dataset for operation prediction and solving. Discover Artificial Intelligence , 5(1):25, 2025

    2025

  61. [61]

    Mathmist: A parallel multilingual benchmark dataset for mathematical problem solving and reasoning

    Mahbub E Sobhani, Md Faiyaz Abdullah Sayeedi, Muhammad Tasnim Mohiuddin, Md Mofijul Islam, and Swakkhar Shatabda. Mathmist: A parallel multilingual benchmark dataset for mathematical problem solving and reasoning. In Findings of the Association for Computational Linguistics: EACL 2026 , pages 2524–2550, 2026

    2026

  62. [62]

    Bangla-bayanno: A 52k-pair bengali visual ques- tion answering dataset with llm-assisted translation refinement

    Mohammed Rakibul Hasan, Rafi Majid, and Ahanaf Tahmid. Bangla-bayanno: A 52k-pair bengali visual ques- tion answering dataset with llm-assisted translation refinement. arXiv preprint arXiv:2508.19887, 2025

    Pith/arXiv arXiv 2025

  63. [63]

    Mathify: Evaluating large language models on mathematical problem solving tasks

    Avinash Anand, Mohit Gupta, Kritarth Prasad, Navya Singla, Sanjana Sanjeev, Jatin Kumar, Adarsh Raj Shivam, and Rajiv Ratn Shah. Mathify: Evaluating large language models on mathematical problem solving tasks. arXiv preprint arXiv:2404.13099, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Benchmarking reasoning robustness in large language models

    Tong Y u, Y ongcheng Jing, Xikun Zhang, Wentao Jiang, Wenjie Wu, Yingjie Wang, Wenbin Hu, Bo Du, and Dacheng Tao. Benchmarking reasoning robustness in large language models. arXiv preprint arXiv:2503.04550, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Math-perturb: Benchmarking llms’ math reasoning abilities against hard perturbations

    Kaixuan Huang, Jiacheng Guo, Zihao Li, Xiang Ji, Jiawei Ge, Wenzhe Li, Yingqing Guo, Tianle Cai, Hui Y uan, Runzhe Wang, et al. Math-perturb: Benchmarking llms’ math reasoning abilities against hard perturbations. arXiv preprint arXiv:2502.06453, 2025

    Pith/arXiv arXiv 2025

  66. [66]

    Formalmath: Benchmarking formal mathematical reasoning of large language models

    Zhouliang Y u, Ruotian Peng, Keyi Ding, Yizhe Li, Zhongyuan Peng, Minghao Liu, Yifan Zhang, Zheng Y uan, Huajian Xin, Wenhao Huang, et al. Formalmath: Benchmarking formal mathematical reasoning of large language models. arXiv preprint arXiv:2505.02735, 2025

    Pith/arXiv arXiv 2025

  67. [67]

    On memorization of large language models in logical reasoning

    Chulin Xie, Y angsibo Huang, Chiyuan Zhang, Da Y u, Xinyun Chen, Bill Y uchen Lin, Bo Li, Badih Ghazi, and Ravi Kumar. On memorization of large language models in logical reasoning. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational...

    2025

  68. [68]

    Polymath: Evaluating mathematical reasoning in multilingual contexts

    Yiming Wang, Pei Zhang, Jialong Tang, Hao-Ran Wei, Baosong Y ang, Rui Wang, Chenshu Sun, Feitong Sun, Ji- ran Zhang, Junxuan Wu, et al. Polymath: Evaluating mathematical reasoning in multilingual contexts. Advances in Neural Information Processing Systems , 38, 2026

    2026

  69. [69]

    Mmath: A multilingual benchmark for mathematical reasoning

    Wenyang Luo, Wayne Xin Zhao, Jing Sha, Shijin Wang, and Ji-Rong Wen. Mmath: A multilingual benchmark for mathematical reasoning. Preprint, 2025

    2025

  70. [70]

    Matheval: A comprehensive benchmark for evaluating large language models on mathematical reasoning capabilities

    Tianqiao Liu, Zui Chen, Zhensheng Fang, Weiqi Luo, Mi Tian, and Zitao Liu. Matheval: A comprehensive benchmark for evaluating large language models on mathematical reasoning capabilities. Frontiers of Digital Education, 2(2):16, 2025

    2025

  71. [71]

    Frontiermath: A benchmark for evaluating advanced mathematical reasoning in ai

    Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, et al. Frontiermath: A benchmark for evaluating advanced mathematical reasoning in ai. arXiv preprint arXiv:2411.04872, 2024

    Pith/arXiv arXiv 2024

  72. [72]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  73. [73]

    Multimath: Bridging visual and mathematical reasoning for large language models

    Shuai Peng, Di Fu, Liangcai Gao, Xiuqin Zhong, Hongguang Fu, and Zhi Tang. Multimath: Bridging visual and mathematical reasoning for large language models. arXiv preprint arXiv:2409.00147, 2024

    Pith/arXiv arXiv 2024

  74. [74]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y ang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  75. [75]

    Advancing mathematical reasoning in language models: The impact of problem-solving data, data synthesis methods, and training stages

    Zui Chen, Tianqiao Liu, Mi Tian, Qing Tong, Weiqi Luo, and Zitao Liu. Advancing mathematical reasoning in language models: The impact of problem-solving data, data synthesis methods, and training stages. arXiv preprint arXiv:2501.14002, 2025

    Pith/arXiv arXiv 2025

  76. [76]

    Evaluating and improving tool-augmented computation-intensive math reasoning

    Beichen Zhang, Kun Zhou, Xilin Wei, Xin Zhao, Jing Sha, Shijin Wang, and Ji-Rong Wen. Evaluating and improving tool-augmented computation-intensive math reasoning. Advances in Neural Information Processing Systems, 36:23570–23589, 2023. 26 A PREPRINT - J ULY 16, 2026

    2023

  77. [77]

    Mario: Math reasoning with code interpreter output–a reproducible pipeline

    Minpeng Liao, Wei Luo, Chengxi Li, Jing Wu, and Kai Fan. Mario: Math reasoning with code interpreter output–a reproducible pipeline. arXiv preprint arXiv:2401.08190, 2024

    Pith/arXiv arXiv 2024

  78. [78]

    Malt: Improving reasoning with multi-agent llm training

    Sumeet Ramesh Motwani, Chandler Smith, Rocktim Jyoti Das, Rafael Rafailov, Ivan Laptev, Philip HS Torr, Fabio Pizzati, Ronald Clark, and Christian Schroeder de Witt. Malt: Improving reasoning with multi-agent llm training. arXiv preprint arXiv:2412.01928, 2024

    arXiv 2024

  79. [79]

    {\dag} dagger: Distractor-aware graph generation for executable reasoning in math problems

    Zabir Al Nazi, Shubhashis Roy Dipta, and Sudipta Kar. {\dag} dagger: Distractor-aware graph generation for executable reasoning in math problems. arXiv preprint arXiv:2601.06853, 2026

    arXiv 2026

  80. [80]

    Structured reasoning with tree-of-thoughts for bengali math word problems

    Aurprita Mahmood, Md Abdul Hadi, Md Sehab Al Islam, Minhajul Islam, et al. Structured reasoning with tree-of-thoughts for bengali math word problems. arXiv preprint arXiv:2512.05580, 2025

    arXiv 2025

Showing first 80 references.