REVIEW 4 major objections 4 minor 13 cited by
A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges
T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The first survey of mathematical reasoning in multimodal LLMs organizes the field into benchmarks, methods, and challenges.
desk verdict Useful survey with a genuinely practical taxonomy, but the 'comprehensive' claim is undercut by a missing search protocol and a fixable five-vs-seven challenge inconsistency. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The organizing machinery is the three-paradigm taxonomy of methodology—LLM as Reasoner, LLM as Enhancer, and LLM as Planner—combined with a four-aspect benchmark analysis (basic focus, task, evaluation, and training data). The taxonomy classifies the over 200 surveyed publications and exposes research gaps, such as the under-explored Planner paradigm and the scarcity of multimodal data augmentation.
What would settle it
A systematic literature search on multimodal mathematical reasoning from 2021 to 2025 that uncovers a substantial body of relevant work absent from the survey, or a methodology paradigm that does not fit the Reasoner/Enhancer/Planner taxonomy, would refute the claim of comprehensive coverage.
Extended reading notes
Core claim
The paper's central claim is that the state of MLLM-based mathematical reasoning can be comprehensively captured by a three-part structure: benchmarks (analyzed by basic focus, task, evaluation, and training data), methodologies (reasoner, enhancer, and planner paradigms), and challenges (seven named obstacles from data scarcity to test-time scaling). It argues that this is the first survey to treat multimodal mathematical reasoning as its own area, distinct from earlier text-only or education-focused surveys. The paper observes that single-modality algebraic reasoning still dominates the method literature, but multimodal approaches have been increasing since 2024, and that the Reasoner role is the most common, while the Planner role is the least explored yet promising for multi-agent systems.
Load-bearing premise
The survey assumes that its selection of over 200 papers, with no described systematic search or inclusion criteria, is comprehensive and representative enough to support the 'first-ever comprehensive analysis' claim.
Editorial extensions
If this is right
- Future work can position new methods as Reasoner, Enhancer, or Planner, enabling direct comparison and hybrid combinations of the three paradigms.
- The seven challenges provide a concrete checklist for research priorities, with addressing them framed as a step toward human-level mathematical reasoning in AI.
- The observed rise in multimodal method papers since 2024 suggests that benchmark and data efforts should shift toward multimodal and dynamic problem settings.
- The paper's anticipation of a hybrid architecture—Enhancer generating data for Reasoners, coordinated by a Planner—offers a concrete design for future multimodal reasoning systems.
Reading between the lines
- The three-paradigm taxonomy may generalize to other reasoning domains, such as scientific or legal reasoning, offering a template for future surveys.
- The survey's inconsistency in counting five versus seven challenges suggests the taxonomy was refined during writing; readers should treat the three-paradigm structure as the more durable contribution.
- A testable extension: apply the taxonomy to classify a held-out set of recent multimodal reasoning papers and measure inter-annotator agreement, to check whether the categories are reliable enough for community adoption.
- The claimed 'first-ever' status depends on how narrowly 'multimodal mathematics' is defined; overlapping surveys exist, but the specific triad of benchmark-method-challenge appears new.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This survey claims to provide the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models (MLLMs), reviewing over 200 publications since 2021. It organizes the field into three dimensions: benchmarks (Section 2), methodologies (Section 3), and challenges (Section 4). The methodology part introduces a Reasoner/Enhancer/Planner taxonomy, and the challenge section lists seven open problems. The paper includes large summary tables of benchmarks, Math-LLMs, and methods, together with additional discussion in appendices.
Significance. If the coverage claim holds, this survey is a potentially useful entry point for researchers working on MLLM-based mathematical reasoning. The three-paradigm taxonomy (Reasoner, Enhancer, Planner) is a sensible organizing principle, and the appendix tables compile a substantial amount of information in one place. The contribution is a structured synthesis rather than a new algorithm or dataset, so its value depends on whether the selection of works is representative and whether the novelty claim is properly delimited. At present, that central condition is not fully verifiable from the manuscript.
major comments (4)
- [Section 1 (Structure) and Limitations] The central claim to be 'comprehensive' is not supported by a reproducible literature-search protocol. The paper does not state which databases were queried, which search strings or time windows were used, or which inclusion and exclusion criteria defined 'relevant studies.' The Limitations section further concedes that 'some relevant studies were overlooked.' These statements are not outright contradictory, but the authors should either provide the selection methodology or temper the 'first-ever comprehensive' claim; otherwise the main contribution cannot be independently checked.
- [Abstract, Section 1, Section 4, Section 5] The number of challenges is internally inconsistent: the Abstract and Section 1 state seven major challenges, Section 4 indeed lists seven, but the Conclusion in Section 5 says 'five key challenges.' Since the challenge taxonomy is one of the three stated contributions of the survey, this inconsistency must be fixed before publication.
- [Table 1 and Section 1 (Scope)] The 'first-ever' claim is not sufficiently differentiated from existing surveys. Table 1 already lists Ahn et al. (2024) as covering LLM4Math with a multimodal checkmark, and the text says prior work 'failed to explore the development and challenges of mathematical reasoning in multimodal settings in depth' without specifying concrete differences in scope, depth, or organizing principle. Please state explicitly how the present survey differs from Ahn et al. and from other concurrent or prior surveys, and why those works do not already occupy the claimed niche.
- [Section 3.3 and Table 5] AlphaGeometry (Trinh et al., 2024) is classified as an LLM-as-Enhancer example and marked as multimodal in Table 5, but AlphaGeometry is not a multimodal large language model: it combines a neural language model with a symbolic deduction engine and does not take visual diagram inputs in the way described elsewhere in the paper. Please justify this inclusion or reclassify it, since the MLLM-specific focus is a core premise of the survey.
minor comments (4)
- [Table 3] The benchmark name appears as 'ErrorRador' in Table 3 but as 'ErrorRadar' in Sections 2.3 and 2.4; please use one spelling consistently.
- [Table 3 legend] The Level code 'H' is used for both 'High School' and 'Hybrid', which makes many rows ambiguous; rename one of the labels (e.g., 'HS' and 'Mix').
- [Section 2.5 and elsewhere] Names such as 'G-LLaV A', 'MA VIS', and 'Math-LLaV A' contain spurious spacing introduced by LaTeX; these should be repaired throughout the manuscript.
- [Appendix D] In the CoLeG-E formula, the ground-truth answer is written as M(qr) on the right-hand side; this should presumably be gt(qr), as in the other formulas in the same appendix.
Circularity Check
No significant circularity: the survey compiles external literature into a taxonomy, and its self-citations are illustrative examples rather than load-bearing premises.
full rationale
This is a survey paper, not a derivation or prediction system: its contributions are a categorization of benchmarks, a three-paradigm methodology taxonomy, and a list of challenges. None of these are obtained by fitting a parameter to data and then 'predicting' the same quantity, and no result is defined in terms of another result by construction. The paper's central claim of being 'the first-ever comprehensive analysis' is a scope claim supported by a comparison table of prior surveys (Table 1) and a stated corpus of 'over 200 publications'; it is not derived from a self-citation. The several self-citations (e.g., ErrorRadar, Yan et al. 2024a, used to illustrate error-detection benchmarks and metrics) are used as examples of existing work, not as authority for the survey's own framework. The Limitations section concedes that 'some relevant studies were overlooked,' but that is an admission of possible incompleteness and a correctness/verifiability concern, not circular reasoning. No equation, mapping, or benchmark result in the paper reduces to its own input, and no uniqueness theorem or prior result by the same authors is invoked to forbid alternatives. The taxonomy and challenge categories are compiled from and checked against the cited external literature, so the survey is self-contained in the sense relevant to circularity.
Assumptions & free parameters
assumptions (2)
- domain assumption The selection of 'over 200 publications' is representative of the field
- domain assumption The three-paradigm taxonomy (Reasoner, Enhancer, Planner) exhaustively categorizes MLLM math reasoning methods
Cite this review
Pith. "Pith review of A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges." pith.science (2026). https://pith.science/paper/GQ6HEDCY
@misc{pith2026241211936,
author = {Pith},
title = {Pith review of: A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges},
year = {2026},
howpublished = {\url{https://pith.science/paper/GQ6HEDCY}},
note = {Machine review of arXiv:2412.11936}
}
read the original abstract
Mathematical reasoning, a core aspect of human cognition, is vital across many domains, from educational problem-solving to scientific advancements. As artificial general intelligence (AGI) progresses, integrating large language models (LLMs) with mathematical reasoning tasks is becoming increasingly significant. This survey provides the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models (MLLMs). We review over 200 studies published since 2021, and examine the state-of-the-art developments in Math-LLMs, with a focus on multimodal settings. We categorize the field into three dimensions: benchmarks, methodologies, and challenges. In particular, we explore multimodal mathematical reasoning pipeline, as well as the role of (M)LLMs and the associated methodologies. Finally, we identify five major challenges hindering the realization of AGI in this domain, offering insights into the future direction for enhancing multimodal reasoning capabilities. This survey serves as a critical resource for the research community in advancing the capabilities of LLMs to tackle complex multimodal reasoning tasks.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 13 Pith papers
-
GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning
A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.
-
Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis
PhantomCircuit traces knowledge overshadowing to attention circuits during training and prunes circuit edges to recover the overshadowed answer.
-
Towards Omnidirectional Reasoning with 360-R1: A Dataset, Benchmark, and GRPO-based Method
OmniVQA is a first open-source dataset and benchmark for 360-degree visual question answering, and 360-R1 uses GRPO with three LLM-based rewards to improve an existing multimodal model on it.
-
Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?
A new benchmark with over 153,000 question-answer pairs shows that eight state-of-the-art multimodal LLMs perform poorly on spatial reasoning in 360-degree panoramic indoor images.
-
Understanding Financial Reasoning in AI: A Multimodal Benchmark and Error Learning Approach
A new multimodal financial reasoning benchmark and a retrieval-based error feedback prompting method that improves model accuracy, with the improvement partly confounded by information leakage.
-
WAR: Workload-Aware Rollouts for Synchronous Agentic Reinforcement Learning
A workload-aware rollout system combining suffix-based speculative decoding (low load) and cache-aware scheduling (high load) speeds synchronous agentic RL rollout by 1.4-1.6x.
-
CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring
A student-teacher multi-agent pipeline with positive-only feedback improves QWK agreement with human essay scores by 21% on a multimodal benchmark, with gains concentrated in traits where baselines were weakest.
-
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
A large collaborative survey organizes LLM and LLM-agent safety issues into a full-stack lifecycle framework from data preparation to deployment.
-
Reasoning Language Models: A Blueprint
A modular blueprint and open-source framework (x1) that presents existing reasoning language model designs as special cases of one unified toolbox.
-
Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities
Activation-frequency analysis identifies sparse units in LLMs that respond to instructions; same-category instructions share more of these units than different-category ones, and fine-tuning measurably changes the sets.
-
Plane Geometry Problem Solving with Multi-modal Reasoning: A Survey
A survey of plane geometry problem solving that classifies methods into an encoder-decoder framework and analyzes hallucination and data leakage in current benchmarks.
-
Are Large Language Models Capable of Deep Relational Reasoning? Insights from DeepSeek-R1 and Benchmark Comparisons
DeepSeek-R1 outperforms GPT-4o and DeepSeek-V3 on family tree and graph reasoning benchmarks at sizes 10 and 20, but all models collapse at size 40.
-
A Survey on Large Language Models for Mathematical Reasoning
Recent advances in LLM mathematical reasoning are organized into comprehension and generation phases, covering methods from prompting to test-time scaling.
Reference graph
Works this paper leans on
-
[1]
Release Trends: The models started emerg- ing in 2020, with a significant increase in the number of releases from 2022 onward, indi- cating a growing interest in developing math- specific LLMs
work page 2020
-
[2]
Parameter Sizes: There is a noticeable trend towards larger parameter sizes, with some models offering up to 130B parameters, re- flecting the increasing computational capacity for handling complex mathematical tasks
-
[3]
Evaluation Benchmarks: Many models are evaluated on popular benchmarks like GSM8K, MATH, and MMLU, highlight- ing the focus on improving performance across well-established mathematical reason- ing datasets
-
[4]
arXiv preprint arXiv:2409.11074
Romath: A mathematical reasoning bench- mark in romanian. arXiv preprint arXiv:2409.11074. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. 2025. Process rein- forcement through implicit rewards. arXiv preprint arXiv:2502.01456. Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li ...
arXiv 2025
-
[5]
Open Source: A significant number of mod- els are open-source, allowing broader access and fostering further research and develop- ment in the field. In summary, the table reflects the rapid develop- ment of specialized Math-LLMs, with an increas- ing trend towards larger models, comprehensive evaluation benchmarks, and support for multilin- gual applicat...
work page 2024
-
[6]
arXiv preprint arXiv:2404.10690
Mathwriting: A dataset for handwritten mathematical expression recognition. arXiv preprint arXiv:2404.10690. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.17452. Aryan Gulati, Brando Miranda, E...
arXiv 2023
-
[7]
arXiv preprint arXiv:2412.16720
Openai o1 system card. arXiv preprint arXiv:2412.16720. Miaomiao Ji, Yanqiu Wu, Zhibin Wu, Shoujin Wang, Jian Yang, Mark Dras, and Usman Naseem. 2025. A survey on progress in llm alignment from the perspective of reward design. arXiv preprint arXiv:2505.02666. Mengzhao Jia, Zhihan Zhang, Wenhao Yu, Fangkai Jiao, and Meng Jiang. 2024. Describe-then-reason:...
arXiv 2025
-
[9]
ATHENA: Mathematical Reasoning with Thought Expansion
Athena: Mathematical reasoning with thought expansion. arXiv preprint arXiv:2311.01036. Eldar Kurtic, Amir Moeini, and Dan Alistarh. 2024. Mathador-lm: A dynamic benchmark for mathemat- ical reasoning on large language models. arXiv preprint arXiv:2406.12572. Guillaume Lample, Timothee Lacroix, Marie-Anne Lachaux, Aurelien Rodriguez, Amaury Hayat, Thibaut...
work page Pith review arXiv 2024
Show all 49 references
-
[10]
Advances in neural information processing systems, 35:26337–26349
Hypertree proof search for neural theorem proving. Advances in neural information processing systems, 35:26337–26349. Jung Hyun Lee, June Yong Yang, Byeongho Heo, Dongyoon Han, and Kang Min Yoo. 2024. Token- supervised value models for enhancing mathemati- cal reasoning capabi...
2024 arXiv
-
[13]
arXiv preprint arXiv:2410.05229
Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models. arXiv preprint arXiv:2410.05229. Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpuro- hit, Oyvind Tafjord, Ashish Sabharwal, Peter Cl...
-
[14]
arXiv preprint arXiv:2502.06563
Large language models meet symbolic provers for logical reasoning evaluation. arXiv preprint arXiv:2502.06563. Jinghui Qin, Zhicheng Yang, Jiaqi Chen, Xiaodan Liang, and Liang Lin. 2023. Template-based contrastive distillation pretraining for math word problem solv- ing. IEEE ...
2023 arXiv
-
[15]
arXiv preprint arXiv:2404.06405
Wu’s method can boost symbolic ai to ri- val silver medalists and alphageometry to outper- form gold medalists at imo geometry. arXiv preprint arXiv:2404.06405. Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, and Yu Cheng. 2025. Prmbench: A fine-grained and challenging ben...
2025 arXiv
-
[16]
arXiv preprint arXiv:2311.07594
How to bridge the gap between modalities: A comprehensive survey on multimodal large language model. arXiv preprint arXiv:2311.07594. SquirrelAiLearning. 2024. Squirrel ai official platfrom. Pragya Srivastava, Manuj Malik, Vivek Gupta, Tanuja Ganu, and Dan Roth. 2024. Evaluati...
2024 arXiv
-
[17]
arXiv preprint arXiv:2406.14219
Proving olympiad algebraic inequalities without human demonstrations. arXiv preprint arXiv:2406.14219. Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and S Yu Philip. 2023a. Multimodal large lan- guage models: A survey. In 2023 IEEE International Conference on Big Data (...
2023 arXiv
-
[19]
In Proceedings of the 1st Workshop on Large Generative Models Meet Multimodal Applications, pages 23–33
Multimodal data augmentation for image cap- tioning using diffusion models. In Proceedings of the 1st Workshop on Large Generative Models Meet Multimodal Applications, pages 23–33. Huajian Xin, Daya Guo, Zhihong Shao, Zhizhou Ren, Qihao Zhu, Bo Liu, Chong Ruan, Wenda Li, and X...
2024 arXiv
-
[20]
arXiv preprint arXiv:2309.05653
Mammoth: Building math generalist mod- els through hybrid instruction tuning. arXiv preprint arXiv:2309.05653. Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. 2024b. Mmmu-pro: A more robust multi-dis...
2024 arXiv
-
[21]
In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4571–4581
Jiuzhang: A chinese pre-trained language model for mathematical problem understanding. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4571–4581. Xin Zhao, Kun Zhou, Beichen Zhang, Zheng Gong, Zhipeng Chen, Yuanhang Zhou, Ji-Rong ...
2023 arXiv
-
[25]
Multilingual Support: While most models are focused on English, a few (e.g., MathGPT & Math-LLM) also support Chinese, showing a trend towards multilingual capabilities
-
[27]
Labeling Noise and Modality Alignment : Multimodal math problems often involve com- plex associations between text, formulas, and charts. Mismatches between text descriptions and images/formulas (e.g., incorrect axis la- bels, contradictions between geometry figures and proble...
2024
-
[28]
❷ Bottleneck in Data Diversity:
Lack of Deep Annotation for Problem Solv- ing Process: Most datasets only provide final answers, lacking intermediate steps such as algebraic transformations or construction of geometric auxiliary lines, making it difficult for models to learn the mathematical thinking chain (...
-
[29]
Limited Coverage of Problem Types and Sce- narios: Existing datasets are mostly focused on basic math areas (e.g., algebraic equations, simple geometry) and insufficiently cover higher-level math ( e.g., topology, discrete mathematics) or real-world scenarios ( e.g., physics m...
-
[30]
❸ Bottleneck in Data Scale:
Monotony in Multimodal Combination Pat- terns: Modal interactions are often simple con- catenations (e.g., text + static charts) without dynamic interactions (e.g., scalable geometric figures), or multi-step cross-modal reasoning (e.g., generating charts from text descriptions...
-
[31]
High Cost of High-Quality Data Acquisi- tion: Mathematical problems need to be de- signed by experts and ensure multimodal con- sistency, which leads to long production cy- cles and high costs. Additionally, there is data scarcity for long-tail problems ( e.g., niche branches ...
-
[32]
❹ Based on recent trends in the latest works, we further propose the following actionable sugges- tions to address these dataset bottlenecks:
Imbalance in Modal Data Volumes : Text data volumes far exceed those of image/sym- bol modalities, leading to models’ insuffi- cient feature extraction capability for non-text modalities. ❹ Based on recent trends in the latest works, we further propose the following actionable...
-
[33]
Innovation in Data Generation Techniques: Combine formal mathematical engines (e.g., Lean, Coq) to generate verifiable reasoning steps, use programmatic rendering tools (e.g., TikZ, GeoGebra) to automatically generate precise charts, and design semi-automated an- notation pipe...
-
[34]
Diversity Enhancement Strategies : Con- struct interdisciplinary, cross-cultural bench- mark datasets ( e.g., math-physics cross- domain problems), utilize crowdsourcing plat- forms to collect real-world scenario problems, and explore controllable data augmentation techniques,...
-
[35]
Scaling and Resource Integration: Encour- age collaborative dataset creation within the academic community (similar to ProofWiki), integrate existing educational resources (e.g., Khan Academy video-text analysis), and use synthetic data to fill long-tail gaps while im- proving...
-
[36]
spatial reason- ing
LLM as Enhancer: Generate mixed-domain problems (e.g., combining algebraic equations with geometric figures) to force the model to learn cross-domain associations. Explic- itly add domain labels ( e.g., "spatial reason- ing" label for geometry problems) to guide the model in d...
-
[37]
Use domain-specific few-shot examples (e.g., providing figure-text associations in geometry) to guide the model in switching reasoning modes
LLM as Reasoner: Fine-tune the model sep- arately for different mathematical domains (e.g., algebra, geometry) to learn domain- specific visual patterns (e.g., encoding geomet- ric properties in figures). Use domain-specific few-shot examples (e.g., providing figure-text assoc...
-
[38]
triangle
LLM as Planner : Based on the problem domain ( e.g., detecting the "triangle" key- word), call specialized tools ( e.g., geomet- ric theorem prover). For composite problems (e.g., algebraic-geometry equations), coordi- nate symbolic computation tools (e.g., Mathe- matica) and ...
-
[39]
The limitation is that labeling error types is costly and it’s difficult to cover all long-tail errors
LLM as Enhancer: Inject cross-modal errors (e.g., plot errors in function curves while the text description is correct) to train the model to detect contradictions. The limitation is that labeling error types is costly and it’s difficult to cover all long-tail errors
-
[40]
computation-logic-conclusion
LLM as Reasoner : Decompose reasoning into "computation-logic-conclusion" stages and cross-check text derivations with graphi- cal information (e.g., verify function extrema calculations using coordinates in the image). The limitation is that self-doubt relies on the model’s p...
-
[41]
The limitation is that tool invoca- tion delays affect real-time performance, and some errors require manually defined detec- tion rules
LLM as Planner : Use OCR tools to ex- tract symbols from figures and compare them with the text description to detect misunder- standings. The limitation is that tool invoca- tion delays affect real-time performance, and some errors require manually defined detec- tion rules. ...
2025
-
[42]
Enhancement of Multimodal Reasoning Chains: During reasoning, generate multi- step visual-symbol joint inference paths. For example, using CoT prompts to guide the model in decomposing geometric shapes into angle, side length, and other symbolic con- straints, and then calling...
2025
-
[43]
Visual-Symbol Alignment Verification: Use Best-of-N sampling to generate multiple can- didate diagram parsing results and call exter- nal OCR tools or geometry validators ( e.g., GeoGebra) to detect visual-text consistency and filter out erroneous explanations (Wu and Nakayama...
2025
-
[44]
If the training data coverage is insufficient, test-time strategies may not be able to compensate (Ke et al., 2025; Chen et al., 2025c)
Limitations: Parsing complex visual details (e.g., topological structures) depends on the pretrained visual encoder’s capabilities. If the training data coverage is insufficient, test-time strategies may not be able to compensate (Ke et al., 2025; Chen et al., 2025c). ❷ Limite...
2025
-
[45]
trian- gle
Dynamic Domain Routing: Use Beam Search Process Reward Model (PRM) to select domain-specific inference paths based on problem types ( e.g., detecting the “trian- gle” keyword and choosing between algebra solvers or geometry theorem provers) (Zhao et al., 2025; Zeng et al., 2025)
2025
-
[46]
Meta-learning Optimization: Fine-tune the model on a small number of domain-specific samples via Test-Time Training (TTT) to quickly adapt to new domains ( e.g., proba- bility and statistics problems)
-
[47]
❸ Error Feedback Limitations:
Limitations: Problems with blurred domain boundaries (e.g., math application problems involving common sense reasoning) may fail due to routing errors. ❸ Error Feedback Limitations:
-
[48]
Process Supervision Reinforcement: Use PRM to validate each step of reasoning in real- time. If an error is detected (e.g., misuse of in- tegration symbols), backtrack and correct the path; combine Self-Consistency by generat- ing multiple inference paths and selecting the one...
2025
-
[49]
Long-tail errors such as rare symbol confusions may be overlooked
Limitations: The reliability of PRM depends on the coverage of error types in the training data. Long-tail errors such as rare symbol confusions may be overlooked. In summary, combining the flexibility of test- time scaling with the specialization of multimodal tools can help ...
-
[147]
Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin
Frontiers Media SA. Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arXiv:2402.00157. Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel- Kedziorski, Yejin ...
2024 arXiv
-
[2019]
arXiv preprint arXiv:1905.13319
Mathqa: Towards interpretable math word problem solving with operation-based formalisms. arXiv preprint arXiv:1905.13319. Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. 2023. ...
1905 arXiv
-
[2021]
arXiv preprint arXiv:2107.13435
Mwp-bert: Numeracy-augmented pre-training for math word problem solving. arXiv preprint arXiv:2107.13435. Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Ye- long Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, et al. 2024. Rho-1: Not all tokens are what you need....
2024 arXiv
-
[2022]
Educational Research Review, 37:100480
Embodied learning and teaching approaches in language education: A mixed studies review. Educational Research Review, 37:100480. Jikun Kang, Xin Zhe Li, Xi Chen, Amirreza Kazemi, Qianyi Sun, Boxing Chen, Dong Li, Xu He, Quan He, Feng Wen, et al. 2024. Mindstar: Enhancing math ...
2024 arXiv
-
[2023]
Advances in Neural Information Processing Systems, 36:5539–5568
Openagi: When llm meets domain ex- perts. Advances in Neural Information Processing Systems, 36:5539–5568. Philippe Gervais, Asya Fadeeva, and Andrii Maksai
-
[2024]
arXiv preprint arXiv:2406.15736
Evaluating large vision-and-language mod- els on children’s mathematical olympiads. arXiv preprint arXiv:2406.15736. Ethan Chern, Haoyang Zou, Xuefeng Li, Jiewen Hu, Ke- hua Feng, Junlong Li, and Pengfei Liu. 2023. Gen- erative ai for math: Abel. https://github.com/ GAIR-NLP/a...
2023 arXiv
-
[2025]
arXiv preprint arXiv:2501.04686
Ursa: Understanding and verifying chain-of- thought reasoning in multimodal mathematics. arXiv preprint arXiv:2501.04686. Jingkun Ma, Runzhe Zhan, Derek F Wong, Yang Li, Di Sun, Hou Pong Chan, and Lidia S Chao. 2024. Vi- saidmath: Benchmarking visual-aided mathematical reasoni...
2024
-
[2256]
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kam- badur, David Rosenberg, and Gideon Mann
IEEE. Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kam- badur, David Rosenberg, and Gideon Mann. 2023b. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564. Ting Wu, Xuefeng Li, and Pengfei Liu. ...
2025 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.