Pith. sign in

REVIEW 3 major objections 3 minor 2 cited by

We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read We-Math 2.0 ties a structured math knowledge hierarchy, difficulty-graded data, and reinforcement learning into one system for improving multimodal mathematical reasoning.

desk verdict The submission is an abstract for a math-reasoning dataset paper glued to an unrelated SAM/tensor paper; there is no actual We-Math 2.0 content to review. read the letter →

arxiv 2508.10433 v1 pith:Y3SJI3VK submitted 2025-08-14 cs.AI cs.CVcs.LG

Runqi Qiao , Qiuna Tan , Peiqing Yang , Yanzi Wang , Xiaowan Wang , Enhui Wan , Sitong Zhou , Guanting Dong
show 6 more authors
Yuchen Zeng Yida Xu Jie Wang Chong Sun Chen Li Honggang Zhang
This is my paper
classification cs.AIcs.CVcs.LG
keywords multimodallargelanguagemodelsmathematicalreasoningreinforcementlearningknowledgehierarchycurriculumbenchmarkdifficultymodelingvisual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal large language models (MLLMs) can read diagrams and formulas but still stumble on multi-step mathematical reasoning. We-Math 2.0 is a unified system designed to fix that: it organizes visual math into a five-level knowledge hierarchy of 491 knowledge points and 1,819 fundamental principles, converts that hierarchy into training datasets with explicit difficulty variation, and trains models with a two-stage reinforcement-learning curriculum. The paper's central claim is that this knowledge-first structure, not merely more data, is what improves mathematical reasoning, and it reports that the resulting model, MathBook-RL, is competitive on four established benchmarks and strong on its own coverage-based benchmark, MathBookEval. A sympathetic reader takes this as evidence that structured knowledge and progressive RL alignment can transfer beyond the training distribution.

What carries the argument

The central object is the MathBook five-level knowledge hierarchy: 491 knowledge points and 1,819 fundamental principles that define the mathematical skills the system teaches and tests. Around it, the machinery includes the three-dimensional difficulty space that produces seven progressive variants per problem for MathBook-Pro; the two-stage RL paradigm (cold-start fine-tuning plus progressive alignment RL with average-reward learning and dynamic data scheduling); and MathBookEval, the benchmark that evaluates every knowledge point with varied reasoning-step lengths.

What would settle it

Conduct a held-out transfer experiment: evaluate MathBook-RL on a set of math problems whose solutions require knowledge points not represented in the 491-point hierarchy, or train a control model on the same volume of problems with knowledge labels and difficulty ordering removed. If the control matches MathBook-RL on MathBookEval and external benchmarks, or if the model fails specifically on outside-hierarchy problems, the system's core claim of knowledge-driven generalization fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a structured mathematical knowledge system can serve as the organizing spine for both data construction and training of multimodal reasoners. The MathBook hierarchy anchors MathBook-Standard, built by dual expansion to cover all 491 knowledge points, and MathBook-Pro, which defines a three-dimensional difficulty space and generates seven progressively harder variants of each problem. On top of these, MathBook-RL first aligns the model to knowledge-oriented chain-of-thought reasoning through cold-start fine-tuning, then applies progressive-alignment RL that uses average-reward learning and dynamic data scheduling across difficulty levels. The em

Load-bearing premise

The load-bearing premise is that the five-level MathBook hierarchy, with its 491 knowledge points and 1,819 principles, completely and correctly represents the mathematical reasoning skills MLLMs need; if that map is incomplete or mislabeled, the datasets, the RL curriculum, and the benchmark built on it all inherit the error.

Editorial extensions

If this is right

  • A model trained through MathBook-RL should generalize to external math benchmarks, since the hierarchy and curriculum rather than benchmark-specific cues drive its gains.
  • Failure analysis becomes possible: because MathBookEval covers every knowledge point, a model's errors can be traced to missing or weak knowledge nodes.
  • The released Standard and Pro datasets give other researchers ready-made knowledge-structured and difficulty-graded training material for multimodal math reasoning.
  • The progressive-alignment result implies that RL curricula for reasoning should order data by difficulty instead of sampling uniformly.
  • MathBookEval offers a knowledge-coverage-aware evaluation axis complementary to existing aggregate benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the five-level hierarchy is the load-bearing taxonomy; if it omits or mislabels reasoning skills that real problems exercise, the training gains and benchmark scores would measure the taxonomy's internal consistency rather than general mathematical ability.
  • Editorial extension: the design could be transferred to other enumerable reasoning domains, such as scientific diagram comprehension or formal proof search, by replacing the math knowledge points with the target domain's knowledge structure.
  • Editorial extension: a direct test of the hierarchy's necessity would be to train a control on the same problems without knowledge-point labels and compare transfer performance; the abstract does not report such a control.
  • Observation: the full-text content bundled with this submission is a different manuscript (on sharpness-aware minimization in tensorized models), so the empirical support for We-Math 2.0 cannot be verified from the supplied full text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The submitted manuscript, arXiv:2508.10433, presents an abstract for "We-Math 2.0," a proposed unified system for visual mathematical reasoning in MLLMs. The abstract describes four contributions: (1) a five-level MathBook knowledge hierarchy with 491 knowledge points and 1,819 principles; (2) MathBook-Standard and MathBook-Pro datasets, including seven progressive difficulty variants per problem; (3) MathBook-RL, a two-stage RL framework with cold-start fine-tuning and progressive alignment; and (4) MathBookEval, a benchmark spanning all 491 knowledge points. The abstract claims that MathBook-RL performs competitively on four widely used benchmarks and strongly on MathBookEval. However, the body of the submitted artifact is an unrelated paper, "Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models," with no MLLM experiments, dataset description, training details, or evaluation results. The empirical claims of We-Math 2.0 are therefore unsupported by the reviewed manuscript.

Significance. If fully realized, the proposed system could be significant: a knowledge-structured curriculum with difficulty-controlled progressive variants and an RL training scheme would address a real gap in multimodal mathematical reasoning. The abstract also makes a falsifiable claim about generalization to external benchmarks. However, none of this significance can be assessed from the submitted artifact. There are no machine-checked proofs, reproducible code, data, or numerical tables. The internal MathBookEval benchmark is described as covering the same 491 knowledge points used to construct the training data, so without item-level contamination controls the reported "strong results" cannot be separated from training-set overlap. The manuscript as submitted provides no evidential basis for the central claims.

major comments (3)
  1. [Full text (entire manuscript)] The submitted artifact contains only an abstract for We-Math 2.0; the body is a different paper on Sharpness-Aware Minimization in tensorized models. There are no MLLM experiments, benchmark protocols, baselines, ablations, dataset statistics, training curves, or evaluation tables. The abstract's central empirical sentence—"MathBook-RL performs competitively with existing baselines on four widely-used benchmarks and achieves strong results on MathBookEval"—is therefore unsupported. This is a load-bearing omission: the claim is empirical, and the reviewed manuscript does not contain the evidence.
  2. [Abstract, contribution (4) and experimental results] MathBookEval is described as covering all 491 knowledge points, the same knowledge hierarchy used to structure MathBook-Standard and MathBook-Pro. Since the training data and evaluation benchmark share their construction schema, "strong results on MathBookEval" may reflect overlap with the training distribution. The manuscript provides no item-level overlap analysis, contamination controls, or evidence that eval problems are disjoint from training variants. This circularity concern is not resolvable from the submitted text and directly affects the generalization claim.
  3. [Abstract, contribution (1)] The five-level knowledge hierarchy with 491 knowledge points and 1,819 principles is asserted without any description of its construction methodology, validation, or coverage analysis. All downstream components—the datasets, the difficulty space, the RL curriculum, and MathBookEval—depend on this hierarchy. Without evidence that the hierarchy is complete and correctly organized, the reported performance gains cannot be attributed to general mathematical reasoning rather than to properties of the specific ontology.
minor comments (3)
  1. [Abstract] The four "widely-used benchmarks" are not named in the abstract or anywhere in the submitted text, making the competitive claim untestable even at the level of benchmark selection.
  2. [Abstract, contribution (2)] The "three-dimensional difficulty space" and the mechanism for generating "7 progressive variants per problem" are not defined. No examples or statistics are provided, so the reader cannot judge the difficulty scaling methodology.
  3. [Abstract, contribution (3)] The two-stage RL framework is described at a high level only. Key details such as the base model, reward function, RL algorithm, and hyperparameter settings are absent. This is a presentation issue given that the main body is missing; these details must appear in any complete submission.

Circularity Check

2 steps flagged · score 6.0 of 10

MathBookEval is built from the same 491 knowledge points that structure the training data, and the submitted body contains none of the claimed external-benchmark experiments.

  1. self definitional [Abstract, contributions (1), (2), (4) and the final experimental-results sentence]
    "(4) MathBookEval: We introduce a comprehensive benchmark covering all 491 knowledge points with diverse reasoning step distributions. Experimental results show that MathBook-RL performs competitively with existing baselines on four widely-used benchmarks and achieves strong results on MathBookEval, suggesting promising generalization in mathematical reasoning."

    MathBookEval is constructed from the same 491 knowledge points (contribution 1) that determine the structure of MathBook-Standard/Pro training data (contribution 2) and drive the RL curriculum (contribution 3). Therefore 'strong results on MathBookEval' measure the model's alignment with the very taxonomy used to generate its training data. The abstract reports no independent item construction, no held-out knowledge-point split, and no contamination control for this in-house benchmark; the evaluation and training sets share their defining labels by construction, making the in-house result partially self-referential.

  2. other [Full text (the body of the submitted artifact)]
    "Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models ... In this work, we leverage scale-invariance to analyze the norm dynamics of SAM in general tensorized models."

    This is not a circularity step but a missing-evidence flag. The submitted full text is an unrelated arXiv paper (2508.10435) about SAM and tensorized models; it contains no MathBook system, no MathBook-Standard/Pro datasets, no MathBook-RL training, and no benchmark evaluation. Consequently, the abstract's claim of competitive performance on 'four widely-used benchmarks' is unsupported by any tables, baselines, protocols, or ablations in the reviewed artifact. This absence removes the external-benchmark evidence that would otherwise mitigate the self-referential nature of the MathBookEval result.

full rationale

The central circularity is confined to the in-house benchmark: MathBookEval covers 'all 491 knowledge points,' and the training pipeline (MathBook-Standard, MathBook-Pro, MathBook-RL) is explicitly organized around that same five-level knowledge hierarchy. Thus the reported 'strong results on MathBookEval' are, at least in part, an evaluation on the same conceptual grid that generated the training signal; this is a partial self-definitional circularity rather than an independent external validation. No equations in the abstract reduce one another, and there is no self-citation chain or imported uniqueness theorem. The body-text mismatch is not itself a circular derivation, but it removes the only independent empirical check named in the abstract (the four widely-used benchmarks), so the self-referential in-house result becomes the only concrete evidence presented. On the 0-10 scale, this is a partial circularity of the kind described by score 6: one or more reported outcomes reduce by construction to the shared knowledge-point taxonomy, while the method's broader design (RL, difficulty scaling, CoT alignment) still has independent content that cannot be assessed from the submitted artifact.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

Because the submission is an abstract-only artifact after the body mismatch, internal parameters cannot be audited. Listed are the design choices that function as free parameters and the domain assumptions that carry the abstract's claims. No invented physical or theoretical entities are proposed; the datasets and benchmark are constructed artifacts, not postulates of new mechanisms.

free parameters (4)
  • Knowledge hierarchy size (491 knowledge points, 1,819 principles) = 491 KPs / 1,819 principles
    Chosen by the authors to define the MathBook knowledge system; the abstract gives no external validation or selection criterion for these counts.
  • Number of progressive variants per problem = 7
    Design choice for MathBook-Pro; the abstract does not justify why seven variants or how the difficulty levels are calibrated.
  • Three-dimensional difficulty space
    The difficulty definition is a hand-designed construction that determines the progressive variants and the RL scheduling; no validation or rubric is given in the abstract.
  • RL hyperparameters (cold-start fine-tuning and progressive alignment stages)
    No details provided; average-reward learning and dynamic data scheduling require hyperparameters that are not disclosed in the abstract or body.
assumptions (4)
  • domain assumption The five-level knowledge hierarchy of 491 knowledge points and 1,819 principles is a complete and correct model of the mathematical reasoning MLLMs need.
    The entire system (Standard/Pro datasets, RL curriculum, MathBookEval) is derived from this hierarchy (abstract contribution 1). If it is wrong or incomplete, the reported gains are artifacts of construction.
  • domain assumption The dual-expansion procedure ensures broad conceptual coverage and correct annotations.
    The abstract claims coverage and flexibility 'through dual expansion' but provides no label-quality audit or coverage statistics.
  • ad hoc to paper Average-reward learning with dynamic data scheduling yields progressive alignment across difficulty levels.
    This is the paper's proposed mechanism (abstract contribution 3); asserted without derivation or ablation in the provided text.
  • domain assumption MathBookEval is a valid external measure of mathematical reasoning.
    As a benchmark built from the same knowledge system used for training, its independence is assumed, not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning." pith.science (2026). https://pith.science/paper/Y3SJI3VK

@misc{pith2026250810433,
  author       = {Pith},
  title        = {Pith review of: We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y3SJI3VK}},
  note         = {Machine review of arXiv:2508.10433}
}
read the original abstract

Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities across various tasks, but still struggle with complex mathematical reasoning. Existing research primarily focuses on dataset construction and method optimization, often overlooking two critical aspects: comprehensive knowledge-driven design and model-centric data space modeling. In this paper, we introduce We-Math 2.0, a unified system that integrates a structured mathematical knowledge system, model-centric data space modeling, and a reinforcement learning (RL)-based training paradigm to comprehensively enhance the mathematical reasoning abilities of MLLMs. The key contributions of We-Math 2.0 are fourfold: (1) MathBook Knowledge System: We construct a five-level hierarchical system encompassing 491 knowledge points and 1,819 fundamental principles. (2) MathBook-Standard & Pro: We develop MathBook-Standard, a dataset that ensures broad conceptual coverage and flexibility through dual expansion. Additionally, we define a three-dimensional difficulty space and generate 7 progressive variants per problem to build MathBook-Pro, a challenging dataset for robust training. (3) MathBook-RL: We propose a two-stage RL framework comprising: (i) Cold-Start Fine-tuning, which aligns the model with knowledge-oriented chain-of-thought reasoning; and (ii) Progressive Alignment RL, leveraging average-reward learning and dynamic data scheduling to achieve progressive alignment across difficulty levels. (4) MathBookEval: We introduce a comprehensive benchmark covering all 491 knowledge points with diverse reasoning step distributions. Experimental results show that MathBook-RL performs competitively with existing baselines on four widely-used benchmarks and achieves strong results on MathBookEval, suggesting promising generalization in mathematical reasoning.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SpaceVista: All-Scale Visual Spatial Reasoning from mm to km

    cs.CV 2025-10 conditional novelty 7.0 of 10

    SpaceVista contributes a 1M-QA, 38K-video all-scale spatial reasoning dataset spanning mm to km, a manually verified benchmark, and a fine-tuned 7B MLLM with scale experts and progressive reward training.

  2. When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Moving a question from typed prompt to pixels drops MLLM accuracy by 17.8 points on average, and prompt-region grounding recovers 8.3 points of the four-benchmark VTS loss at matched training cost.

Reference graph

Works this paper leans on

79 extracted references · 71 canonical work pages · cited by 2 Pith papers

  1. [1]

    Akiba, T.; Sano, S.; Yanase, T.; Ohta, T.; and Koyama, M. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, 2623–2631

    work page 2019

  2. [2]

    Andriushchenko, M.; Croce, F.; M¨uller, M.; Hein, M.; and Flammarion, N. 2023. A modern look at the rela- tionship between sharpness and generalization. arXiv preprint arXiv:2302.07011

    arXiv 2023

  3. [3]

    Andriushchenko, M.; and Flammarion, N. 2022. To- wards understanding sharpness-aware minimization. In International Conference on Machine Learning, 639–

    work page 2022

  4. [4]

    Arora, S.; Cohen, N.; Hu, W.; and Luo, Y . 2019. Im- plicit regularization in deep matrix factorization. Ad- vances in Neural Information Processing Systems, 32

    work page 2019

  5. [5]

    Z.; and Raghunathan, A

    Baek, C.; Kolter, J. Z.; and Raghunathan, A. 2024. Why is SAM Robust to Label Noise? InThe Twelfth In- ternational Conference on Learning Representations

    work page 2024

  6. [6]

    Bahri, D.; Mobahi, H.; and Tay, Y . 2022. Sharpness- Aware Minimization Improves Language Model Gen- eralization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), 7360–7371

    work page 2022

  7. [7]

    Barrett, D.; and Dherin, B. 2021. Implicit Gradient Regularization. In International Conference on Learn- ing Representations

    work page 2021

  8. [8]

    L.; Long, P

    Bartlett, P. L.; Long, P. M.; and Bousquet, O. 2023. The dynamics of sharpness-aware minimization: Bouncing across ravines and drifting towards wide minima. Jour- nal of Machine Learning Research, 24(316): 1–36

    work page 2023

Show all 79 references
  1. [9]

    Belkin, M.; Hsu, D.; Ma, S.; and Mandal, S. 2019. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences , 116(32): 15849– 15854

    2019

  2. [10]

    Bisla, D.; Wang, J.; and Choromanska, A. 2022. Low- pass filtering sgd for recovering flat optima in the deep learning optimization landscape. In International Con- ference on Artificial Intelligence and Statistics, 8299–

    2022

  3. [11]

    H.; and Mamitsuka, H

    Cao, T.; Sun, L.; Nguyen, C. H.; and Mamitsuka, H

  4. [12]

    M.; Biggio, L.; Orvieto, A.; Proske, F

    Compagnoni, E. M.; Biggio, L.; Orvieto, A.; Proske, F. N.; Kersting, H.; and Lucchi, A. 2023. An sde for modeling sam: Theory and insights. In Inter- national Conference on Machine Learning , 25209– 25253. PMLR

    2023

  5. [13]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Com- puter Vision and Pattern Recognition, 248–255. Ieee

    2009

  6. [14]

    Deng, J.; Pang, J.; Zhang, B.; and Guo, G. 2025. Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 16208–16216

    2025

  7. [15]

    T.; Zhen, L.; Goh, R

    Du, J.; Yan, H.; Feng, J.; Zhou, J. T.; Zhen, L.; Goh, R. S. M.; and Tan, V . Y . 2021. Efficient sharpness-aware minimization for improved training of neural networks. arXiv preprint arXiv:2110.03141

    2021 arXiv

  8. [16]

    Du, J.; Zhou, D.; Feng, J.; Tan, V .; and Zhou, J. T. 2022. Sharpness-aware training for free. Advances in Neural Information Processing Systems, 35: 23439–23451

    2022

  9. [17]

    S.; Hu, W.; and Lee, J

    Du, S. S.; Hu, W.; and Lee, J. D. 2018. Algorithmic regularization in learning deep homogeneous models: Layers are automatically balanced.Advances in Neural Information Processing Systems, 31

    2018

  10. [18]

    Dua, D.; Wang, Y .; Dasigi, P.; Stanovsky, G.; Singh, S.; and Gardner, M. 2019. DROP: A Reading Comprehen- sion Benchmark Requiring Discrete Reasoning Over Paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Ling...

    2019

  11. [19]

    Foret, P.; Kleiner, A.; Mobahi, H.; and Neyshabur, B. 2020. Sharpness-aware minimization for effi- ciently improving generalization. arXiv preprint arXiv:2010.01412

    2020 arXiv

  12. [20]

    Ge, R.; Huang, F.; Jin, C.; and Yuan, Y . 2015. Es- caping from saddle points—online stochastic gradient for tensor decomposition. In Conference on learning theory, 797–842. PMLR

    2015

  13. [21]

    E.; Bhojanapalli, S.; Neyshabur, B.; and Srebro, N

    Gunasekar, S.; Woodworth, B. E.; Bhojanapalli, S.; Neyshabur, B.; and Srebro, N. 2017. Implicit regular- ization in matrix factorization. Advances in Neural Information Processing Systems, 30

    2017

  14. [22]

    Hariz, K.; Kadri, H.; Ayache, S.; Moakher, M.; and Artieres, T. 2022. Implicit regularization with polyno- mial growth in deep tensor factorization. In Interna- tional Conference on Machine Learning, 8484–8501. PMLR

    2022

  15. [23]

    Hariz, K.; Kadri, H.; Ayache, S.; Moakher, M.; and Arti`eres, T. 2024. Implicit regularization in deep tucker factorization: Low-rankness via structured sparsity. In International Conference on Artificial Intelligence and Statistics, 2359–2367. PMLR

    2024

  16. [24]

    Hayashi, K.; Yamaguchi, T.; Sugawara, Y .; and Maeda, S.-i. 2019. Exploring unexplored tensor network de- compositions for convolutional neural networks. Ad- vances in Neural Information Processing Systems, 32

    2019

  17. [25]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778

    2016

  18. [26]

    Hrinchuk, O.; Khrulkov, V .; Mirvakhabova, L.; Orlova, E.; and Oseledets, I. 2020. Tensorized Embedding Lay- ers. In Findings of the Association for Computational Linguistics: EMNLP 2020, 4847–4860

    2020

  19. [27]

    J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y .; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y .; Wang, S.; Wang, L.; and Chen, W. 2022. LoRA: Low- Rank Adaptation of Large Language Models. In Inter- national Conference on Learning Representations

    2022

  20. [28]

    Ilbert, R.; Odonnat, A.; Feofanov, V .; Virmaux, A.; Paolo, G.; Palpanas, T.; and Redko, I. 2024. SAM- former: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Min- imization and Channel-Wise Attention. In Inter- national Conference on Ma...

    2024

  21. [29]

    Ishida, T.; Yamane, I.; Sakai, T.; Niu, G.; and Sugiyama, M. 2020. Do We Need Zero Training Loss After Achieving Zero Training Error? In International Conference on Machine Learning, 4604–4614. PMLR

    2020

  22. [30]

    Ji, J.; Li, G.; Fu, J.; Afghah, F.; Guo, L.; Yuan, X.; and Ma, X. 2024. A single-step, sharpness-aware mini- mization is all you need to achieve efficient and accu- rate sparse training. Advances in Neural Information Processing Systems, 37: 44269–44290

    2024

  23. [31]

    Jiang, L.; Huang, D.; Liu, M.; and Yang, W. 2020. Beyond synthetic noise: Deep learning on controlled noisy labels. In International Conference on Machine Learning, 4804–4815. PMLR

    2020

  24. [32]

    Jie, S.; and Deng, Z.-H. 2023. Fact: Factor-tuning for lightweight adaptation on vision transformer. In Proceedings of the AAAI conference on artificial intel- ligence, volume 37, 1060–1068

    2023

  25. [33]

    X.; and Hospedales, T

    Kim, M.; Li, D.; Hu, S. X.; and Hospedales, T. 2022. Fisher sam: Information geometry and sharpness aware minimisation. In International Conference on Machine Learning, 11148–11161. PMLR

    2022

  26. [34]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2015. Adam: A Method for Stochastic Optimization. In ICLR (Poster)

    2015

  27. [35]

    G.; and Bader, B

    Kolda, T. G.; and Bader, B. W. 2009. Tensor decompo- sitions and applications. SIAM review, 51(3): 455–500

    2009

  28. [36]

    Kossaifi, J.; Panagakis, Y .; Anandkumar, A.; and Pan- tic, M. 2019. Tensorly: Tensor learning in python. Journal of Machine Learning Research, 20(26): 1–6

    2019

  29. [37]

    Krizhevsky, A.; Hinton, G.; et al. 2009. Learning multiple layers of features from tiny images

    2009

  30. [38]

    Kwon, J.; Kim, J.; Park, H.; and Choi, I. K. 2021. Asam: Adaptive sharpness-aware minimization for scale-invariant learning of deep neural networks. In International Conference on Machine Learning, 5905–

    2021

  31. [39]

    Li, B.; Zhang, L.; and He, N. 2024. Implicit regu- larization of sharpness-aware minimization for scale- invariant problems. Advances in Neural Information Processing Systems, 37: 44444–44478

    2024

  32. [40]

    Li, N.; Pan, Y .; Chen, Y .; Ding, Z.; Zhao, D.; and Xu, Z. 2022. Heuristic rank selection with progressively searching tensor ring network. Complex & Intelligent Systems, 8(2): 771–785

    2022

  33. [41]

    Li, T.; He, Z.; Li, Y .; Wang, Y .; Shang, L.; and Huang, X. 2025. Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape. In Forty-second International Conference on Machine Learning

    2025

  34. [42]

    Li, Z.; Chen, S.; Yang, J.; and Luo, L. 2024. Efficiency calibration of implicit regularization in deep networks via self-paced curriculum-driven singular value selec- tion. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24,...

    2024

  35. [43]

    Liu, T.; Li, Y .; Wei, S.; Zhou, E.; and Zhao, T. 2021. Noisy gradient descent converges to flat minima for nonconvex matrix factorization. In International Con- ference on Artificial Intelligence and Statistics, 1891–

    2021

  36. [44]

    Liu, Y .; Mai, S.; Cheng, M.; Chen, X.; Hsieh, C.-J.; and You, Y . 2022. Random sharpness-aware minimization. Advances in Neural Information Processing Systems, 35: 24543–24556

    2022

  37. [45]

    Liu, Y .; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V

  38. [46]

    B.; Wang, D.; Leth-Espensen, C

    Loeschcke, S. B.; Wang, D.; Leth-Espensen, C. M.; Belongie, S.; Kastoryano, M.; and Benaim, S. 2024. Coarse-To-Fine Tensor Trains for Compact Visual Rep- resentations. In International Conference on Machine Learning, 32612–32642. PMLR

    2024

  39. [47]

    Loshchilov, I.; and Hutter, F. 2019. Decoupled Weight Decay Regularization. In International Conference on Learning Representations

    2019

  40. [48]

    D.; Chen, D.; and Arora, S

    Malladi, S.; Gao, T.; Nichani, E.; Damian, A.; Lee, J. D.; Chen, D.; and Arora, S. 2023. Fine-tuning lan- guage models with just forward passes. Advances in Neural Information Processing Systems, 36: 53038– 53075

    2023

  41. [49]

    Memmel, E.; Menzen, C.; Schuurmans, J.; Wesel, F.; and Batselier, K. 2024. Position: Tensor Networks are a Valuable Asset for Green AI. In International Con- ference on Machine Learning, 35340–35353. PMLR

    2024

  42. [50]

    Mueller, M.; Vlaar, T.; Rolnick, D.; and Hein, M. 2023. Normalization layers are all that sharpness-aware min- imization needs. Advances in Neural Information Pro- cessing Systems, 36: 69228–69252

    2023

  43. [51]

    Neyshabur, B.; Bhojanapalli, S.; McAllester, D.; and Srebro, N. 2017. Exploring generalization in deep learning. Advances in Neural Information Processing Systems, 30

    2017

  44. [52]

    Novikov, A.; Podoprikhin, D.; Osokin, A.; and Vetrov, D. P. 2015. Tensorizing neural networks. Advances in Neural Information Processing Systems, 28

    2015

  45. [53]

    Pan, Y .; Su, Z.; Liu, A.; Jingquan, W.; Li, N.; and Xu, Z. 2022. A unified weight initialization paradigm for tensorial convolutional neural networks. In Interna- tional conference on machine learning, 17238–17257. PMLR

    2022

  46. [54]

    Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32

    2019

  47. [55]

    Phan, A.-H.; Sobolev, K.; Sozykin, K.; Ermilov, D.; Gusak, J.; Tichavsk `y, P.; Glukhov, V .; Oseledets, I.; and Cichocki, A. 2020. Stable low-rank tensor de- composition for compression of convolutional neural network. In European Conference on Computer Vision, 522–539. Springer

    2020

  48. [56]

    Rajpurkar, P.; Zhang, J.; Lopyrev, K.; and Liang, P

  49. [57]

    Razin, N.; Maman, A.; and Cohen, N. 2022. Im- plicit regularization in hierarchical tensor factoriza- tion and deep convolutional neural networks. In In- ternational Conference on Machine Learning, 18422– 18462. PMLR

    2022

  50. [58]

    Si, C.; Wang, X.; Yang, X.; Xu, Z.; Li, Q.; Dai, J.; Qiao, Y .; Yang, X.; and Shen, W. 2025. Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning. In The Thirteenth International Conference on Learning Representations

    2025

  51. [59]

    N.; and Topin, N

    Smith, L. N.; and Topin, N. 2019. Super-convergence: Very fast training of neural networks using large learn- ing rates. In Artificial intelligence and machine learn- ing for multi-domain operations applications, volume 11006, 369–386. SPIE

    2019

  52. [60]

    Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1): 1929–1958

    2014

  53. [61]

    Van Rooyen, B.; Menon, A.; and Williamson, R. C

  54. [62]

    Veeramacheneni, L.; Wolter, M.; Kuehne, H.; and Gall, J. 2025. Canonical Rank Adaptation: An Efficient Fine-Tuning Strategy for Vision Transformers. In Forty-second International Conference on Machine Learning

    2025

  55. [63]

    Wang, A.; Pruksachatkun, Y .; Nangia, N.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in Neural Information Processing Systems, 32

    2019

  56. [64]

    Wang, A.; Qiu, Y .; Bai, M.; Jin, Z.; Zhou, G.; and Zhao, Q. 2024. Generalized tensor decomposition for under- standing multi-output regression under combinatorial shifts. Advances in Neural Information Processing Systems, 37: 47559–47635

    2024

  57. [65]

    Wang, W.; Sun, Y .; Eriksson, B.; Wang, W.; and Ag- garwal, V . 2018. Wide compression: Tensor ring nets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 9329–9338

    2018

  58. [66]

    Wei, C.; Kakade, S.; and Ma, T. 2020. The implicit and explicit regularization effects of dropout. In In- ternational Conference on Machine Learning, 10181– 10192. PMLR

    2020

  59. [67]

    Wen, K.; Ma, T.; and Li, Z. 2022. How sharpness- aware minimization minimizes sharpness? In The Eleventh International Conference on Learning Repre- sentations

    2022

  60. [68]

    Xie, W.; Pethick, T.; and Cevher, V . 2024. Sampa: Sharpness-aware minimization parallelized. Advances in Neural Information Processing Systems, 37: 51333– 51357

    2024

  61. [69]

    Yang, Y .; Zhou, J.; Wong, N.; and Zhang, Z. 2024. LoRETTA: Low-Rank Economic Tensor-Train Adap- tation for Ultra-Low-Parameter Fine-Tuning of Large Language Models. In Proceedings of the 2024 Con- ference of the North American Chapter of the Associa- tion for Computational Li...

    2024

  62. [70]

    Yaras, C.; Wang, P.; Balzano, L.; and Qu, Q. 2024. Compressible Dynamics in Deep Overparameter- ized Low-Rank Learning & Adaptation. In Inter- national Conference on Machine Learning , 56946– 56965. PMLR

    2024

  63. [71]

    Yin, M.; Sui, Y .; Liao, S.; and Yuan, B. 2021. Towards efficient tensor decomposition-based dnn model com- pression with optimization framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10674–10683

    2021

  64. [72]

    Zhang, C.; Bengio, S.; Hardt, M.; Recht, B.; and Vinyals, O. 2021. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3): 107–115

    2021

  65. [73]

    V .; et al

    Zhang, S.; Roller, S.; Goyal, N.; Artetxe, M.; Chen, M.; Chen, S.; Dewan, C.; Diab, M.; Li, X.; Lin, X. V .; et al

  66. [74]

    Zhao, Q.; Zhou, G.; Xie, S.; Zhang, L.; and Cichocki, A. 2016. Tensor ring decomposition. arXiv preprint arXiv:1606.05535

    2016 arXiv

  67. [2015]

    Advances in Neural Infor- mation Processing Systems, 28

    Learning with symmetric label noise: The im- portance of being unhinged. Advances in Neural Infor- mation Processing Systems, 28

  68. [2016]

    In Proceedings of the 2016 Con- ference on Empirical Methods in Natural Language Processing, 2383–2392

    SQuAD: 100,000+ Questions for Machine Com- prehension of Text. In Proceedings of the 2016 Con- ference on Empirical Methods in Natural Language Processing, 2383–2392

    2016

  69. [2019]

    arXiv preprint arXiv:1907.11692

    Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    1907 arXiv

  70. [2022]

    arXiv preprint arXiv:2205.01068

    Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    arXiv

  71. [2024]

    In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, 3780–3788

    Learning low-rank tensor cores with probabilis- tic L0-regularized rank selection for model compres- sion. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, 3780–3788

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.