Pith. sign in

REVIEW 3 major objections 6 minor 250 references

A list of formulas for π, e, γ, Catalan's constant and zeta values is offered as a numerical-checkable test of whether AI can prove or rediscover identities in the Ramanujan–Apéry tradition.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 17:07 UTC pith:O2OIKFWR

load-bearing objection Clean, usable AI-math challenge set of new special-value formulas; the encrypted proofs are a temporary but real soft spot, not a fatal one. the 3 major comments →

arxiv 2607.09721 v1 pith:O2OIKFWR submitted 2026-06-27 math.HO cs.AImath.NT

The Ramanujan Challenge For AI

Michael Shalyt , Rotem Kalisch , Carsten Schneider , Hila Barkan , Elyasheev Leibtag , John Campbell , Shachar Weinbaum , Tali Monderer
show 2 more authors
Ashvni Narayanan Ido Kaminer
This is my paper
classification math.HO cs.AImath.NT MSC 11Y6068T2033F1011J82
keywords Ramanujan-type formulasApéry limitsmathematical constantsAI evaluationcontinued fractionszeta valuesCatalan's constantknot polynomials
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a focused set of explicit formulas for classical constants—polynomial continued fractions, Apéry-style limits, holonomic recurrences, series of harmonic numbers, matrix products, and one integral over knot-polynomial roots. Each formula can be verified to thousands of digits, yet converting the numerical agreement into a proof may require non-obvious mathematics from several domains. Roughly half the statements already possess proofs known to the authors (kept encrypted for an initial period); the rest are high-precision conjectures. The authors argue that this combination supplies research-level problems free of training-data contamination, thereby giving a concrete way to measure whether AI systems can move from numerical discovery to genuine mathematical argument. Success would show that machines can rediscover or invent structures that historically required human insight, while failure would mark the present boundary of that capability.

Core claim

The paper claims that the listed identities hold: a polynomial continued fraction equals 6/(3−π), two linear recurrences have Apéry limits equal to Euler's constant γ and to π+e, a double sum of squared binomial coefficients times squared harmonic numbers equals a closed combination of polylogarithms and zeta values, matrix products and four-term recurrences converge to Catalan's constant and to ζ(2)+ζ(3), a 4 imes4 matrix product converges extremely rapidly to √10005/π, an integral of a Godbillon–Vey form over a branch of the A-polynomial of the knot 7₂ equals 4π²/85, and the gcd of the scaled Apéry numbers is of the form e^{o(n)}. These statements, some already proved and some still open,

What carries the argument

Apéry-style limits and related holonomic recurrences (or matrix products) whose successive rational terms converge to the target constant; numerical verification to arbitrary precision supplies objective evidence while the proof itself may demand creative identification of the underlying arithmetic or geometric structure.

Load-bearing premise

That the proofs already known to the authors are correct and that the high-precision numerical matches for the open formulas will eventually admit rigorous proofs rather than remaining coincidences.

What would settle it

A computer-algebra derivation or formal proof that any one of the claimed identities fails, or a rigorous demonstration that the integral over the A-polynomial of 7₂ (or the gcd claim for Apéry numbers) diverges from the asserted closed form.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes a focused AI-evaluation challenge consisting of explicit formulas (continued fractions, holonomic recurrences, matrix products, series, and one integral) for classical constants such as π, e, γ, Catalan's constant G, ζ(2)+ζ(3), and √10005/π. Section 2 presents identities whose proofs the authors claim to hold privately and will keep encrypted for a short initial period; Section 3 presents two numerically supported open conjectures (a Godbillon–Vey-type integral over the A-polynomial of the knot 7₂ equaling 4π²/85, and a gcd claim asserting that Apéry's irrationality-measure bound for ζ(3) is tight for his sequences). The formulas are written with full polynomial coefficients and initial conditions so that they can be checked to arbitrary precision, and a short discussion argues that computer-algebra derivations may count as sufficient solutions for the challenge.

Significance. If the Section 2 identities are correct and the Section 3 conjectures hold, the paper supplies a clean, contamination-resistant benchmark in the spirit of First Proof and FrontierMath, specialized to the classical tradition of Ramanujan- and Apéry-style formulas. Strengths that deserve explicit credit are: (i) every claim is stated with complete, numerically falsifiable data (recurrences, initial values, matrix entries, integration limits); (ii) proven and open items are separated; (iii) the discussion of what should count as proof in the age of AI is a useful, concrete contribution for experimental mathematics. The open problems (especially the knot-polynomial integral and the Apéry gcd claim) are of independent mathematical interest. The work is therefore significant for both AI evaluation and the experimental-constants community, provided the private proofs are eventually released and the classification of items is made consistent.

major comments (3)
  1. [§2.8] §2.8 ends with the wording “we conjecture that lim PN,j/QN,j = √10005/π”, yet §2 is introduced as the section of “proven problems” whose proofs “are known to the authors”. Every other item in §2 is phrased “Prove: …”. Either 2.8 belongs in §3, or the verb should be “Prove”. As written, the proven/conjectural partition that the paper advertises is internally inconsistent on a load-bearing classification.
  2. [§2 (esp. 2.2, 2.3) and §4] The central claim that the §2 identities “hold” and are suitable as settled evaluation targets rests on author-held proofs that referees (and readers) cannot inspect. For holonomic limits such as 2.2 (γ) and 2.3 (π+e), the claimed limit is sensitive to every polynomial coefficient and every initial condition; high-precision numerical agreement alone does not distinguish a true identity from a high-order coincidence. The manuscript should either (a) deposit the proofs or CAS certificates with the journal under embargo with a stated release date, or (b) reclassify any item whose proof is not yet public as a conjecture. Without one of these, the “proven” half of the challenge cannot be independently validated.
  3. [§4 Discussion] §4 asserts that “a derivation carried out using symbolic libraries within established computer algebra systems” is “sufficient evidence of a valid solution” for the challenge. That convention is reasonable for an AI contest, but it is not the same as a classical mathematical proof. The paper should state explicitly which of the encrypted §2 solutions are human-readable proofs and which are CAS certificates, and should give minimal acceptance criteria (e.g., fully expanded certificate, independent re-execution, or Lean formalization) so that AI submissions can be judged uniformly.
minor comments (6)
  1. [§1–§3 headings] Section headings appear concatenated in the source (“AFEW REMARKS”, “THEQUESTIONS”, “THECONJECTURES”). Insert spaces for readability.
  2. [front-matter table] The contributor table at the front lists authors for most items but leaves 2.3, 2.6 and 2.8 without named contributors; either complete the table or mark them as group contributions.
  3. [§2.1] In 2.1 the continued-fraction value is written 6/(3−π). A brief remark that this is equivalent to a more standard form (or why the reciprocal form is preferred) would help readers checking numerics.
  4. [§3.1] §3.1 quotes approximate roots α≈0.349269… and β≈0.406813…; giving a short isolating polynomial or interval with a certified enclosure would make the integral limits fully rigorous to state.
  5. [Abstract / §1] The phrase “for a short initial period” (abstract and §1) is never quantified. A concrete embargo length or release trigger would strengthen the challenge design.
  6. [§1] References to contemporaneous AI-math benchmarks (First Proof, FrontierMath, Riemann-Bench, Humanity’s Last Exam) are appropriate; a one-sentence comparison of verification protocols would situate the present challenge more sharply.

Circularity Check

0 steps flagged

No circularity: the paper poses independent identities and open conjectures as AI challenges; it contains no derivation chain that reduces a claimed prediction to its own inputs by construction.

full rationale

The manuscript is a challenge list, not a derivation paper. Section 2 states explicit formulas (continued fractions, holonomic recurrences, matrix products, series) whose limits equal named constants, with proofs held privately by the authors; Section 3 states two open conjectures. None of the statements is obtained by fitting a free parameter to data and then re-labeling the fit as a prediction, nor by defining a quantity in terms of the target constant and then recovering that constant. Self-citations (Raayoni et al. 2021, Elimelech et al. 2023, Weinbaum et al. 2025) appear only as historical context for the Ramanujan-Machine tradition and are not invoked to force uniqueness, to smuggle an ansatz, or to underwrite any of the listed identities. Numerical verifiability is independent of the private proofs. Consequently the derivation chain is empty of circular steps and the paper is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The paper is a challenge collection rather than a derivation from first principles. It rests on standard analytic number theory (existence of the indicated limits, holonomic recurrences, A-polynomials of knots) plus the authors' private knowledge that certain proofs exist. No free parameters are fitted to external data; initial conditions are part of the explicit statements. No new physical or mathematical entities are postulated.

axioms (3)
  • standard math The indicated polynomial recurrences and matrix products possess well-defined asymptotic ratios that equal the claimed constants.
    Invoked for every Apéry-style limit and matrix product (Sections 2.2–2.8).
  • domain assumption The A-polynomial of the prime knot 7₂ is correctly given and the real algebraic curve segment between the stated roots exists with the claimed monotonicity.
    Required for the integral identity in Section 3.1; taken from knot-theory literature.
  • ad hoc to paper Author-held proofs of the Section 2 identities are mathematically valid.
    Stated in the introduction and problem statements; proofs remain encrypted.

pith-pipeline@v1.1.0-grok45 · 13124 in / 2071 out tokens · 26898 ms · 2026-07-14T17:07:29.538552+00:00 · methodology

0 comments
read the original abstract

To help evaluate the mathematical skills of current AI systems, we present a set of formulas for fundamental mathematical constants. These problems are attractive for AI evaluation because they are concrete and can be checked numerically to arbitrary precision, yet proving them may require non-obvious mathematics. Mathematical constants such as $\pi$, $e$, Catalan's constant, and special values of the Riemann zeta function have fascinated mathematicians for centuries. The search for formulas evaluating mathematical constants has produced some of the most beautiful mathematics in the field, especially in cases that yield irrationality proofs or fast convergence rates. Ramanujan's legacy is emblematic of this tradition. The list we provide contains two types of problems: formulas whose proofs are known to the authors but will remain encrypted for a short initial period; and formulas that are not yet proven. We are curious to see the achievements of AI in both cases.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

250 extracted references · 9 canonical work pages

  1. [1]

    2017 , publisher =

    Leland McInnes and John Healy and Steve Astels , title =. 2017 , publisher =. doi:10.21105/joss.00205 , url =

    doi:10.21105/joss.00205 2017

  2. [2]

    Nature , title =

    Gal Raayoni and Shahar Gottlieb and Yahel Manor and George Pisha and Yoav Harris and Uri Mendlovic and Doron Haviv and Yaron Hadad and Ido Kaminer , doi =. Nature , title =

  3. [3]

    On the Connection Between Irrationality Measures and Polynomial Continued Fractions , year =

    Nadav Ben David and Guy Nimri and Uri Mendlovic and Yahel Manor and Ido Kaminer , journal =. On the Connection Between Irrationality Measures and Polynomial Continued Fractions , year =

  4. [4]

    Lagarias , doi =

    Jeffrey C. Lagarias , doi =. Bulletin of the American Mathematical Society , title =

  5. [5]

    John Wallis , title =

  6. [6]

    Leonhard Euler , title =

  7. [7]

    John Olsen , title =

  8. [8]

    2019 , publisher=

    Diophantine approximation and transcendence theory , author=. 2019 , publisher=

    2019

  9. [9]

    arXiv preprint arXiv:2210.03391 , year=

    On cellular rational approximations to (5) , author=. arXiv preprint arXiv:2210.03391 , year=

    arXiv

  10. [10]

    Tweaking the Beukers integrals in search of more miraculous irrationality proofs a la Ap

    Dougherty-Bliss, Robert and Koutschan, Christoph and Zeilberger, Doron , journal=. Tweaking the Beukers integrals in search of more miraculous irrationality proofs a la Ap. 2022 , publisher=

    2022

  11. [11]

    Bulletin of the London Mathematical Society , volume=

    A note on the irrationality of (2) and (3) , author=. Bulletin of the London Mathematical Society , volume=. 1979 , publisher=

    1979

  12. [12]

    2009 , eprint=

    On linear forms containing the Euler constant , author=. 2009 , eprint=

    2009

  13. [13]

    Fifth IEEE/ACM international workshop on grid computing , pages=

    Boinc: A system for public-resource computing and storage , author=. Fifth IEEE/ACM international workshop on grid computing , pages=. 2004 , organization=

    2004

  14. [14]

    and Zeilberger, Doron , title =

    Petkovsek, Marko and Wilf, Herbert S. and Zeilberger, Doron , title =

  15. [15]

    2002 , publisher=

    Wolfram, Stephen and others , volume=. 2002 , publisher=

    2002

  16. [16]

    Journal of applied logic , volume=

    Buchberger, Bruno and Crǎciun, Adrian and Jebelean, Tudor and Kov. Journal of applied logic , volume=

  17. [17]

    Bailey, David and Borwein, Jonathan and Calkin, Neil and Luke, Russell and Girgensohn, Roland and Moll, Victor , year=

  18. [18]

    1988 , booktitle =

    On Conjectures of Graffiti , editor =. 1988 , booktitle =. doi:https://doi.org/10.1016/S0167-5060(08)70776-3 , url =

    work page doi:10.1016/s0167-5060(08)70776-3 1988

  19. [19]

    Lambert, Johann Heinrich , journal=

  20. [20]

    Nature , volume=

    Davies, Alex and Veli. Nature , volume=

  21. [21]

    Fawzi, Alhussein and Balog, Matej and Huang, Aja and Hubert, Thomas and Romera-Paredes, Bernardino and Barekatain, Mohammadamin and Novikov, Alexander and R Ruiz, Francisco J and Schrittwieser, Julian and Swirszcz, Grzegorz and others , journal=

  22. [22]

    Modular Functions of One Variable I: Proceedings International Summer School University of Antwerp, RUCA July 17--August 3, 1972 , pages=

    Modular forms of half integral weight , author=. Modular Functions of One Variable I: Proceedings International Summer School University of Antwerp, RUCA July 17--August 3, 1972 , pages=. 1973 , publisher=

    1972

  23. [23]

    arXiv preprint arXiv:2308.11829 , year=

    Algorithm-assisted discovery of an intrinsic order among mathematical constants , author=. arXiv preprint arXiv:2308.11829 , year=

    Pith/arXiv arXiv

  24. [24]

    Berggren, Lennart and Borwein, Jonathan and Borwein, Peter and Lambert, M , journal=. M. 2004 , publisher=

    2004

  25. [25]

    Nature , volume=

    Solving olympiad geometry without human demonstrations , author=. Nature , volume=. 2024 , publisher=

    2024

  26. [26]

    arXiv preprint arXiv:2401.08500 , year=

    Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering , author=. arXiv preprint arXiv:2401.08500 , year=

    Pith/arXiv arXiv

  27. [27]

    Hardy, Godfrey Harold and Wright, Edward Maitland and others , year=

  28. [28]

    van der Poorten, Alf , journal=

  29. [29]

    1748 , publisher=

    Euler, Leonhard , volume=. 1748 , publisher=

  30. [30]

    Bowman, Douglas and McLaughlin, James , Journal =

  31. [31]

    Cuyt, Annie and Petersen, Vigdis and Verdonk, Brigitte and Waadeland, Haakon and Jones, William B. , year=

  32. [32]

    Ben David, Nadav and Nimri, Guy and Mendlovic, Uri and Manor, Yahel and Kaminer, Ido , journal=

  33. [33]

    Ferguson, Helaman and Bailey, David , year=

  34. [34]

    Finch , year=

    Steven R. Finch , year=

  35. [35]

    , author=

    MaLARea: a Metasystem for Automated Reasoning in Large Theories. , author=. ESARLT , volume=

  36. [36]

    CoRR , volume =

    Kaliszyk, Cezary and Urban, Josef , title =. CoRR , volume =. 2012 , url =. 1211.7012 , timestamp =

    Pith/arXiv arXiv 2012

  37. [37]

    , journal=

    Wang, H. , journal=. Toward Mechanical Mathematics , year=

  38. [38]

    Artificial intelligence , volume=

    Why AM and EURISKO appear to work , author=. Artificial intelligence , volume=. 1984 , publisher=

    1984

  39. [39]

    1982 , issn =

    The nature of heuristics , journal =. 1982 , issn =. doi:https://doi.org/10.1016/0004-3702(82)90036-4 , url =

    work page doi:10.1016/0004-3702(82)90036-4 1982

  40. [40]

    1982 , publisher=

    Knowledge-based Systems in Artificial Intelligence , author=. 1982 , publisher=

    1982

  41. [41]

    arXiv preprint math/0402462 , year=

    Real numbers with polynomial continued fraction expansions , author=. arXiv preprint math/0402462 , year=

    Pith/arXiv arXiv

  42. [42]

    and Bouldin, Donald W

    Davies, David L. and Bouldin, Donald W. , journal=. A Cluster Separation Measure , year=

  43. [43]

    and Kriegel, Hans-Peter and Sander, J\"

    Ankerst, Mihael and Breunig, Markus M. and Kriegel, Hans-Peter and Sander, J\". OPTICS: ordering points to identify the clustering structure , year =. doi:10.1145/304181.304187 , journal =

    doi:10.1145/304181.304187

  44. [44]

    arXiv preprint arXiv:2111.04468 , year=

    On the connection between irrationality measures and polynomial continued fractions , author=. arXiv preprint arXiv:2111.04468 , year=

    arXiv

  45. [45]

    and Haberland, Matt and Reddy, Tyler and Cournapeau, David and Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and Bright, Jonathan and

    Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and Haberland, Matt and Reddy, Tyler and Cournapeau, David and Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and Bright, Jonathan and. Nature Methods , year =

  46. [46]

    Harris and K

    Charles R. Harris and K. Jarrod Millman and St. Array programming with. 2020 , month = sep, journal =. doi:10.1038/s41586-020-2649-2 , publisher =

    doi:10.1038/s41586-020-2649-2 2020

  47. [47]

    Github-gmpy , url =

    Case Van Horsen , year =. Github-gmpy , url =

  48. [48]

    NeurIPS , year=

    Measuring Mathematical Problem Solving With the MATH Dataset , author=. NeurIPS , year=

  49. [49]

    2021 , eprint=

    Training Verifiers to Solve Math Word Problems , author=. 2021 , eprint=

    2021

  50. [50]

    Advances in neural information processing systems , volume=

    Mathematical capabilities of chatgpt , author=. Advances in neural information processing systems , volume=

  51. [51]

    8th International Conference on Learning Representations, ICLR 2020 , year =

    Guillaume Lample and François Charton , title =. 8th International Conference on Learning Representations, ICLR 2020 , year =

    2020

  52. [52]

    O lympiad B ench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems

    He, Chaoqun and Luo, Renjie and Bai, Yuzhuo and Hu, Shengding and Thai, Zhen and Shen, Junhao and Hu, Jinyi and Han, Xu and Huang, Yujie and Zhang, Yuxiang and Liu, Jie and Qi, Lei and Liu, Zhiyuan and Sun, Maosong. O lympiad B ench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems. Proceedings of the ...

    doi:10.18653/v1/2024.acl-long.211 2024

  53. [53]

    Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction

    Li, Xiaoyuan and Wang, Wenjie and Li, Moxin and Guo, Junrong and Zhang, Yang and Feng, Fuli. Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction. Findings of the Association for Computational Linguistics: ACL 2024. 2024. doi:10.18653/v1/2024.findings-acl.673

    doi:10.18653/v1/2024.findings-acl.673 2024

  54. [54]

    2025 , url=

    Zhongshen Zeng and Pengguang Chen and Shu Liu and Haiyun Jiang and Jiaya Jia , booktitle=. 2025 , url=

    2025

  55. [55]

    Advances in Neural Information Processing Systems , year=

    A Careful Examination of Large Language Model Performance on Grade School Arithmetic , author=. Advances in Neural Information Processing Systems , year=

  56. [56]

    2025 , url=

    Seyed Iman Mirzadeh and Keivan Alizadeh and Hooman Shahrokhi and Oncel Tuzel and Samy Bengio and Mehrdad Farajtabar , booktitle=. 2025 , url=

    2025

  57. [57]

    arXiv preprint arXiv:2411.18104 , year=

    Training and Evaluating Language Models with Template-based Data Generation , author=. arXiv preprint arXiv:2411.18104 , year=

    Pith/arXiv arXiv

  58. [58]

    2025 , eprint=

    Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges , author=. 2025 , eprint=

    2025

  59. [59]

    Neurips Safe Generative AI Workshop 2024 , year=

    Large Language Model Benchmarks Do Not Test Reliability , author=. Neurips Safe Generative AI Workshop 2024 , year=

    2024

  60. [60]

    2025 , eprint=

    Beyond the Singular: The Essential Role of Multiple Generations in Effective Benchmark Evaluation and Analysis , author=. 2025 , eprint=

    2025

  61. [61]

    Are NLP Models really able to Solve Simple Math Word Problems?

    Patel, Arkil and Bhattamishra, Satwik and Goyal, Navin. Are NLP Models really able to Solve Simple Math Word Problems?. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. doi:10.18653/v1/2021.naacl-main.168

    doi:10.18653/v1/2021.naacl-main.168 2021

  62. [62]

    International Conference on Learning Representations , year=

    miniF2F: a cross-system benchmark for formal Olympiad-level mathematics , author=. International Conference on Learning Representations , year=

  63. [63]

    2025 , archivePrefix=

    MathConstruct: Challenging LLM Reasoning with Constructive Proofs , author=. 2025 , archivePrefix=

    2025

  64. [64]

    2025 , eprint=

    U-MATH: A University-Level Benchmark for Evaluating Mathematical Skills in LLMs , author=. 2025 , eprint=

    2025

  65. [65]

    2024 , eprint=

    MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data , author=. 2024 , eprint=

    2024

  66. [66]

    2025 , eprint=

    Large Language Models and Mathematical Reasoning Failures , author=. 2025 , eprint=

    2025

  67. [67]

    Zhou, Zihao and Wang, Qiufeng and Jin, Mingyu and Yao, Jie and Ye, Jianan and Liu, Wei and Wang, Wei and Huang, Xiaowei and Huang, Kaizhu , title =. Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances i...

    work page doi:10.1609/aaai.v38i17.29949 2024

  68. [68]

    2025 , url=

    Kaixuan Huang and Jiacheng Guo and Zihao Li and Xiang Ji and Jiawei Ge and Wenzhe Li and Yingqing Guo and Tianle Cai and Hui Yuan and Runzhe Wang and Yue Wu and Ming Yin and Shange Tang and Yangsibo Huang and Chi Jin and Xinyun Chen and Chiyuan Zhang and Mengdi Wang , booktitle=. 2025 , url=

    2025

  69. [69]

    The Thirteenth International Conference on Learning Representations , year=

    Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist , author=. The Thirteenth International Conference on Learning Representations , year=

  70. [70]

    2024 , eprint=

    Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap , author=. 2024 , eprint=

    2024

  71. [71]

    Wolfram|Alpha: Computational Intelligence , howpublished =

  72. [72]

    PeerJ Computer Science , issn =

    SymPy: symbolic computing in Python , author =. PeerJ Computer Science , issn =

  73. [73]

    2025 , month =

    Introducing. 2025 , month =

    2025

  74. [74]

    2025 , month =

    OpenAI O3 and O4-Mini System Card , author=. 2025 , month =

    2025

  75. [75]

    2024 , eprint=

    FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI , author=. 2024 , eprint=

    2024

  76. [76]

    2025 , url=

    Xin Xu and Jiaxin ZHANG and Tianhao Chen and Zitong Chao and Jishan Hu and Can Yang , booktitle=. 2025 , url=

    2025

  77. [77]

    2024 , journal=

    OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI , author=. 2024 , journal=

    2024

  78. [78]

    2019 , url =

    Brazilian Mathematical Olympiad, University Level , title =. 2019 , url =

    2019

  79. [79]

    A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners

    Jiang, Bowen and Xie, Yangxinyu and Hao, Zhuoqun and Wang, Xiaomeng and Mallick, Tanwi and Su, Weijie J and Taylor, Camillo Jose and Roth, Dan. A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.272

    doi:10.18653/v1/2024.emnlp-main.272 2024

  80. [80]

    M ath B ench: Evaluating the Theory and Application Proficiency of LLM s with a Hierarchical Mathematics Benchmark

    Liu, Hongwei and Zheng, Zilong and Qiao, Yuxuan and Duan, Haodong and Fei, Zhiwei and Zhou, Fengzhe and Zhang, Wenwei and Zhang, Songyang and Lin, Dahua and Chen, Kai. M ath B ench: Evaluating the Theory and Application Proficiency of LLM s with a Hierarchical Mathematics Benchmark. Findings of the Association for Computational Linguistics: ACL 2024. 2024...

    doi:10.18653/v1/2024.findings-acl.411 2024

Showing first 80 references.