Pith. sign in

REVIEW 3 major objections 4 minor 20 references

TopoAgent claims that replacing linear reasoning with a dependency graph of visually grounded atomic steps lifts multimodal science accuracy to a 66.3% average across six foundation models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 01:27 UTC pith:E37KJWQF

load-bearing objection A coherent, well-engineered agent framework with a genuinely novel assembly, but the evidence does not yet support the claim that topological routing causes the gains. the 3 major comments →

arxiv 2607.14658 v1 pith:E37KJWQF submitted 2026-07-16 cs.AI

TopoAgent: A Self-Evolving Topological Agent for Multimodal Scientific Reasoning

Mingze Xu , Yinghui Li , Jiayi Kuang , Zhanhui Kang , Di Yin , Ying Shen , Xing Sun , Yuxing Han This is my paper
classification cs.AI
keywords multimodal scientific reasoningdirected acyclic graph planningcontext isolationatomic decompositionadaptive fissiontool-augmented agentshallucination mitigationlong-horizon reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that multimodal language models fail at scientific reasoning mainly because of how they are orchestrated, not just because of model capacity: linear planners force a non-linear process into a single accumulating prompt, entangling visual perception with symbolic deduction and drowning later steps in irrelevant history. TopoAgent tries to fix this by decomposing each question into small, visually grounded atoms, arranging them in a directed acyclic graph of prerequisites, and feeding each reasoning step only the completed results of its direct parents. When an atom proves too hard for the available tools, the framework splits it at runtime into finer sub-atoms, degrading task granularity instead of aborting. On mathematics, physics, and chemistry benchmarks across six models, the paper reports a global average accuracy of 66.3%, ahead of linear planners and multi-agent frameworks, and attributes the margin to context isolation and adaptive fission.

Core claim

On the paper's own terms, the central discovery is that a science-reasoning agent built on a DAG of visually grounded atoms — with strict context isolation and runtime atomic fission — outperforms both linear planners and generic multi-agent systems on multimodal math, physics, and chemistry questions. The reported global average across six foundation models is 66.3%, with the largest single gain on Olympiad-level physics, where the global average rises from 30.9% to 41.2%. The ablations attribute roughly 1.4 points of the average to DAG planning and 0.6 points to atomic fission. In successful runs, 12.65% of mathematics atoms and 9.88% of physics atoms were resolved only after fission, and

What carries the argument

The load-bearing object is the directed acyclic graph G = (A, E) produced by the decomposer. Each atom A_i is a minimal executable unit binding a local objective to a visual region; edges encode prerequisite dependencies. The context-isolation rule feeds executing node A_i only the stored results of its direct parents, C(A_i) = ⋃_{A_j ∈ P(A_i)} M(A_j), so historical noise from parallel or irrelevant branches never reaches the planner. When a node fails to reach a STOP state within T_max steps, adaptive fission F(A_i, trace) replaces it with finer sub-atoms A_i.1 … A_i.k, and hierarchical aggregation M_virtual(A_i) = ⋃_k M(A_i.k) lets descendants read the sub-results as if the parent had succ

Load-bearing premise

The entire pipeline assumes the front-end decomposer builds a correct and complete dependency graph of visually grounded atomic tasks; the paper reports no decomposition success rate or edge accuracy, and fission only re-splits failed nodes, so a wrong or missing edge cannot be repaired downstream.

What would settle it

Take a sample of problems TopoAgent solves correctly, delete one prerequisite edge from the decomposer's graph, and rerun. If final accuracy does not drop, the DAG topology is not actually load-bearing; if it drops sharply, context isolation is the operative mechanism. A complementary test: inject a plausible but wrong visual variable into an atom's grounding and check whether the wrong value survives verification and reaches the final answer despite fission.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any long-horizon reasoning pipeline that concatenates full execution histories should expect attention dilution and accuracy decay as depth grows; routing only prerequisite states should keep both token use and accuracy stable.
  • In domains with branching parallel sub-problems, such as geometry and kinematics, runtime granularity degradation should recover a substantial fraction of failures that blind retries cannot.
  • The architecture should transfer across different multimodal models without per-model prompt tuning, since the reported gains hold for both proprietary and open-weight models.
  • Chemistry, where reasoning is more sequential and pipeline-like, should benefit less from both context isolation and fission, matching the paper's small chemistry deltas.
  • The token-stability result implies the approach can scale to deeper reasoning chains than linear planners before hitting context-window limits.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next step the paper leaves implicit is to verify the decomposer itself: an automated check that every visual variable named in the question appears in some atom, and every atom's prerequisites are actually needed, would close the largest unmeasured gap.
  • Because fission re-splits only the node that failed, a stronger version of the framework could mine the failure trace for missing or mis-wired edges and repair the graph itself, turning local granularity adaptation into global self-repair.
  • The 66.3% average is a joint product of the decomposer and the executor; holding the decomposer fixed and varying only the planner would isolate how much of the gain is due to graph topology versus the quality of upfront decomposition.
  • The context-isolation principle is not specific to science: any multi-step task where a later step depends on a few precise intermediate states — code fixes, document question answering, tool orchestration — could adopt the same DAG-plus-isolation pattern.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. TopoAgent replaces linear agent trajectories with a DAG of visually-grounded atomic tasks for multimodal scientific reasoning. A front-end Decomposer fractures a query into atoms, a DAG planner enforces context isolation by giving each atom only the memory states of its direct prerequisites (Eq. 1), and an Adaptive Atomic Fission mechanism splits failed atoms into sub-atoms at runtime (Eqs. 2–3). The paper reports evaluations on MathVista, Multi-math, SymbolBench subsets, OlympiadBench, and MMMU across six MLLMs, claiming a global average accuracy of 66.3% versus 62.8% for the best baseline, with ablations showing performance drops of 1.4% without DAG planning and 0.6% without fission. Additional analyses report token efficiency, robustness across cognitive levels, and a qualitative case study.

Significance. If the empirical claims are reliable, TopoAgent offers a conceptually attractive and computationally plausible alternative to linear agent designs for multimodal scientific reasoning. The explicit algorithm specification, the controlled ablations, and the per-domain breakdown are useful assets, and the token-efficiency analysis targets a real failure mode of accumulating full histories. However, the central causal claim that DAG-based context isolation drives the reported gains is not yet established: the paper provides no significance tests, no measurement of decomposition correctness, and no mechanism by which fission repairs missing graph edges or mis-grounded visual variables. The evaluation also leans heavily on SymbolBench, a benchmark from the same author group, so the breadth of the claimed superiority needs independent confirmation. The architecture is promising, but the evidence as presented is not sufficient for the strength of the conclusions.

major comments (3)
  1. [Experiments, Main Results (Table 1); Abstract] The paper repeatedly uses 'significantly outperforms' and 'substantial margin' (Abstract, Introduction, Experiments), but no significance tests, confidence intervals, or repeated-run distributions are reported. The ± values in Table 1 are not accompanied by an explanation of the number of runs or a paired-comparison procedure. Moreover, the per-cell results are inconsistent with the global claim: with GPT-5, TopoAgent is below AutoGen on MathVista (78.3 vs 80.2) and below OctoTools on Multi-math (74.3 vs 78.5). The 'model-agnostic robustness' conclusion needs a paired significance analysis or at least a bootstrap across benchmark subsets; without this, the headline margin over AutoGen (66.3 vs 62.0) could be driven by a few columns.
  2. [Methodology, Eq. (1), Eq. (3), Algorithm 1; Qualitative Analysis] The causal claim that DAG-based context isolation produces the gains is load-bearing but unproven. Eq. (1) supplies each atom only the states of its direct prerequisites P(A_i). If the Decomposer misses an edge, mis-grounds a visual variable, or misclassifies a dependency, downstream nodes never receive the missing information. Adaptive Atomic Fission (Eqs. 2–3) only re-splits a node that failed to reach STOP; it does not add missing edges, correct wrong variable grounding, or re-run the Decomposer. The statement after Eq. (3) that fission enables 'robust problem-solving even with imperfect initial decompositions' is therefore unsupported by the described mechanism. The paper reports no decomposition success rate, edge accuracy, or failure analysis, so the observed gains may partly reflect the Decomposer's prompt structure rather than the topological routing. At minimum, the authors shou
  3. [Experiments, Datasets; Robustness Across Cognitive Complexities] Three of the seven benchmark columns (Math-Sym, Physics-Sym, Chem-Sym) come from SymbolBench (Li et al. 2026), and the cognitive-level taxonomy in Figure 6 is also attributed to that work. The author list of the current paper includes Yinghui Li and other authors who appear prominently in the SymbolBench citation, so this is an in-group benchmark. The paper does not disclose this overlap. Because the 'state-of-the-art' claim rests substantially on these columns, the authors should either add at least one independent benchmark from outside their group or clearly discuss the overlap and provide results that hold when SymbolBench columns are excluded. This is not a circularity in the mathematical sense, but it is a correctness risk for the generalization claim.
minor comments (4)
  1. [Experiments, Table 1] The ± values are not defined. Specify whether they are standard deviations over repeated runs, and how many runs were averaged. Without this, the precision implied by e.g. 78.3±1.4 cannot be assessed.
  2. [Context Isolation and Token Efficiency (Figure 5)] The text describes 'exponential token growth' for OctoTools, but the bar chart appears to show linear or super-linear growth over six steps. Please use a more precise characterization (e.g., 'rapid growth') or provide data supporting exponential scaling.
  3. [Methodology, Algorithm 1] Algorithm 1 always passes the original image I to the planner (line 13) regardless of context isolation. The relationship between the isolated prerequisite context C(A_i) and the raw image should be clarified: does the image act as an additional channel that could reintroduce the visual noise the framework aims to filter?
  4. [General] The manuscript does not state whether code, prompts, or benchmark evaluation scripts will be released. Providing the Decomposer and Verifier prompts, as well as the fission trigger conditions, would materially aid reproducibility.

Circularity Check

0 steps flagged

Minor same-group citation in SymbolBench/taxonomy, but no circular derivation: performance claims rest on benchmark comparisons, not fitted parameters or self-referential definitions.

full rationale

TopoAgent is an empirical systems paper: its central equations are algorithmic definitions, not fitted relationships. Eq. (1) defines the context C(A_i) as the union of memory states of direct prerequisite nodes; Eq. (2) defines atomic fission; Eq. (3) defines hierarchical aggregation of sub-atom results. None is fit to or defined by the benchmark outcomes they are used to explain, so there is no fitted-input-called-prediction or self-definitional circularity. The headline 66.3% is a measured average over seven benchmark columns; four columns (MathVista, Multi-math, OlympiadBench, MMMU) are external, and excluding the three SymbolBench columns still leaves TopoAgent ahead of the strongest external baselines on those four columns, so the same-group SymbolBench benchmark (Li et al. 2026, sharing authors Yinghui Li and Jiayi Kuang) is not load-bearing for the qualitative conclusion. The robustness analysis uses the cognitive taxonomy from the same citation, but that is an analytic lens, not a fitted prediction. The assertion in the Adaptive Atomic Fission section that fission enables 'robust problem-solving even with imperfect initial decompositions' (after Eq. 3) is not backed by decomposition-success or edge-accuracy measurements; I flag it as a missing-support passage. That is a correctness/robustness threat, not circularity. Score 2 reflects the minor non-load-bearing self-citation; no circular step is present.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 2 invented entities

The framework's effectiveness rests on a set of untested behavioral assumptions about the MLLM decomposer, verifier, and tool environment, plus a benchmark mix that includes self-authored subsets. There are no fitted physical or model parameters; the free parameters are engineering hyperparameters (T_max, k, decomposition granularity) that are not swept or reported.

free parameters (3)
  • T_max (maximum execution steps per atom)
    Chosen 'strict horizon' in Algorithm 1 line 12 and Figure 7 as sub_task_max_steps. It determines when fission triggers and is never swept or reported; no sensitivity analysis is given.
  • Fission sub-atom count k
    Eq. (2) lets the LLM decide k from the failure trace; no constraint, upper bound, or evaluation of this choice is provided.
  • Atom decomposition granularity
    The Decomposer determines atom boundaries and dependency edges; no prompt/protocol or consistency metric is reported. The context-isolation claim depends entirely on this behavior.
axioms (6)
  • domain assumption The Decomposer produces accurate visually-grounded atoms and a complete dependency DAG.
    Methodology, Visually-Grounded Atomic Decomposition and Eq. (1). If an edge or visual anchor is wrong, context isolation hides the missing information from downstream nodes; no decomposition accuracy metric is reported.
  • domain assumption Context isolation: direct prerequisite outputs are sufficient for solving each atom.
    Eq. (1) constructs C(A_i) only from P(A_i). The paper asserts noise reduction outweighs loss of global context, but this is not tested against variants that add selected extra context.
  • domain assumption The Verifier's STOP/failure decision is reliable and consistent.
    Algorithm 1 lines 17-20 gate success and trigger fission; no definition, accuracy, or validation of the Verifier is provided.
  • domain assumption External tools return deterministic, correct results when they reach STOP.
    Algorithm 1 lines 13-15 execute arbitrary tools; context isolation can only be as good as the tool outputs, and tool errors would propagate along DAG edges.
  • domain assumption Linear context accumulation is a primary cause of the reported performance gap.
    Introduction and the Context Isolation section assert this; the only supporting evidence is Figure 5, which lacks raw numbers and a controlled comparison.
  • domain assumption The selected benchmarks, including self-authored SymbolBench subsets, are representative of multimodal scientific reasoning.
    Datasets section; three of seven columns are SymbolBench subsets from Li et al. 2026 (same research group), and their inclusion in the headline average is not justified or analyzed for selection effects.
invented entities (2)
  • Atom (and fissioned sub-atoms) no independent evidence
    purpose: Minimal, visually-grounded executable unit in the DAG; the unit of context isolation and fission.
    Framework abstraction; no formal semantics, code, or external prediction is tied to it, so it cannot be falsified outside the paper's own benchmark results.
  • Hierarchical dependency mapping / virtual memory M_virtual(A_i) no independent evidence
    purpose: Aggregates sub-atom results under the parent namespace so downstream nodes can query A_i after fission (Eq. 3).
    Internal bookkeeping construct; no independent evidence or specification of aggregation semantics beyond the equation is given.

pith-pipeline@v1.3.0-alltime-deepseek · 13333 in / 14520 out tokens · 149591 ms · 2026-08-02T01:27:03.850498+00:00 · methodology

0 comments
read the original abstract

While Multimodal Large Language Models (MLLMs) excel in general tasks, rigorous scientific reasoning remains challenging due to the limitations of monolithic, linear planning. Such sequential designs often suffer from visual-semantic misalignment, long-context hallucinations, and brittle execution under fixed task granularity. We propose TopoAgent, a self-evolving topological framework that replaces linear trajectories with dynamic, state-isolated graph evolution. TopoAgent first employs a front-end decomposer to fracture complex queries into visually-grounded atoms. These atoms are organized into a Directed Acyclic Graph (DAG) based on their dependencies, enabling strict context isolation to shield the reasoning engine from irrelevant historical noise. Furthermore, we introduce adaptive atomic fission, which dynamically splits bottleneck nodes into finer-grained sub-atoms at runtime when tool capability boundaries are exceeded. Extensive experiments across mathematics, physics, and chemistry benchmarks demonstrate that TopoAgent significantly outperforms state-of-the-art linear agent frameworks, providing a robust, noise-resistant, and self-correcting paradigm for autonomous scientific reasoning.

Figures

Figures reproduced from arXiv: 2607.14658 by Di Yin, Jiayi Kuang, Mingze Xu, Xing Sun, Yinghui Li, Ying Shen, Yuxing Han, Zhanhui Kang.

Figure 1
Figure 1. Figure 1: Comparison of global average accuracy across six [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overall architecture of the proposed TopoAgent framework. Given a complex multimodal scientific query, the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of atomic task resolution methods by [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of recovery rates among different [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance radar charts across cognitive complexities in Mathematics, Physics, and Chemistry. TopoAgent exhibits [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: A qualitative case study illustrating the execution trajectory of TopoAgent. The framework first constructs a Directed [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 10 linked inside Pith

  1. [4]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19552–19564

    Microvqa: A multi- modalreasoningbenchmarkformicroscopy-basedscientific research. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19552–19564. Cui,Y.;Yao,X.;Qin,Y.;Li,X.;Wang,S.;andHu,G.2025. Evaluating large language models on multimodal chemistry olympiad exams.Communications Chemistry. Du, S.; Zhao, J.; Shi, J.; Xie...

    2025

  2. [5]

    Guo,X.;Kuang,J.;Pan,L.;Li,Y.;Li,Y.;Zheng,H.-T.;Shen, Y.; Yin, D.; and Sun, X

    Chartcitor: Multi-agent framework for fine-grained chart visual attribution.arXiv preprint arXiv:2502.00989. Guo,X.;Kuang,J.;Pan,L.;Li,Y.;Li,Y.;Zheng,H.-T.;Shen, Y.; Yin, D.; and Sun, X

    Pith/arXiv arXiv

  3. [6]

    He, C.; Luo, R.; Bai, Y.; Hu, S.; Thai, Z.; Shen, J.; Hu, J.; Han,X.;Huang,Y.;Zhang,Y.;etal.2024

    EvoConfig: Self-Evolving Multi-AgentSystemsforEfficientAutonomousEnvironment Configuration.arXiv preprint arXiv:2601.16489. He, C.; Luo, R.; Bai, Y.; Hu, S.; Thai, Z.; Shen, J.; Hu, J.; Han,X.;Huang,Y.;Zhang,Y.;etal.2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad- level bilingual multimodal scientific problems. InProceed- ings...

    arXiv 2024

  4. [7]

    Huang, W.-C.; Zou, H

    Glm-4.5 v and glm-4.1 v-thinking: Towards versatile multimodal rea- soning with scalable reinforcement learning.arXiv preprint arXiv:2507.01006. Huang, W.-C.; Zou, H. P.; Wu, Y.; Li, D.; Chen, Y.; Zhang, W.; Li, Y.; Zangari, A.; Guo, J.; Miao, C.; et al

    Pith/arXiv arXiv

  5. [8]

    InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 43403–43446

    Deep Research with Open-Domain Evaluation and Multi-Stage Guardrails for Safety. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 43403–43446. Kuang, J.; Huang, H.; Li, Y.; Liang, X.; Xu, Z.; Li, Y.; Tan, X.;Qu,C.;Zhang,M.;Shen,Y.;etal.2026. Atomicthinking of llms: Decoupling and exploring ...

    arXiv 2026

  6. [10]

    Li, Y.; Li, Y.; Wang, X.; Jiang, Y.; Zhang, Z.; Zheng, X.; Wang, H.; Zheng, H.-T.; Huang, F.; Zhou, J.; et al

    Cognitive Mismatch in Multimodal Large Language Mod- els for Discrete Symbol Understanding.arXiv preprint arXiv:2603.18472. Li, Y.; Li, Y.; Wang, X.; Jiang, Y.; Zhang, Z.; Zheng, X.; Wang, H.; Zheng, H.-T.; Huang, F.; Zhou, J.; et al. 2025c. Benchmarking multimodal retrieval augmented generation with dynamic vqa dataset and self-adaptive planning agent. I...

    Pith/arXiv arXiv 2025

  7. [12]

    Liu,H.;Liu,J.;Liu,S.;Duan,H.;Li,Y.;Su,M.;Liu,X.;Zhai, G.;Fang,X.;Ma,Q.;etal.2025

    Tan- gramPuzzle: Evaluating Multimodal Large Language Mod- els with Compositional Spatial Reasoning.arXiv preprint arXiv:2601.16520. Liu,H.;Liu,J.;Liu,S.;Duan,H.;Li,Y.;Su,M.;Liu,X.;Zhai, G.;Fang,X.;Ma,Q.;etal.2025. ATLAS:AHigh-Difficulty, MultidisciplinaryBenchmarkforFrontierScientificReason- ing.arXiv preprint arXiv:2511.14366. Lu,J.;Qin,J.;Qiao,L.;Li,Y....

    arXiv 2025

  8. [13]

    Lu, P.; Chen, B.; Liu, S.; Thapa, R.; Boen, J.; and Zou, J

    Mathvista:Evaluatingmathematicalreasoningoffoundation modelsinvisualcontexts.arXivpreprintarXiv:2310.02255. Lu, P.; Chen, B.; Liu, S.; Thapa, R.; Boen, J.; and Zou, J. 2025b. Octotools: An agentic framework with extensible tools for complex reasoning.arXiv preprint arXiv:2502.11271. Luo,Y.;Wang,F.;Cheng,Q.;Yu,F.;Lei,H.;Yan,J.;Li,C.; Chen, J.; Zhao, Y.; Wa...

    Pith/arXiv arXiv

  9. [14]

    Lupidi, A.; Gauri, B.; Foster, T

    P1-VL: bridg- ing visual perception and scientific reasoning in physics Olympiads.arXiv preprint arXiv:2602.09443. Lupidi, A.; Gauri, B.; Foster, T. S.; Omari, B. A.; Magka, D.;Pepe,A.;Audran-Reiss,A.;Aghamelu,M.;Baldwin,N.; Cipolina-Kun,L.;etal.2026. AIRS-Bench:aSuiteofTasks for Frontier AI Research Science Agents.arXiv preprint arXiv:2602.06855. OpenAI

    arXiv 2026

  10. [15]

    Qin, Y.; Liang, S.; Ye, Y.; Zhu, K.; Yan, L.; Lu, Y.; Lin, Y.; Cong, X.; Tang, X.; Qian, B.; et al

    Multimath: Bridging visual and mathemati- cal reasoning for large language models.arXiv preprint arXiv:2409.00147. Qin, Y.; Liang, S.; Ye, Y.; Zhu, K.; Yan, L.; Lu, Y.; Lin, Y.; Cong, X.; Tang, X.; Qian, B.; et al

    Pith/arXiv arXiv

  11. [16]

    Schick, T.; Dwivedi-Yu, J.; Dessì, R.; Raileanu, R.; Lomeli, M.;Hambro,E.;Zettlemoyer,L.;Cancedda,N.;andScialom, T.2023

    Toolllm: Facilitating large language models to master 16000+ real- world apis.arXiv preprint arXiv:2307.16789. Schick, T.; Dwivedi-Yu, J.; Dessì, R.; Raileanu, R.; Lomeli, M.;Hambro,E.;Zettlemoyer,L.;Cancedda,N.;andScialom, T.2023. Toolformer:Languagemodelscanteachthemselves to use tools.Advances in neural information processing systems, 36: 68539–68551. ...

    Pith/arXiv arXiv 2023

  12. [17]

    Wang, M.; Lin, R.; Hu, K.; Jiao, J.; Chowdhury, N.; Chang, E.; and Patwardhan, T

    Challenging the boundaries of reasoning: An olympiad-level math benchmark for large language models.arXiv preprint arXiv:2503.21380. Wang, M.; Lin, R.; Hu, K.; Jiao, J.; Chowdhury, N.; Chang, E.; and Patwardhan, T

    Pith/arXiv arXiv

  13. [18]

    FrontierScience: Evaluating AI’sAbilitytoPerformExpert-LevelScientificTasks.arXiv preprint arXiv:2601.21165. Wei,J.;Wang,X.;Schuurmans,D.;Bosma,M.;Xia,F.;Chi, E.;Le,Q.V.;Zhou,D.;etal.2022.Chain-of-thoughtprompt- ing elicits reasoning in large language models.Advances in neural information processing systems, 35: 24824–24837. Wu,Q.;Bansal,G.;Zhang,J.;Wu,Y....

    arXiv 2022

  14. [19]

    InPro- ceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, 383–398

    Productagent: Benchmarking conversational prod- uct search agent with asking clarification questions. InPro- ceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, 383–398. Yu, F.; Wan, H.; Cheng, Q.; Zhang, Y.; Chen, J.; Han, F.; Wu, Y.; Yao, J.; Hu, R.; Ding, N.; et al. 2025a. HiPhO: How Far Are (M) LLMs fro...

    arXiv 2025

  15. [20]

    Scien- tists’ First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning.arXiv preprint arXiv:2506.10521

    arXiv

  16. [2022]

    InFindings of the Association for Computational Linguistics: ACL 2022, 3202–3213

    The past mis- takeisthefuturewisdom:Error-drivencontrastiveprobabil- ity optimization for chinese spell checking. InFindings of the Association for Computational Linguistics: ACL 2022, 3202–3213. Liu, D.; Kuang, J.; Li, Y.; Li, Y.; Yin, D.; Cao, H.; Sun, X.; Shen, Y.; Zheng, H.-T.; Lin, L.; et al

    2022

  17. [2023]

    Burgess, J.; Nirschl, J

    Chemcrow: Augmenting large-language models with chemistry tools.arXiv preprint arXiv:2304.05376. Burgess, J.; Nirschl, J. J.; Bravo-Sánchez, L.; Lozano, A.; Gupte, S. R.; Galaz-Montoya, J. G.; Zhang, Y.; Su, Y.; Bhowmik, D.; Coman, Z.; et al

    Pith/arXiv arXiv

  18. [2024]

    Li, X.; Wu, R.; Liu, X.; Wang, X.; Hu, J.; Bai, Z.; Zeng, B.; Liang, H.; Chen, L.; Chen, M.; et al

    LangChain. Li, X.; Wu, R.; Liu, X.; Wang, X.; Hu, J.; Bai, Z.; Zeng, B.; Liang, H.; Chen, L.; Chen, M.; et al. 2025a. SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoning.arXiv preprint arXiv:2511.08151. Li, Y.; Kuang, J.; Huang, H.; Xu, Z.; Liang, X.; Yu, Y.; Lu, W.; Li, Y.; Tan, X.; Qu, C.; et al. 2025b. One exam- ple shown, many...

    arXiv

  19. [2025]

    Besta, M.; Blach, N.; Kubicek, A.; Gerstenberger, R.; Podstawski, M.; Gianinazzi, L.; Gajda, J.; Lehmann, T.; Niewiadomski, H.; Nyczyk, P.; et al

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631. Besta, M.; Blach, N.; Kubicek, A.; Gerstenberger, R.; Podstawski, M.; Gianinazzi, L.; Gajda, J.; Lehmann, T.; Niewiadomski, H.; Nyczyk, P.; et al

    Pith/arXiv arXiv

  20. [2026]

    Anthropic

    Toward Na- tive Multimodal Modeling: A Roadmap.arXiv preprint arXiv:2605.25343. Anthropic

    Pith/arXiv arXiv