Pith. sign in

REVIEW 4 major objections 5 minor 41 references

ToolSciVer claims that reinforcement-learned, type-aware visual tool use improves multimodal scientific claim verification over non-tool, prompt-only, and general tool-use baselines.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Reinforcement-learned, type-aware visual tool calls improve multimodal scientific claim verification on SCIVER and MuSciClaims.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection Useful recipe for training VLMs to call evidence-extraction tools for scientific claim verification, but the unnamed chart-parser backend and a few methodological gaps mean the headline gains need some controls before I'd trust them fully. the 4 major comments →

arxiv 2607.16131 v1 pith:QIX7A7QK submitted 2026-07-17 cs.CL cs.AI

ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning

Binglin Zhou , Peng Shi , Ryo Kamoi , Nan Zhang , Rui Zhang This is my paper
classification cs.CL cs.AI
keywords multimodal scientific claim verificationtool-augmented vision-language modelsreinforcement learningGRPOvisual evidence extractiontable and chart reasoningfigure-centric claimsSCIVER and MuSciClaims
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal scientific claim verification asks whether a claim is supported by evidence that lives in tables, charts, and figures as well as text. The paper argues that the bottleneck is not reasoning but evidence acquisition: models fail because they cannot locate and read the decisive visual content. ToolSciVer attacks this by giving a vision-language model three tools specialized to scientific visuals and using GRPO with a composite reward to teach when and how to call them. Across two benchmarks and five open models, the trained tool-use policy beats plain chain-of-thought, prompt-only tool access, and general-purpose RL tool-use baselines, and it acquires the needed evidence more often. If the results hold, it suggests that learned, task-aligned tool use is a practical route to more trustworthy automated verification of figure-grounded scientific claims.

Core claim

ToolSciVer formulates multimodal scientific claim verification as an evidence-acquisition problem: before deciding whether a claim is supported, the model must first locate and read the specific table row, chart value, or figure region the claim depends on. To that end it equips a vision-language model with three tools matched to scientific visual types—Table Focus, which returns a chosen row or column from an OCR-derived table; Chart Parse, which converts a plot into structured text such as JSON entries; and Region Zoom, which crops a normalized coordinate region for dense or multi-panel figures. The policy is trained with GRPO under a composite reward that combines answer correctness, form

What carries the argument

The load-bearing object is the RL-trained tool-use policy over a compact, type-aware tool suite: Table Focus (OCR-backed row/column extraction), Chart Parse (chart-to-structured-text conversion), and Region Zoom (normalized-coordinate cropping). The behavior is shaped by a composite reward whose distinctive term is a group-relative tool-efficiency coefficient: among correct rollouts in a GRPO group, it rewards the trajectories that use close to the minimum number of tool calls and penalizes unnecessary calls when a direct answer suffices, while a dense tool-interaction penalty grades malformed calls and execution failures. This combination is what turns a raw ability to call tools into a sel

Load-bearing premise

The central claim assumes the external tools are reliable and that the 'stronger VLM' used for chart parsing is not so capable that it, rather than the trained tool-selection policy, is doing the decisive evidence reading—if the parser is near-oracle or overlaps with the model that annotates gold evidence, the reported gains would not measure what the headline claims.

What would settle it

Run ToolSciVer again with a fixed, publicly available chart parser (e.g., OCR plus a rule-based plot-to-table converter) instead of the stronger VLM, keeping all other components identical. If accuracy on the figure-centric MuSciClaims benchmark falls from the reported ~79% back toward the non-tool CoT level (~61% for the smallest backbone), the gains are attributable to the parser backend rather than the learned tool-use policy. A second check: compare the chart parser's outputs against the gold-evidence annotations used in the REAR analysis; if they come from the same model, the evidence-acq

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Across five open vision-language models from three families, ToolSciVer improves over non-tool chain-of-thought on both SCIVER and MuSciClaims, with the clearest gains on the figure-centric MuSciClaims benchmark.
  • Simply exposing the same tools through prompting is unstable and can underperform non-tool CoT on SCIVER for most backbones, so task-aligned training, not tool access alone, drives the improvement.
  • The efficiency reward raises accuracy by 4.24 points on SCIVER and 5.14 points on MuSciClaims while cutting tool calls per example from 1.05 to 0.74 and average response length from about 2,760 to 1,362 tokens.
  • The trained policy routes calls to the matching tool family (table tools for tables, chart parsing for charts, zoom for multi-panel figures) and increases tool execution success, and given the same gold evidence it verifies 11.33 more points than the base model.
  • The evidence acquisition rate (REAR) rises from about 83% for both non-tool and prompt-only variants to 90% for ToolSciVer, with the largest gain on multi-panel figures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because Chart Parse is described as using a 'stronger VLM', the accuracy gains on chart-heavy claims may partly reflect that backend's reading ability rather than the RL-trained selection policy; swapping in a fixed, weaker parser would separate the two contributions.
  • The same recipe—type-aware extraction tools plus a group-local efficiency reward—could transfer to other evidence-grounded tasks such as table-based fact-checking or document figure QA, where the bottleneck is also localizing the right visual evidence.
  • The REAR and gold-evidence annotations are produced by a strong external model; if that annotator is closely related to the chart parser, the reported evidence-acquisition improvement could be partially confounded. A human-verified evidence-locality annotation would make the comparison cleaner.
  • The efficiency coefficient uses the minimum tool calls among correct rollouts in the group, so the policy effectively learns to match the cheapest correct attempt per instance—suggesting it internalises instance difficulty rather than a fixed global tool budget.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces ToolSciVer, a tool-augmented framework for multimodal scientific claim verification (MSCV). It equips a VLM with three type-aware visual tools — OCR-based Table Focus, Chart Parse (whose backend is described only as a 'stronger VLM'), and Region Zoom — and trains the policy with GRPO under a composite reward that combines answer correctness, format validity, length control, tool-use efficiency, and tool-interaction penalties. Experiments on SCIVER and MuSciClaims across five open VLMs from three families compare against non-tool CoT, prompt-only tool use, and two RL-based tool-use baselines (VTOOL-R1, OpenThinkImg), with additional diagnostics on evidence acquisition, tool selection, efficiency, and evidence-conditioned reasoning. The paper claims state-of-the-art performance among tool-use baselines and attributes gains to learned, type-aware selective tool use.

Significance. The controlled comparison is a real strength: the same backbones, same training splits, and, for tool-use methods, the same scheduler are used, and all trainable baselines are trained on the same MSCV data. The paper also goes beyond final accuracy with mechanism-level analyses (REAR, tool routing, efficiency, evidence-conditioned reasoning), and it ships code. If the headline results are reproducible, the task-aligned tool suite and RL-based selective tool use would be a meaningful contribution to MSCV. However, the central attribution of the accuracy gains to the learned policy is currently not separable from the capabilities of the unnamed Chart Parse backend, and the main table lacks significance testing, so the strength of the claim is not yet fully supported.

major comments (4)
  1. [§3.3, Table 2] The Chart Parse tool is described as using a 'stronger VLM' but the model is never named, its version is not given, and no error rate or ablation is reported. Table 2's largest gains are on MuSciClaims, a figure-centric benchmark, so the policy's apparent evidence acquisition could be substantially outsourced to the hidden backend. This is load-bearing for the central claim: without holding the parser fixed or replacing it with a known open-source parser (and corrupting/ablating its output), one cannot tell whether the learned RL policy or the hidden parser produces the reported gains. Please disclose the backend and add an ablation with the backend replaced by an open-source chart parser, or with gold structured chart text, reporting accuracy deltas.
  2. [§5.2, Table 6] REAR and the evidence-conditioned evaluation use 'GPT-5.5-assisted annotation' to define claim-relevant evidence. The same section uses the Chart Parse tool, whose backend is not named. If the Chart Parse backend is GPT-5.5 or a model of the same family, then the REAR diagnostic is partly circular: the tool's output is judged relevant by a model from the same family that may also produce that output. Table 6 then evaluates reasoning on gold evidence generated by that same model. Please state explicitly whether Chart Parse's backend is the same model family as the annotation model, provide an independence guarantee, and ideally validate REAR with human-annotated evidence or with evidence from a different model.
  3. [Table 2] The main accuracy table contains no confidence intervals, significance tests, or multiple-seed variation. Some of the headline comparisons are very small — e.g., Qwen3.5-4B ties VTOOL-R1 on SCIVER Overall (81.20 vs 81.20), Qwen3.5-9B leads by 0.20 points on SCIVER and 0.20 on MuSciClaims, and many subset differences are in the 1–2 point range. For the claim of 'consistently improves' and 'state-of-the-art performance,' the authors should report bootstrap confidence intervals and/or paired significance tests (e.g., McNemar for per-example predictions), or repeat training with multiple random seeds and report mean ± std. This is needed to rule out noise as an explanation for the observed cross-backbone gains.
  4. [Abstract, §5.1] The paper calls ToolSciVer 'state-of-the-art' while its own Table 2 shows Claude Sonnet 4.6 at 79.41 on MuSciClaims Overall, above the best ToolSciVer result (78.81). The qualification 'compared with strong tool-use baselines' appears only in the abstract, but the phrase 'achieves state-of-the-art performance on different backbone model families' is overbroad. Please either restrict the claim explicitly to open-source models and to the controlled tool-use comparison, or benchmark against proprietary models/other published MSCV systems under the same setting before making a global SOTA statement.
minor comments (5)
  1. [§4 / §5.1] The proprietary GPT-4o, GPT-5.4, and Claude Sonnet 4.6 rows are external references and not part of the controlled comparison, yet they appear in the main results table. Consider placing them in a separate table or clearly marking them so readers do not compare them against open-source tool-use methods as if the comparison were controlled.
  2. [§3.2, Figure 1] The overview figure and the run-through example are helpful, but the figure caption is very dense. Splitting the example into a separate panel with numbered tool-call steps would improve clarity.
  3. [§4, Baselines] The prompt-only tool-use baseline is cited as [25] (Interleaving retrieval with chain-of-thought reasoning), which is about retrieval rather than tool use. A more appropriate citation would be ReAct [36] or a similar tool-using prompting method; please verify the citation.
  4. [Appendix B.4] The diagnostic subsets are drawn from SCIVER for table/chart and from MuSciClaims for general figures, but the sample sizes (100 per type) are not justified and the subsets are not shown to be balanced in difficulty. Please report the per-subset sample sizes and, ideally, a human or LLM check of the construction criteria.
  5. [§5.2] The REAR definition says 'contains the required visual evidence' but the matching criterion is not formalized (substring? semantic? exact value?). Please specify the matching procedure and report inter-annotator agreement if annotation is used.

Circularity Check

0 steps flagged

No significant circularity: main results rest on external benchmark accuracy, not on fitted or self-referential definitions.

full rationale

The paper's central claim is an empirical accuracy comparison on SCIVER and MuSciClaims against non-tool, prompt-only, and RL tool-use baselines. Those accuracy numbers are measured against external ground-truth labels, so the headline result is not defined in terms of the method's own outputs. The GRPO reward in Eq. (1) uses the ground-truth label for the answer-reward term; using labels as training reward and then reporting held-out accuracy is standard RL evaluation, not a circular prediction. The REAR and gold-evidence diagnostics in Sections 5.2 and 5.5 use GPT-5.5-assisted annotations of claim-relevant evidence, and the Chart Parse tool is described only as 'using stronger VLM' in Section 3.3. The paper nowhere states that the chart parser and the annotator are the same model, so any claim that the diagnostic is self-referential would require speculation rather than a quoted reduction. The stated Limitation acknowledges that performance is bounded by external tool reliability, which is a robustness and transparency concern, not a circularity. No load-bearing self-citation chain or ansatz-smuggling via citation is present. Therefore no circular step can be exhibited from the manuscript text, and the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

Most free parameters are RL hyperparameters chosen by hand. The more important postulate is that external tool outputs are accurate and independent of the evaluation annotator. The central contribution does not introduce new theoretical entities.

free parameters (5)
  • Reward weights λ_fmt, λ_len, λ_tool = 0.1 (each)
    Hand-set weights in Eq. (1); no sensitivity analysis despite directly shaping the learned tool policy.
  • Maximum tool budget c_max = 5
    Hard cap on tool calls per rollout; no ablation shown.
  • Length soft-cache C and max length L_max = C=768, L_max=4096
    Set by hand; defines where the length penalty begins.
  • Tool-interaction penalty coefficients = -0.20, -0.10, -0.30, -0.15, -0.40
    Hand-set per-error penalties in Eq. (3); no sensitivity analysis.
  • GRPO group size G and KL coefficient β
    Not reported; the GRPO objective (Eq. 4) and advantage normalization depend on these values, so exact reproduction is impossible without them.
axioms (4)
  • standard math GRPO with clipped surrogate objective is a valid policy update for this reward
    Eq. (4) follows Shao et al. 2024; treated as background, with no convergence proof or variance analysis in the paper.
  • domain assumption SCIVER and MuSciClaims binary labels are ground truth and splits are not leaked
    All accuracy numbers depend on benchmark correctness and split integrity; leakage into the tool parser or training data is not analyzed.
  • domain assumption Tool outputs (OCR rows, parsed chart values, zoomed crops) are accurate enough that downstream errors do not dominate
    Acknowledged in the Limitation section as a bound; the main results assume this bound is not exceeded.
  • ad hoc to paper The unnamed Chart Parse backend and the GPT-5.5-annotated REAR gold evidence are independent
    If the same model is used for tool observation and for defining 'claim-relevant evidence,' REAR is self-confirming; the paper provides no disambiguation or ablation.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning." pith.science (2026). https://pith.science/paper/QIX7A7QK

@misc{pith2026260716131,
  author       = {Pith},
  title        = {Pith review of: ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QIX7A7QK}},
  note         = {Machine review of arXiv:2607.16131}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodal Scientific Claim Verification (MSCV) requires models to verify scientific claims using visually grounded evidence from papers, including figures, tables, charts, and textual context. However, existing methods often fail because they struggle to locate decisive visual evidence, accurately read structured scientific visuals, and integrate multimodal observations into reliable reasoning. We introduce ToolSciVer, the first tool-augmented framework for MSCV to our knowledge. ToolSciVer equips a VLM with three type-aware visual tools, table row/column focus, chart-to-structure parsing, and high-resolution region zoom, which convert dense scientific visuals into explicit, claim-facing evidence, and trains the policy with Group Relative Policy Optimization (GRPO) under a composite reward of answer correctness, format validity, length control, tool-use efficiency, and tool-validity penalties. Experiments on SciVer and MuSciClaims datasets on five VLMs from three model families (Qwen, InternVL, Gemma) demonstrate that our method achieves superior performance compared to four competitive baselines including prompting-based and RL-based tool-use methods, highlighting the effectiveness of learned, type-aware tool use for scientific claim verification.

Figures

Figures reproduced from arXiv: 2607.16131 by Binglin Zhou, Nan Zhang, Peng Shi, Rui Zhang, Ryo Kamoi.

Figure 3
Figure 3. Figure 3: Validation loss across epochs. Tool Call Available Tool Set Chart Parse Table Focus Region Zoom Turn 1 Reasoning: Need ResNet-50 performance. Use Table Focus for [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 2
Figure 2. Figure 2: Case study of chart evidence acquisition. The [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training dynamics for sparse versus dense tool-interaction penalties. Curves are smoothed [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 2 canonical work pages

  1. [1]

    Zero-shot scientific claim verification using LLMs and citation text

    Carlos Alvarez, Maxwell Bennett, and Lucy Wang. Zero-shot scientific claim verification using LLMs and citation text. InProceedings of the Fourth Workshop on Scholarly Document Pro- cessing (SDP 2024), pages 269–276, Bangkok, Thailand, 2024. Association for Computational Linguistics. URLhttps://aclanthology.org/2024.sdp-1.25/

    2024

  2. [2]

    TabFact: A large-scale dataset for table-based fact verification

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. TabFact: A large-scale dataset for table-based fact verification. InInternational Conference on Learning Representations, 2020. URL https://openreview. net/forum?id=rkeJRhNYDH

    2020

  3. [3]

    Gemma 4: Byte for byte, the most capable open models, April 2026

    Clement Farabet and Olivier Lacombe. Gemma 4: Byte for byte, the most capable open models, April 2026. URL https://blog.google/innovation-and-ai/technology/ developers-tools/gemma-4/

    2026

  4. [4]

    Refocus: Visual editing as a chain of thought for structured image understanding

    Xingyu Fu, Minqian Liu, Zhengyuan Yang, John Richard Corring, Yijuan Lu, Jianwei Yang, Dan Roth, Dinei Florencio, and Cha Zhang. Refocus: Visual editing as a chain of thought for structured image understanding. InProceedings of the 42nd International Conference on Machine Learning, volume 267 ofProceedings of Machine Learning Research, pages 17783–17805. ...

    2025

  5. [5]

    Visual programming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14953–14962, 2023

    2023

  6. [6]

    Format matters: The robustness of multimodal LLMs in reviewing evidence from tables and charts

    Xanh Ho, Yun-Ang Wu, Sunisth Kumar, Florian Boudin, Atsuhiro Takasu, and Akiko Aizawa. Format matters: The robustness of multimodal LLMs in reviewing evidence from tables and charts. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31014–31022, 2026. doi: 10.1609/aaai.v40i37.40361. URL https://arxiv.org/abs/2511. 10075

    doi:10.1609/aaai.v40i37.40361 2026

  7. [7]

    SciClaimEval: Cross-modal claim verification in scientific papers

    Xanh Ho, Yun-Ang Wu, Sunisth Kumar, Tian Cheng Xia, Florian Boudin, André Greiner-Petter, and Akiko Aizawa. SciClaimEval: Cross-modal claim verification in scientific papers. In Proceedings of the 15th Language Resources and Evaluation Conference, Palma de Mallorca, Spain, 2026. ELRA Language Resources Association. doi: 10.48550/arXiv.2602.07621. URL http...

    doi:10.48550/arxiv.2602.07621 2026

  8. [8]

    Smith, and Ranjay Krishna

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. InAdvances in Neural Information Processing Systems, volume 37, 2024. doi: 10.52202/079017-4423

    doi:10.52202/079017-4423 2024

  9. [9]

    Verltool: Towards holistic agentic reinforcement learning with tool use.arXiv preprint arXiv:2509.01055, 2025

    Dongfu Jiang, Yi Lu, Zhuofeng Li, Zhiheng Lyu, Ping Nie, Haozhe Wang, Alex Su, Hui Chen, Kai Zou, Chao Du, Tianyu Pang, and Wenhu Chen. Verltool: Towards holistic agentic reinforcement learning with tool use.arXiv preprint arXiv:2509.01055, 2025. doi: 10.48550/ arXiv.2509.01055. URLhttps://arxiv.org/abs/2509.01055

    doi:10.48550/arxiv.2509.01055 2025

  10. [10]

    DVQA: Un- derstanding data visualizations via question answering

    Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. DVQA: Un- derstanding data visualizations via question answering. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5648–5656,

  11. [11]

    FigureQA: An annotated figure dataset for visual reasoning

    Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Akos Kadar, Adam Trischler, and Yoshua Bengio. FigureQA: An annotated figure dataset for visual reasoning. InICLR 2018 Workshop, 2018. doi: 10.48550/arXiv.1710.07300. URL https://openreview.net/forum? id=H1mz0OyDz

    Pith/arXiv arXiv doi:10.48550/arxiv.1710.07300 2018

  12. [12]

    Multicheck: Strengthening web trust with unified multimodal fact verification.arXiv preprint arXiv:2508.05097, 2025

    Aditya Kishore, Gaurav Kumar, and Jasabanta Patro. Multicheck: Strengthening web trust with unified multimodal fact verification.arXiv preprint arXiv:2508.05097, 2025. doi: 10.48550/ arXiv.2508.05097. URLhttps://arxiv.org/abs/2508.05097. 10

    work page doi:10.48550/arxiv.2508.05097 2025

  13. [13]

    MuSciClaims: Multimodal scientific claim ver- ification

    Yash Kumar Lal, Manikanta Bandham, Mohammad Saqib Hasan, Apoorva Kashi, Mahnaz Koupaee, and Niranjan Balasubramanian. MuSciClaims: Multimodal scientific claim ver- ification. InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computa- tional Ling...

    2025

  14. [14]

    DePlot: One-shot visual language reasoning by plot-to-table translation

    Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Ken- ton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. DePlot: One-shot visual language reasoning by plot-to-table translation. InFindings of the Association for Computational Linguistics: ACL 2023, pages 10381–10399, Toronto, Canada, 2023. As- sociation f...

    doi:10.18653/v1/2023.findings-acl.660 2023

  15. [15]

    MatCha: Enhancing visual language pretraining with math reasoning and chart derendering

    Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Eisenschlos. MatCha: Enhancing visual language pretraining with math reasoning and chart derendering. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12756...

    2023

  16. [16]

    SCITAB: A challenging benchmark for compositional reasoning and claim verification on scientific tables

    Xinyuan Lu, Liangming Pan, Qian Liu, Preslav Nakov, and Min-Yen Kan. SCITAB: A challenging benchmark for compositional reasoning and claim verification on scientific tables. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7787–7813, Singapore, 2023. Association for Computational Linguistics. doi: 10.18653/ v...

    2023

  17. [17]

    ChartQA: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. InFindings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279, Dublin, Ireland,

    2022

  18. [18]

    Toolrl: Reward is all tool learning needs.arXiv preprint arXiv:2504.13958,

    Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs.arXiv preprint arXiv:2504.13958,

    Pith/arXiv arXiv

  19. [19]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https: //qwen.ai/blog?id=qwen3.5

    2026

  20. [20]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. InAdvances in Neural Information Processing Systems, 2023. URLhttps://openreview.net/forum?id=Yacmpz84TH

    2023

  21. [21]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. doi: 10.48550/arXiv.2402.03300. URLhttps://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.03300 2024

  22. [22]

    Openthinkimg: Learning to think with images via visual tool reinforcement learning.arXiv preprint arXiv:2505.08617, 2025

    Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, and Yu Cheng. Openthinkimg: Learning to think with images via visual tool reinforcement learning.arXiv preprint arXiv:2505.08617, 2025. doi: 10.48550/arXiv.2505.08617. URLhttps://arxiv.org/abs/2505.08617

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.08617 2025

  23. [23]

    ViperGPT: Visual inference via python execution for reasoning

    Dídac Surís, Sachit Menon, and Carl V ondrick. ViperGPT: Visual inference via python execution for reasoning. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2023

    2023

  24. [24]

    FEVER: A large-scale dataset for fact extraction and VERification

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER: A large-scale dataset for fact extraction and VERification. InProceedings of the 2018 Conference 11 of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819, New Orleans, Louisiana,

    2018

  25. [25]

    Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 10014–10037, 2023

    2023

  26. [26]

    Fact or fiction: Verifying scientific claims

    David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. Fact or fiction: Verifying scientific claims. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online, 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020. emnl...

    doi:10.18653/v1/2020 2020

  27. [27]

    SciFact-open: Towards open-domain scientific claim verification

    David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, and Hannaneh Hajishirzi. SciFact-open: Towards open-domain scientific claim verification. InFindings of the Association for Computational Linguistics: EMNLP 2022, pages 4719–4734, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics. doi: 10.18653/v1/20...

    doi:10.18653/v1/2022 2022

  28. [28]

    doi: 10.18653/v1/N18-1074

    Association for Computational Linguistics. doi: 10.18653/v1/N18-1074. URL https: //aclanthology.org/N18-1074/

    doi:10.18653/v1/n18-1074

  29. [29]

    SciVer: Evaluating foundation models for multimodal scientific claim verification

    Chengye Wang, Yifei Shen, Zexi Kuang, Arman Cohan, and Yilun Zhao. SciVer: Evaluating foundation models for multimodal scientific claim verification. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8562–8579, Vienna, Austria, 2025. Association for Computational Linguistics. doi: 10. ...

    2025

  30. [30]

    Piecing it all together: Verifying multi-hop multimodal claims

    Haoran Wang, Aman Rangapur, Xiongxiao Xu, Yueqing Liang, Haroon Gharwi, Carl Yang, and Kai Shu. Piecing it all together: Verifying multi-hop multimodal claims. InProceedings of the 31st International Conference on Computational Linguistics, pages 7453–7469, Abu Dhabi, UAE, 2025. Association for Computational Linguistics. URL https://aclanthology.org/ 2025...

    2025

  31. [31]

    Acting less is reasoning more! teaching model to act efficiently.arXiv preprint arXiv:2504.14870, 2025

    Hongru Wang, Cheng Qian, Wanjun Zhong, Xiusi Chen, Jiahao Qiu, Shijue Huang, Bowen Jin, Mengdi Wang, Kam-Fai Wong, and Heng Ji. Acting less is reasoning more! teaching model to act efficiently.arXiv preprint arXiv:2504.14870, 2025. doi: 10.48550/arXiv.2504.14870. URL https://arxiv.org/abs/2504.14870

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.14870 2025

  32. [32]

    MultiVerS: Improving scientific claim verification with weak supervision and full-document context

    David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, and Hannaneh Hajishirzi. MultiVerS: Improving scientific claim verification with weak supervision and full-document context. InFindings of the Association for Computational Linguistics: NAACL 2022, pages 61–76, Seattle, United States, 2022. Association for Computational Linguistics. doi: 10.186...

    2022

  33. [33]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  34. [34]

    Vtool-r1: Vlms learn to think with images via reinforcement learning on multimodal tool use.arXiv preprint arXiv:2505.19255, 2025

    Mingyuan Wu, Jingcheng Yang, Jize Jiang, Meitang Li, Kaizhuo Yan, Hanchao Yu, Minjia Zhang, Chengxiang Zhai, and Klara Nahrstedt. Vtool-r1: Vlms learn to think with images via reinforcement learning on multimodal tool use.arXiv preprint arXiv:2505.19255, 2025. doi: 10.48550/arXiv.2505.19255. URLhttps://arxiv.org/abs/2505.19255

    doi:10.48550/arxiv.2505.19255 2025

  35. [35]

    MM-REACT: Prompting ChatGPT for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. MM-REACT: Prompting ChatGPT for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023. doi: 10.48550/arXiv. 2303.11381. URLhttps://arxiv.org/abs/2303.11381. 12

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.11381 2023

  36. [36]

    Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025. doi: 10.48550/arXiv.2508.18265. URLhttps://arxiv.org/abs/2508.18265

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.18265 2025

  37. [37]

    M 2-TabFact: Multi-document multi-modal fact verification with vi- sual and textual representations of tabular data

    Mingyang Zhou, Lingyu Zhang, Sophia Horng, Maximillian Chen, Kung-Hsiang Huang, and Shih-Fu Chang. M 2-TabFact: Multi-document multi-modal fact verification with vi- sual and textual representations of tabular data. InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 26239–26256, Vienna, Austria, 2025. Associa- tion for Computat...

    work page doi:10.18653/v1/2025.findings-acl.1345 2025

  38. [40]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InThe Eleventh International Conference on Learning Representations, 2023. URL https://openreview. net/forum?id=WE_vluYUL-X

    2023

  39. [2018]

    URL https://openaccess.thecvf.com/content_cvpr_2018/html/Kafle_ DVQA_Understanding_Data_CVPR_2018_paper.html

  40. [2022]

    doi: 10.18653/v1/2022.findings-acl.177

    Association for Computational Linguistics. doi: 10.18653/v1/2022.findings-acl.177. URLhttps://aclanthology.org/2022.findings-acl.177/

    doi:10.18653/v1/2022.findings-acl.177 2022

  41. [2025]

    URLhttps://arxiv.org/abs/2504.13958

    doi: 10.48550/arXiv.2504.13958. URLhttps://arxiv.org/abs/2504.13958

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.13958

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.