Pith. sign in

REVIEW 2 major objections 106 references

A vision-language model that rechecks its own answers during RL training learns stronger multimodal reasoning without any external verifier.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 07:59 UTC pith:NLBDJIXQ

load-bearing objection Clean multi-turn GRPO recipe that interleaves binary self-verification and shows consistent 2–4 pt gains plus declining turns; the missing random-verdict control is a real but not fatal gap. the 2 major comments →

arxiv 2607.10966 v1 pith:NLBDJIXQ submitted 2026-07-13 cs.AI

SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning

Mingyuan Wu , Jingcheng Yang , Shengyi Qian , Xudong Wang , Jize Jiang , Qifan Wang , Aashu Singh , Khoi Pham
show 6 more authors
Fei Liu Zhaolun Su Zhuokai Zhao Klara Nahrstedt Jianyu Wang Hanchao Yu
This is my paper
classification cs.AI
keywords self-verificationmultimodal reasoningvision-language modelsreinforcement learningGRPOself-correctionbootstrapping
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a vision-language model can bootstrap its own multimodal reasoning by folding a simple self-check into reinforcement learning rollouts. For each query the model first proposes an answer, then issues a binary Yes/No verdict on that answer using the same weights; a No forces a rethink while a Yes (or a turn limit) freezes the answer for the outcome reward. Trained this way with group-relative policy optimization, the method substantially lifts accuracy on chart, table, and general visual-reasoning benchmarks relative to ordinary GRPO run on identical data and hyperparameters. Over training the model gradually stops needing multiple verification rounds yet keeps the higher accuracy, which the authors read as evidence that generation and verification converge and that self-correction has been internalized. The practical claim is that inference-time self-refinement and RL post-training can be fused into one loop that needs no extra critics or human labels.

Core claim

Integrating binary self-verification turns into multi-turn GRPO rollouts for vision-language models produces large accuracy gains over standard single-pass GRPO on the same data, while the average number of verification turns falls and pure-run accuracy approaches the accuracy obtained with final verification, showing that the policy learns to emit answers it would itself accept.

What carries the argument

Self-Verified Reasoner (SVR-R1): a multi-turn rollout in which the same policy first generates a candidate answer, then emits a binary Yes/No self-verdict; No triggers a rethink trigger and regeneration, Yes or a turn cap freezes the answer for the outcome-based reward, with verification tokens masked from the GRPO loss.

Load-bearing premise

That a vision-language model's own binary Yes/No check is reliable enough to supply a useful training signal rather than random or systematically biased noise.

What would settle it

Train the identical architecture and data under standard GRPO versus SVR-R1; if the self-verification loop yields no accuracy lift on the same chart/table hold-outs and verification-turn counts do not decline while pure-run accuracy stays flat, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The paper proposes SVR-R1, a multi-turn GRPO framework for VLMs that interleaves generation and binary self-verification under shared weights. For each query the model produces an answer, then a Yes/No self-verdict; No triggers a rethink (up to a turn cap), after which only the final answer receives an outcome-based reward (LLM judge or rule match). Verification tokens are masked from the loss. On ChartQA, TableVQA and ThinkLite-VL, SVR-R1 (3B/7B Qwen2.5-VL) improves accuracy over matched single-pass GRPO by roughly 2–4 points; training dynamics show declining verification turns while pure-run accuracy approaches final-verification accuracy. The authors interpret this as internalization of self-correction that closes the verification–generation gap, without external critics or process rewards.

Significance. If the gains are genuinely driven by a useful self-verdict signal, the work supplies a simple, open-source recipe that sits at the underexplored intersection of inference-time self-refinement and RL post-training for VLMs. Strengths include controlled same-data/same-hyperparameter comparisons, pure-run vs final-verification and entropy-control ablations, honest reporting of failure on the hard 11k MCTS subset, and planned code release. These elements make the empirical contribution reproducible and useful even if the precise mechanism remains partly open.

major comments (2)
  1. The central attribution of the 2–4-point gains (Tables 1–3, Figures 2 and 5) to a better-than-random binary self-verdict is not isolated. The only controlled baseline is single-pass GRPO. There is no multi-turn control that forces the same number of generation steps (or the same rethink-trigger text of Figure 9) while replacing the model’s Yes/No with a random or always-Yes oracle. Without that ablation it remains possible that extra tokens, forced rethinking, or prompt phrasing, rather than a useful self-verdict, drive both the accuracy lift and the later decline in verification turns. The hard-subset failure (Appendix B.3) is consistent with either story.
  2. Section 3.2 and the verification–generation-gap citation in Section 1 treat the binary self-verdict as a useful learning signal, yet the manuscript never reports the accuracy of the self-verifier itself (agreement with ground truth, false-positive/false-negative rates, or calibration) on the training or validation distributions. If the verdict is near chance or systematically biased, the multi-turn protocol injects noise; documenting verifier quality is load-bearing for the claimed mechanism.

Circularity Check

0 steps flagged

No circularity: empirical multi-turn RL method with external outcome rewards; no derivation reduces a claimed result to its inputs by construction.

full rationale

SVR-R1 is an empirical systems/method paper. The training objective (Eq. 1–2) is standard GRPO on final answers only; self-verification tokens are explicitly loss-masked and receive no process reward. Outcome rewards are supplied by an external LLM judge or rule-based matcher against ground-truth labels (Section 3.3, 4.1). The reported accuracy gains (Tables 1–3) and the observed decline in verification turns (Figures 2, 5, 15) are measured against held-out benchmarks and a matched single-pass GRPO baseline trained on identical data and hyperparameters; they are not algebraic consequences of any fitted free parameter or self-defined quantity. No uniqueness theorem, ansatz, or prior result by the same authors is invoked as a load-bearing premise that forces the central claim. The verification–generation gap is cited as motivation (Song et al., 2025; Liao et al., 2025) but is not used to define or tautologically guarantee the reported improvements. Consequently the derivation chain contains no circular step.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 1 invented entities

Empirical method paper. Free parameters are ordinary RL hyper-parameters chosen by the authors; axioms are standard domain assumptions about GRPO stability and the usefulness of binary self-verification; no new physical or mathematical entities are postulated.

free parameters (5)
  • max_self_verification_rounds = 3 (main); 5 (hard split)
    Hard cap (3 for chart/table, 3 or 5 for ThinkLite) that terminates rethink loops; chosen by hand and directly affects both compute and final answer selection.
  • KL_coefficient_beta = 1e-3 / 1e-2
    Penalty strength against the reference policy (1e-3 chart/table, 1e-2 ThinkLite); standard GRPO free parameter that controls update size.
  • clip_epsilon / clip_high = 2.0 (main)
    GRPO clipping thresholds (standard 2.0, high-entropy ablation 2.8); chosen to control exploration.
  • rollout_group_size = 16 / 32
    Number of Monte-Carlo samples per prompt for group-relative advantage (16 or 32); affects variance of the GRPO baseline.
  • learning_rate = 1e-6
    AdamW initial LR 1e-6 used for all runs; standard free hyper-parameter.
axioms (4)
  • domain assumption A VLM can act as a better-than-random binary verifier of its own outputs, supplying a useful learning signal when interleaved with generation.
    Stated in Section 1 via the verification-generation gap hypothesis and used throughout the multi-turn protocol (Section 3.2).
  • ad hoc to paper Outcome-based binary reward (LLM judge or rule match) is sufficient; no process reward on intermediate verification tokens is required.
    Explicit design choice in Section 3.3; verification tokens are masked from the loss.
  • domain assumption GRPO with group-relative advantages remains stable under multi-turn rollouts that include self-verification context.
    Assumed by adopting GRPO (Shao et al.) and the asynchronous multi-turn framework (Sheng et al.) without additional stability proofs.
  • ad hoc to paper Masking self-verification tokens prevents conflicting objectives between generation and verification.
    Section 3.3, inspired by Search-R1; not derived, chosen to stabilize training.
invented entities (1)
  • Self-Verified Reasoner (SVR-R1) multi-turn protocol no independent evidence
    purpose: Interleave generation and binary self-verification inside GRPO rollouts so that only the first self-affirmed (or turn-capped) answer receives the outcome reward.
    The named framework is the paper's central algorithmic contribution; it is a procedural construct rather than a new physical entity, and its value is measured solely by the reported accuracy gains.

pith-pipeline@v1.1.0-grok45 · 22292 in / 3107 out tokens · 34396 ms · 2026-07-14T07:59:21.504347+00:00 · methodology

0 comments
read the original abstract

We introduce Self-Verified Reasoner (SVR-R1), a multi-turn RL framework that turns a model's own verification into a learning signal for multimodal reasoning. For each query, the model proposes an answer using the same weights, and issues a binary self-verdict (Yes/No). A 'No' triggers a second-chance rethink; a 'Yes,' or a turn cap, finalizes the output for computing the outcome-based reward. SVR-R1 is implemented with GRPO and an asynchronous multi-turn rollout framework and needs no external supervision or auxiliary critics. We evaluate SVR-R1 on vision-language reasoning benchmarks and show that it improves accuracy by a large margin over strong standard GRPO baselines. Training dynamics show decreasing reliance on verification-fewer verification turns, yet higher test accuracy-indicating that the gap between verification and generation narrows as the policy internalizes self-correction and chooses the most confident answer via our framework. SVR-R1 bridges the less explored intersection of inference-time self-refinement and RL training for VLMs, offering a simple yet effective recipe for bootstrapping multimodal reasoning. We will open-source \textbf{SVR-R1} to facilitate future research in VLMs.

Figures

Figures reproduced from arXiv: 2607.10966 by Aashu Singh, Fei Liu, Hanchao Yu, Jianyu Wang, Jingcheng Yang, Jize Jiang, Khoi Pham, Klara Nahrstedt, Mingyuan Wu, Qifan Wang, Shengyi Qian, Xudong Wang, Zhaolun Su, Zhuokai Zhao.

Figure 1
Figure 1. Figure 1: Merging VLM Self-Verification Loop into RL Rollout When given a second chance to think, hu￾mans often reason their way toward bet￾ter solutions on complex problems. Re￾cently, Large Language Models (LLMs) have demonstrated a similar pattern of iterative rethinking in the reasoning pro￾cess (OpenAI, 2024a), particularly when fine-tuned with reinforcement learning (RL) on task-specific rewards (DeepSeek￾AI, … view at source ↗
Figure 2
Figure 2. Figure 2: Validation Reasoning Accuracy (%) vs. Training Steps. SVR-R1 compared with standard GRPO (Shao et al., 2024) on the Qwen2.5-VL (Bai et al., 2025) 3B model, with Mean number of Turns decreasing. We demonstrate the effectiveness of SVR￾R1 across multiple challenging multi-modal table and chart reasoning benchmarks (Fu et al., 2025), as well as general reasoning tasks (Wang et al., 2025b), showing signifi￾can… view at source ↗
Figure 3
Figure 3. Figure 3: Multi-rollout with self-verification. Each rollout, along with additional prompts, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Multi-Modal GRPO w. Self Verification Training Pipeline. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: SVR-R1 surpasses standard GRPO on table tasks. Decreased Verification Turns and In￾creased Confidence. Throughout training, we observe that both validation and train￾ing verification turns gradually decrease, as the models become increasingly confident in their initial answers and tend to affirm their responses: In Figures 2 and 15, these curves converge to approximately 2 - one generation step followed by… view at source ↗
Figure 6
Figure 6. Figure 6: SVR-R1 outperforms high-entropy baselines. High-entropy variants are trained [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Self-reflection of trained model in an example from [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Self-Verifier Turn, with Previous Queries and Response. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Self-Generator Turn with the Rethinking Trigger. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Full Prompt for VLM Inference or Rollout, with an In-context Example and Clear [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Prompt for LLM Judge in Semi-open Questions. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Full Prompt for VLM Inference or Rollout for ThinkLite-VL Dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Mean number of Turns vs. Training Steps on train/val splits. We allow a maximum of five verification turns, counting each generation and verification round. et al., 2015); FigureQA, ScienceQA, and OK-VQA for image understanding (Kahou et al., 2018; Lu et al., 2022a; Marino et al., 2019); and IconQA and TabMWP for chart understanding (Lu et al., 2022b; 2023). Difficult Data Selection: 11K Split. A key cont… view at source ↗
Figure 15
Figure 15. Figure 15: SVR-R1 surpasses standard GRPO on table tasks. [PITH_FULL_IMAGE:figures/full_fig_p018_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

106 extracted references · 12 linked inside Pith

  1. [1]

    and Le, Quoc V

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Ichter, Brian and Xia, Fei and Chi, Ed H. and Le, Quoc V. and Zhou, Denny , title =. Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =. 2022 , isbn =

    2022

  2. [2]

    2024 , eprint=

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters , author=. 2024 , eprint=

    2024

  3. [3]

    Le and Ed H

    Xuezhi Wang and Jason Wei and Dale Schuurmans and Quoc V. Le and Ed H. Chi and Sharan Narang and Aakanksha Chowdhery and Denny Zhou , title =. The Eleventh International Conference on Learning Representations,. 2023 , url =

    2023

  4. [4]

    Chain-of-Thought Reasoning Without Prompting , url =

    Wang, Xuezhi and Zhou, Denny , booktitle =. Chain-of-Thought Reasoning Without Prompting , url =

  5. [5]

    2015 , eprint=

    Distilling the Knowledge in a Neural Network , author=. 2015 , eprint=

    2015

  6. [6]

    2024 , eprint=

    Self-Improvement in Language Models: The Sharpening Mechanism , author=. 2024 , eprint=

    2024

  7. [7]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , url =

    DeepSeek-AI , journal =. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , url =

  8. [8]

    Proceedings of CVPR , year=

    MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI , author=. Proceedings of CVPR , year=

  9. [9]

    arXiv preprint arXiv:2404.12390 , year=

    BLINK: Multimodal Large Language Models Can See but Not Perceive , author=. arXiv preprint arXiv:2404.12390 , year=

    Pith/arXiv arXiv

  10. [10]

    2024 , url =

    Llama 3 Model Card , author=. 2024 , url =

    2024

  11. [11]

    Aha Moment

    R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model , author=. 2025 , eprint=

    2025

  12. [12]

    Chen, Liang and Li, Lei and Zhao, Haozhe and Song, Yifan and Vinci , title =

  13. [13]

    2025 , eprint=

    R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization , author=. 2025 , eprint=

    2025

  14. [14]

    2025 , eprint=

    Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models , author=. 2025 , eprint=

    2025

  15. [15]

    2025 , eprint=

    Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement , author=. 2025 , eprint=

    2025

  16. [16]

    2025 , eprint=

    OpenVLThinker: An Early Exploration to Complex Vision-Language Reasoning via Iterative Self-Improvement , author=. 2025 , eprint=

    2025

  17. [17]

    2025 , eprint=

    VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning , author=. 2025 , eprint=

    2025

  18. [18]

    2025 , eprint=

    SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models , author=. 2025 , eprint=

    2025

  19. [19]

    2025 , eprint=

    ReTool: Reinforcement Learning for Strategic Tool Use in LLMs , author=. 2025 , eprint=

    2025

  20. [20]

    2025 , eprint=

    Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning , author=. 2025 , eprint=

    2025

  21. [21]

    2025 , eprint=

    ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning , author=. 2025 , eprint=

    2025

  22. [22]

    2024 , eprint=

    LLaVA-CoT: Let Vision Language Models Reason Step-by-Step , author=. 2024 , eprint=

    2024

  23. [23]

    2024 , eprint=

    OpenAI o1 System Card , author=. 2024 , eprint=

    2024

  24. [24]

    2024 , eprint=

    Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models , author=. 2024 , eprint=

    2024

  25. [25]

    2025 , eprint=

    Qwen2.5-VL Technical Report , author=. 2025 , eprint=

    2025

  26. [26]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  27. [27]

    2021 , eprint=

    Learning Transferable Visual Models From Natural Language Supervision , author=. 2021 , eprint=

    2021

  28. [28]

    Proceedings of the 12th International Conference on Learning Representations (ICLR) , year =

    Training Language Models to Self-Correct via Reinforcement Learning , author =. Proceedings of the 12th International Conference on Learning Representations (ICLR) , year =

  29. [29]

    2025 , eprint=

    SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild , author=. 2025 , eprint=

    2025

  30. [30]

    Proceedings of the 42st International Conference on Machine Learning , series =

    ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding , author=. Proceedings of the 42st International Conference on Machine Learning , series =. 2025 , publisher =

    2025

  31. [31]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Abe, Kenshi and Ariu, Kaito and Sakamoto, Mitsuki and Iwasaki, Atsushi , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  32. [32]

    2024 , eprint=

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. 2024 , eprint=

    2024

  33. [33]

    The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=

    Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning , author=. The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=

  34. [34]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Visual Chain-of-Thought Prompting for Knowledge-Based Visual Reasoning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  35. [35]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  36. [36]

    2024 , eprint=

    Visual CoT: Unleashing Chain-of-Thought Reasoning in Multi-Modal Language Models , author=. 2024 , eprint=

    2024

  37. [37]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Visual programming: Compositional visual reasoning without training , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  38. [38]

    arXiv preprint arXiv:2406.09403 , year=

    Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models , author=. arXiv preprint arXiv:2406.09403 , year=

    Pith/arXiv arXiv

  39. [39]

    Proceedings of IEEE International Conference on Computer Vision (ICCV) , year=

    ViperGPT: Visual Inference via Python Execution for Reasoning , author=. Proceedings of IEEE International Conference on Computer Vision (ICCV) , year=

  40. [40]

    arXiv preprint arXiv:2302.00923 , year=

    Multimodal chain-of-thought reasoning in language models , author=. arXiv preprint arXiv:2302.00923 , year=

    Pith/arXiv arXiv

  41. [41]

    Advances in Neural Information Processing Systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in Neural Information Processing Systems , volume=

  42. [42]

    arXiv preprint arXiv:2312.14925 , volume=

    A survey of reinforcement learning from human feedback , author=. arXiv preprint arXiv:2312.14925 , volume=

    arXiv

  43. [43]

    Journal of Cognitive Neuroscience , volume=

    Reinforcement learning , author=. Journal of Cognitive Neuroscience , volume=

  44. [44]

    arXiv preprint arXiv:2510.20176 , year=

    Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding , author=. arXiv preprint arXiv:2510.20176 , year=

    arXiv

  45. [45]

    arXiv preprint arXiv:2511.03773 , year=

    Scaling Agent Learning via Experience Synthesis , author=. arXiv preprint arXiv:2511.03773 , year=

    arXiv

  46. [46]

    Preference optimization with multi-sample comparisons , author=

  47. [47]

    arXiv preprint arXiv:2505.15074 , year=

    DISCO Balances the Scales: Adaptive Domain-and Difficulty-Aware Reinforcement Learning on Imbalanced Data , author=. arXiv preprint arXiv:2505.15074 , year=

    arXiv

  48. [48]

    arXiv preprint arXiv:2506.06923 , year=

    Boosting LLM Reasoning via Spontaneous Self-Correction , author=. arXiv preprint arXiv:2506.06923 , year=

    Pith/arXiv arXiv

  49. [49]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Autonomous Multimodal Reasoning via Implicit Chain-of-Vision , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  50. [50]

    Advances in Neural Information Processing Systems , volume=

    Simpo: Simple preference optimization with a reference-free reward , author=. Advances in Neural Information Processing Systems , volume=

  51. [51]

    Advances in Neural Information Processing Systems , volume=

    Iterative reasoning preference optimization , author=. Advances in Neural Information Processing Systems , volume=

  52. [52]

    Journal of artificial intelligence research , volume=

    Reinforcement learning: A survey , author=. Journal of artificial intelligence research , volume=

  53. [53]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  54. [54]

    arXiv preprint arXiv:2403.13787 , year=

    Rewardbench: Evaluating reward models for language modeling , author=. arXiv preprint arXiv:2403.13787 , year=

    Pith/arXiv arXiv

  55. [55]

    2022 , eprint=

    Training language models to follow instructions with human feedback , author=. 2022 , eprint=

    2022

  56. [56]

    arXiv preprint arXiv:2406.09411 , year=

    MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding , author=. arXiv preprint arXiv:2406.09411 , year=

    Pith/arXiv arXiv

  57. [57]

    arXiv preprint arXiv:2404.19205 , year=

    TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domains , author=. arXiv preprint arXiv:2404.19205 , year=

    Pith/arXiv arXiv

  58. [58]

    arXiv preprint arXiv:1508.00305 , year=

    Compositional semantic parsing on semi-structured tables , author=. arXiv preprint arXiv:1508.00305 , year=

    Pith/arXiv arXiv

  59. [59]

    arXiv preprint arXiv:2212.09662 , year=

    Matcha: Enhancing visual language pretraining with math reasoning and chart derendering , author=. arXiv preprint arXiv:2212.09662 , year=

    Pith/arXiv arXiv

  60. [60]

    2023 , booktitle=

    DePlot: One-shot visual language reasoning by plot-to-table translation , author=. 2023 , booktitle=

    2023

  61. [61]

    arXiv preprint arXiv:2203.10244 , year=

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning , author=. arXiv preprint arXiv:2203.10244 , year=

    Pith/arXiv arXiv

  62. [62]

    arXiv preprint arXiv:1909.02164 , year=

    Tabfact: A large-scale dataset for table-based fact verification , author=. arXiv preprint arXiv:1909.02164 , year=

    Pith/arXiv arXiv 1909

  63. [63]

    2024 , journal =

    HybridFlow: A Flexible and Efficient RLHF Framework , author =. 2024 , journal =

    2024

  64. [64]

    2025 , eprint=

    Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? , author=. 2025 , eprint=

    2025

  65. [65]

    2025 , eprint=

    Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models , author=. 2025 , eprint=

    2025

  66. [66]

    2024 , eprint=

    Large Language Monkeys: Scaling Inference Compute with Repeated Sampling , author=. 2024 , eprint=

    2024

  67. [67]

    2025 , eprint=

    Scaling Test-Time Compute Without Verification or RL is Suboptimal , author=. 2025 , eprint=

    2025

  68. [68]

    2023 , eprint=

    Self-Refine: Iterative Refinement with Self-Feedback , author=. 2023 , eprint=

    2023

  69. [69]

    2025 , eprint=

    Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs , author=. 2025 , eprint=

    2025

  70. [70]

    2024 , eprint=

    Recursive Introspection: Teaching Language Model Agents How to Self-Improve , author=. 2024 , eprint=

    2024

  71. [71]

    2024 , eprint=

    Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models , author=. 2024 , eprint=

    2024

  72. [72]

    2024 , eprint=

    Think before you speak: Training Language Models With Pause Tokens , author=. 2024 , eprint=

    2024

  73. [73]

    2023 , eprint=

    Let's Verify Step by Step , author=. 2023 , eprint=

    2023

  74. [74]

    2024 , eprint=

    MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts , author=. 2024 , eprint=

    2024

  75. [75]

    2023 , eprint=

    G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model , author=. 2023 , eprint=

    2023

  76. [76]

    2025 , eprint=

    LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model , author=. 2025 , eprint=

    2025

  77. [77]

    2025 , eprint=

    Can Large Reasoning Models Self-Train? , author=. 2025 , eprint=

    2025

  78. [78]

    2023 , eprint=

    Large Language Models are Better Reasoners with Self-Verification , author=. 2023 , eprint=

    2023

  79. [79]

    Large Language Models Can Self-Improve

    Huang, Jiaxin and Gu, Shixiang and Hou, Le and Wu, Yuexin and Wang, Xuezhi and Yu, Hongkun and Han, Jiawei. Large Language Models Can Self-Improve. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.67

    doi:10.18653/v1/2023.emnlp-main.67 2023

  80. [80]

    , title =

    Zelikman, Eric and Wu, Yuhuai and Mu, Jesse and Goodman, Noah D. , title =. Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =. 2022 , isbn =

    2022

Showing first 80 references.