Vero: An Open RL Recipe for General Visual Reasoning

Danqi Chen; Gabriel Sarch; Haoyang Wu; Linrong Cai; Qunzhong Wang; Zhuang Liu

REVIEW 2 major objections 5 minor 4 cited by

A single-stage open RL recipe on a balanced six-category visual dataset produces broad gains across charts, STEM, spatial tasks, and open-ended chat without teacher distillation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 09:36 UTC pith:VRNYWBAP

load-bearing objection Solid open multi-domain RL recipe for VLMs: public 600K mixture, task-routed rewards, and clean evidence that joint training beats single-category RL on both scores and CoT style. the 2 major comments →

arxiv 2604.04917 v3 pith:VRNYWBAP submitted 2026-04-06 cs.CV cs.AIcs.CL

Vero: An Open RL Recipe for General Visual Reasoning

Gabriel Sarch , Linrong Cai , Qunzhong Wang , Haoyang Wu , Danqi Chen , Zhuang Liu This is my paper

classification cs.CV cs.AIcs.CL

keywords visual reasoningvision-language modelsreinforcement learningmulti-task RLopen training recipestask-routed rewardschain-of-thoughtdataset mixture

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks what it actually takes to train a vision-language model that reasons well across many kinds of visual work—charts and OCR, science diagrams, spatial and embodied tasks, recognition, grounding and counting, and open-ended captioning or instruction following—when the strongest commercial systems keep their data and RL pipelines closed. It claims that one carefully designed stage of reinforcement learning is enough: build a 600K-sample mixture drawn evenly from 59 datasets across those six categories, pair it with reward functions that route each answer type to an appropriate verifier, and train with sequence-level policy optimization. Under controlled comparisons this mixture beats prior open RL datasets, and applying the recipe to five different starting models yields average gains of roughly three to five points on a 30-benchmark suite. Ablations further claim that single-category training produces weak or negative transfer and elicits distinct chain-of-thought styles, so joint multi-category exposure is necessary for general capability. The authors release the data, code, and models so others can study and extend the recipe.

Core claim

Broad visual reasoning can be obtained from a fully open, single-stage RL recipe: a uniformly mixed 600K-sample dataset spanning six task categories plus task-routed rewards that handle heterogeneous answer formats. Trained this way, models gain 2.9–5.4 average points over their starting checkpoints on a 30-benchmark suite, and an instruct-model variant can surpass a thinking-model baseline without any distillation warm start. Different categories elicit distinct reasoning patterns; learning them jointly, rather than in isolation, is what produces the broad gains.

What carries the argument

Vero-600K with task-routed rewards: a 600K-sample training mixture assembled from 59 datasets, allocated uniformly across six capability categories, and scored by type-specific verifiers (exact match, numeric, IoU, web-action fields, instruction checks, and LLM-as-judge) inside a GSPO-style RL loop.

Load-bearing premise

That the authors’ six-category taxonomy and 30-benchmark suite, scored under their decoding settings and chosen judges, are a fair stand-in for general visual reasoning outside that suite.

What would settle it

Train the same recipe on a held-out model family, then evaluate on a large set of visual-reasoning benchmarks that were never used in VeroEval construction or filtering; if the multi-category mixture no longer beats strong single-domain or distillation baselines on those external suites, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid open multi-domain RL recipe for VLMs: public 600K mixture, task-routed rewards, and clean evidence that joint training beats single-category RL on both scores and CoT style.

read the letter

The thing worth knowing is that this is a careful open systems paper, not a theory paper. They ship Vero-600K (59 datasets, six categories, uniform mix), a task-routed reward stack that actually handles boxes, actions, IF constraints, and open-ended judge scores, and multi-seed controlled RL runs showing their mixture beats prior open RL sets under the same GSPO setup. Gains of +2.9 to +5.4 overall on five post-trained bases, and Instruct-trained Vero beating Qwen3-VL-Thinking without distillation, are the headline numbers. The ablations that matter are the single-category runs: negative transfer is real, especially for captioning and grounding, and the CoT probes (behavior rates + skill probe at 0.77 accuracy) show the categories pull out different reasoning styles rather than one generic "think longer" mode.

What is new is the breadth plus the joint-training necessity claim, not GRPO/GSPO itself. Prior open work (OpenMMReasoner, VL-Rethinker, Perception-R1, OneThinker) was narrower; they do the controlled comparison and the behavior analysis that those papers mostly skipped. Reproducibility intent is high: data, code, models, filtering prompts, judge prompt with anti-hacking rules. Math and citation pattern look fine for this genre—standard GSPO with sequence-level ratios, asymmetric clip, soft overlong penalty; citations cover the right baselines.

Soft spots are the usual ones and not load-bearing against the claim as stated. VeroEval is their suite; judges are Qwen-family; no video or multi-turn; free knobs (alpha, clip eps, mixture alpha) exist but are ablated enough that the main story does not hang on one of them. Taxonomy is pragmatic, not proven minimal. That is a genre limit, not a collapse.

This is for people doing multimodal post-training who want a reproducible multi-domain RL baseline and evidence that mixture design is the lever. I would bring it to reading group, cite the data/recipe and the transfer/behavior results, and send it to peer review. Engage.

Referee Report

2 major / 5 minor

Summary. The paper presents Vero, a fully open single-stage RL recipe for general visual reasoning. It constructs Vero-600K (600K samples from 59 datasets across six task categories: Chart & OCR, STEM, Spatial & Action, Knowledge & Recognition, Grounding/Counting/Search, and Captioning & IF), pairs it with task-routed accuracy rewards plus format and overlong terms under a GSPO-style objective, and evaluates on VeroEval (30 benchmarks). Under controlled RL settings, Vero-600K outperforms prior open RL datasets (ViRL-39k, OpenMMReasoner-74k, LLaVA-OV-1.5-RL-70k). Applied to five post-trained bases, Vero yields +2.9 to +5.4 overall points; Vero-Qwen3I-8B beats Qwen3-VL-8B-Thinking by +3.8 without distillation. Ablations show uniform category mixing beats difficulty/area/length weighting, multi-route rewards beat math_verify, GSPO is preferred, and single-category training produces negative transfer and distinct CoT behaviors that joint training mitigates. All data, code, and models are released.

Significance. If the results hold, this is a high-value open systems contribution: it shows that breadth of carefully filtered multi-category RL data plus task-routed rewards, not distillation warm-starts or multi-stage curricula, is sufficient for broad visual-reasoning gains on open VLMs. Strengths include multi-seed controlled dataset comparisons (Fig. 6), multi-base application (Table 3), SFT-vs-RL and reward/algorithm ablations (Table 4), and behavioral analyses linking categories to distinct CoT profiles and skills (Figs. 7, 11–13). Full public release of Vero-600K/1.6M, VeroEval, code, and models makes the recipe reproducible and extensible—rare for multimodal RL work of this scope.

major comments (2)

The central claim of “general visual reasoning” rests on VeroEval and the six-category taxonomy (Section 3.1–3.2, Table A2). The suite is curated by the authors with difficulty/quality/diversity criteria and evaluated under family-specific decoding and Qwen3-32B / Qwen3-VL-32B judges (Section 4, Appendix C). Gains are consistent within this suite, but the manuscript does not report results on held-out external suites (e.g., standard MMMU full, SEED-Bench, or non-overlapping chart/GUI sets) under independent protocols. A short external-suite table or explicit limitation on suite dependence would strengthen the generality claim without changing the within-suite evidence.
Section 5 / Figure 7 and Section 7 argue that joint multi-category training is necessary because single-category runs yield negative transfer and distinct CoT/skill profiles. The single-category runs use 100k samples (1 epoch) while the full mixture uses 600k; the compute-controlled 100k mix is positive but smaller. The paper should more clearly separate “joint training is necessary for broad gains” from “more data and longer training help,” e.g., by reporting a matched-step single-category vs mix comparison at the full 600k budget or stating the compute-control caveat more prominently in the abstract/conclusion.

minor comments (5)

Table 3 mixes official report numbers and author-evaluated † scores; a short note on which baselines used identical decoding/judge settings would improve fairness of head-to-head comparisons.
Figure 1 and the bottom-row category bars are dense; axis labels and which model is “Vero Initial” vs final could be clearer for readers skimming.
Appendix A.3–A.4 filtering rules and the LLM-judge prompt (Listing B.3) are thorough; a one-paragraph main-text summary of residual failure modes (e.g., remaining ambiguous Knowledge items) would help practitioners.
Limitations (Section 8) correctly note no video/multi-turn and small model scale (7–9B); a sentence on whether the recipe is expected to transfer to larger bases would be useful.
Typos/notation: occasional “Qwen35” vs “Qwen3.5”; ensure consistent naming of Vero-Qwen3I-8B / Vero-Qwen3T-8B across Table 3 and figures.

Circularity Check

0 steps flagged

No significant circularity: empirical multi-task RL gains measured on external benchmarks, not derived by construction from fitted inputs or self-definition.

full rationale

Vero is systems/empirical work. The load-bearing claims (Vero-600K beats prior open RL sets under controlled training from the same base; +2.9–5.4 overall on five post-trained models; joint six-category training needed because single-category runs show negative transfer and distinct CoT profiles) are established by training runs, held-out VeroEval scores, and ablations (Figs. 1, 6–7, 11–13; Tables 1–4), not by algebraic identity or by renaming a fit as a prediction. Rewards are outcome-based against ground-truth answers or constrained judges (Eqs. 1–4, task-routed verifiers in Sec. 3.3); success is not redefined as the training objective. Use of Qwen-family models for question/answer filtering and LLM-as-judge is standard tooling, not a self-citation uniqueness chain that forces the result. No uniqueness theorem, ansatz smuggled via self-citation, or self-definitional step appears in the derivation. The paper is self-contained against external benchmarks under fixed protocols; circularity score is therefore 0.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 4 invented entities

As an empirical ML systems paper, load-bearing structure is not a formal derivation but a stack of modeling and evaluation choices: the six-category taxonomy, uniform mixture, GSPO hyperparameters, reward routing (including LLM-as-judge), and the VeroEval selection criteria. Free parameters are training/eval hyperparameters and mixture design knobs. Invented entities are the constructed dataset, eval suite, and taxonomy used as the operational definition of general visual reasoning.

free parameters (6)

format_reward_weight_alpha = 0.2
Accuracy/format mix uses α=0.2 in the total reward; chosen by authors, not derived.
GSPO_clip_epsilons = 0.0003 / 0.0004
Asymmetric clip bounds ε_low=0.0003 and ε_high=0.0004 control policy updates.
learning_rate_and_batch = 1e-6; batch 256; G=8
LR 1e-6, batch 256, G=8 rollouts, 2000–2343 steps are hand-set training knobs that affect reported gains.
mixture_weighting_exponent_alpha = ratio max/min = 1.6
Non-uniform mixture ablations tune α so max/min domain ratio equals 1.6; uniform wins, but the comparison depends on this design choice.
overlong_penalty_buffer = B=2048, λ=1.0
Soft length penalty buffer B=2048 and λ=1.0 shape response length and can affect open-ended quality.
LLM_judge_score_normalization = (s-1)/9; w=0.5
Open-ended reward maps 1–10 judge scores to [0,1] via (s-1)/9 and blends IF and judge with w=0.5 when both apply.

axioms (5)

domain assumption Outcome-based RL (GSPO/GRPO-style) with verifiable or judge rewards improves multimodal reasoning without requiring teacher CoT distillation.
Core training premise throughout Sections 1–4; supported by prior LLM/VLM RL literature but not proved here.
ad hoc to paper The six task categories partition the skills needed for general visual reasoning and are the right axes for balanced multi-task RL.
Taxonomy is author-defined (Section 3.1), motivated by transfer failures and prior eval conventions, then validated post hoc by ablations.
domain assumption LLM-as-judge scores (with anti-hacking guidelines) are sufficiently aligned rewards for captioning and instruction following.
Used for open-ended training and several evals (Section 3.3, 6; Appendix B.3); residual reward-hacking risk is acknowledged and mitigated by prompt rules.
domain assumption Filtering with a large VLM/LLM for ambiguity, verifiability, and answer canonicity improves RL signal more than it biases the distribution.
Section 3.2 and Table 1 show mixed per-category effects but overall adoption of both question and answer filters.
standard math Standard sequence-level policy optimization math (importance ratios, group-normalized advantages, clipping) is a valid training objective for VLMs.
Equations (1)–(2) and Appendix B.2 import GRPO/GSPO machinery as given.

invented entities (4)

Vero-600K / Vero-1.6M multi-task RL mixture independent evidence
purpose: Provide balanced open training data across six visual reasoning categories.
Constructed artifact from 59 datasets with author filtering and uniform category allocation; central to all claims.
VeroEval 30-benchmark suite independent evidence
purpose: Operationalize evaluation of broad visual reasoning along the same six categories.
Author-curated composite; gains are primarily reported on this suite, so independence from training design is partial though individual benchmarks preexist.
Task-routed multi-verifier reward system independent evidence
purpose: Score heterogeneous answer formats (numeric, MC, boxes, actions, open text) under one RL loop.
Composition of existing verifiers plus judge; the routing design is paper-specific and load-bearing for multi-task claims.
Six-category visual-reasoning taxonomy for RL no independent evidence
purpose: Define mixture balance and interpret transfer/behavior analyses.
Conceptual scaffold introduced for training; not a physical entity, but an author-postulated partition that structures the recipe.

pith-pipeline@v1.1.0-grok45 · 52099 in / 4021 out tokens · 36518 ms · 2026-07-13T09:36:18.128899+00:00 · methodology

0 comments

read the original abstract

What does it take to build a visual reasoner that works across charts, science, spatial understanding, and open-ended tasks? The strongest vision-language models (VLMs) suggest that broad visual reasoning is within reach, yet their closed data and reinforcement learning (RL) pipelines make their gains difficult to study, reproduce, or extend. We introduce Vero, a family of fully open VLMs that match or exceed existing open-weight models across diverse visual reasoning tasks. We scale RL data and rewards across six broad task categories, constructing Vero-600K, a 600K-sample dataset from 59 datasets, and designing task-routed rewards that handle heterogeneous answers. Across VeroEval, our 30-benchmark suite, Vero-600K outperforms existing RL datasets under controlled comparisons. Applied to five starting models, Vero variants gain 2.9-5.4 points on average over their initial models. Notably, Vero-Qwen3I-8B, trained on the Instruct model, surpasses Qwen3-VL-8B-Thinking by 3.8 points on average without additional distillation. Systematic ablations reveal that different task categories elicit distinct reasoning patterns and that broad gains depend on learning them jointly rather than in isolation. All data, code, and models are publicly available.

Figures

Figures reproduced from arXiv: 2604.04917 by Danqi Chen, Gabriel Sarch, Haoyang Wu, Linrong Cai, Qunzhong Wang, Zhuang Liu.

**Figure 1.** Figure 1: Vero achieves state-of-the-art performance across six task categories using a fully open RL recipe. Top left: Training curves versus RL FLOPs for Vero-600K compared with existing open RL datasets, all finetuned from Qwen2.5- VL-7B-Instruct; dashed lines indicate training beyond one epoch. Top center: Summary of the broad task categories targeted in Vero-600K and VeroEval. Top right: Vero compared to Qwen3-… view at source ↗

**Figure 2.** Figure 2: Composition of Vero-600K. The inner ring shows six task categories, each allocated 100K samples (600K total), and the outer ring shows their 59 constituent datasets. The categories represent real-world use cases and cover distinct visual reasoning capabilities (Sections 5 and 7). Categories are uniformly sampled to balance learning across tasks. 1 Introduction Vision-language models (VLMs) are increasingly… view at source ↗

**Figure 3.** Figure 3: Vero-600K data curation pipeline. Starting from over 250 candidate datasets, we assign each to one of six task categories and apply multi-stage selection and filtering: heuristic screening (size, resolution, answer format), manual quality control, LLM-based question filtering for ambiguity and verifiability, and answer filtering for stable reward computation. The retained data are combined into a uniformly… view at source ↗

**Figure 4.** Figure 4: Examples from each task category illustrate the breadth of [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

**Figure 5.** Figure 5: Accuracy verifiers for our multi-task reward. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

**Figure 6.** Figure 6: Per-category RL training curves. Evaluation score vs. RL FLOPs for Vero-600K and three prior open RL datasets, all training Qwen2.5-VL-7B-Instruct. Dashed segments indicate training beyond one epoch. Vero-600K leads on five of six categories throughout training; on STEM it remains within 2 points of the best prior dataset. 4.1 Evaluation Results on VeroEval We report results in [PITH_FULL_IMAGE:figures/fu… view at source ↗

**Figure 7.** Figure 7: Diverse task mixing eliminates negative cross-task transfer. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗

**Figure 8.** Figure 8: RL on different task categories leads to varying [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗

**Figure 10.** Figure 10: Performance improves with greater training data exposure. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗

**Figure 11.** Figure 11: Single-task training elicits a range of behavioral profiles, many of which are distinct. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗

**Figure 12.** Figure 12: Task category skills are largely distinct. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗

**Figure 13.** Figure 13: Single-task training elicits distinct behavioral skills. [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients
cs.CL 2026-06 unverdicted novelty 7.0

ZPPO improves distillation to small vision-language models by using binary and negative candidate prompts plus a replay buffer for hard questions, outperforming standard distillation and GRPO on a 31-benchmark suite w...
Beacon: Knowing When and How to Perform Agentic Visual Reasoning
cs.CV 2026-07 conditional novelty 6.0

Beacon improves agentic visual reasoning by teaching models when tools are necessary and how to use them for net gains, via necessity-aware rewards and hint-guided RL.
MIRROR: Learning from the Other View for Multi-Modal Reasoning
cs.AI 2026-07 conditional novelty 6.0

An RL method that selects the best-performing view of each geometry problem as an internal teacher and distills it into weaker views improves VLM reasoning accuracy and consistency.
Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoning
cs.CV 2026-07 conditional novelty 6.0

RLVR training on 64,000 procedurally generated Trace instances improves Qwen2.5-VL macro-average on 24 external visual reasoning benchmarks by 3.51 points at 3B and 4.06 points at 7B.

Reference graph

Works this paper leans on

4 extracted references · cited by 4 Pith papers

[1]

Spatial & Action receives the largest share (0.273) due to its low initial accuracy, while STEM receives the smallest (0.170)

Difficulty-weighted(rd ∝( 1 −acc d)α, α= 0.475): Up-weights domains where the model performs poorly after the profiling run. Spatial & Action receives the largest share (0.273) due to its low initial accuracy, while STEM receives the smallest (0.170)
[2]

Chart & OCR and Spatial & Action receive the largest shares ( ∼0.23 each), while Knowledge & Recognition receives the smallest (0.148)

Reasoning-length-weighted( rd ∝L α d, α= 0.144): Up-weights domains whose responses require longer chains of thought. Chart & OCR and Spatial & Action receive the largest shares ( ∼0.23 each), while Knowledge & Recognition receives the smallest (0.148). We also evaluate the inverse scheme (rd ∝L −α d ), which favors domains with shorter reasoning traces
[3]

Grounding, Counting & Search receives the largest share (0.244), while Spatial & Action receives the smallest (0.153)

Image-area-weighted( rd ∝A α d, α= 0.443): Up-weights domains with larger input images. Grounding, Counting & Search receives the largest share (0.244), while Spatial & Action receives the smallest (0.153)
[4]

REASONING

Without Knowledge & Recognition: Sets rd = 0 for Knowledge & Recognition and distributes weight equally among the remaining four domains (rd = 0.25 each). This ablation tests whether the lowest-gain category can be dropped without harming overall performance. 32 B Training Details B.1 System Prompt We provide the system prompt forVeroduring training and e...

2025

[1] [1]

Spatial & Action receives the largest share (0.273) due to its low initial accuracy, while STEM receives the smallest (0.170)

Difficulty-weighted(rd ∝( 1 −acc d)α, α= 0.475): Up-weights domains where the model performs poorly after the profiling run. Spatial & Action receives the largest share (0.273) due to its low initial accuracy, while STEM receives the smallest (0.170)

[2] [2]

Chart & OCR and Spatial & Action receive the largest shares ( ∼0.23 each), while Knowledge & Recognition receives the smallest (0.148)

Reasoning-length-weighted( rd ∝L α d, α= 0.144): Up-weights domains whose responses require longer chains of thought. Chart & OCR and Spatial & Action receive the largest shares ( ∼0.23 each), while Knowledge & Recognition receives the smallest (0.148). We also evaluate the inverse scheme (rd ∝L −α d ), which favors domains with shorter reasoning traces

[3] [3]

Grounding, Counting & Search receives the largest share (0.244), while Spatial & Action receives the smallest (0.153)

Image-area-weighted( rd ∝A α d, α= 0.443): Up-weights domains with larger input images. Grounding, Counting & Search receives the largest share (0.244), while Spatial & Action receives the smallest (0.153)

[4] [4]

REASONING

Without Knowledge & Recognition: Sets rd = 0 for Knowledge & Recognition and distributes weight equally among the remaining four domains (rd = 0.25 each). This ablation tests whether the lowest-gain category can be dropped without harming overall performance. 32 B Training Details B.1 System Prompt We provide the system prompt forVeroduring training and e...

2025