Pith. sign in

REVIEW 4 major objections 8 minor 3 cited by

StAR claims a ~7B segmentation model can match a 72B agent by retrofitting every pillar of RLVR and adding mask-level voting.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:10 UTC pith:S4I2KRQ7

load-bearing objection A solid, honestly-ablated engineering paper whose main uncertainty is measurement, not method: the +7.0/+4.9 gIoU gains on the authors' own benchmark rest on test-set-tuned hyperparameters and author-run baselines, but the external-benchmark gains suggest the core direction is real. the 4 major comments →

arxiv 2603.14382 v2 pith:S4I2KRQ7 submitted 2026-03-15 cs.CV

StAR: Segment Anything Reasoner

Seokju Yun , Dongheon Lee , Noori Bae , Jaesung Jun , Chanseul Cho , Youngmin Ro This is my paper
classification cs.CV
keywords reasoning segmentationreinforcement learning with verifiable rewardsGRPOLoRAtest-time scalingmajority votingmultimodal LLMbenchmark construction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that current reinforcement-learning-with-verifiable-rewards pipelines for reasoning segmentation leave most of the base model's visual reasoning capability untapped. It proposes four coordinated design changes—high-rank LoRA with a large learning rate, a fine-grained mask-IoU reward added to MLLM-level accuracy rewards, rollout-expanded selective tuning that samples 256 rollouts and updates on 16 extreme-advantage ones, and a label-before-coordinates answer format—and reports that these changes raise gIoU on its new ReasonSeg-X test set from 42.2 to 49.2 and on ReasonSeg-R from 64.8 to 69.7 over the existing 7B baseline. It also introduces a mask-level majority-voting scheme that lets a 7B model approach the performance of a 72B sequential agent, and constructs a benchmark organized around four reasoning types to expose these gains. If the comparisons are fair, the core truth is that reasoning ability already dormant in the model can be elicited by training-time exploration and a semantic anchor, rather than by scaling parameters.

Core claim

The central claim is that the reasoning bottleneck in current RLVR-based reasoning segmentation is not the base MLLM's capacity but the design of every RLVR component, and that simultaneously fixing parameter tuning, reward granularity, rollout strategy, and answer format 'surfaces dormant reasoning.' Concretely, the paper shows a 7B Qwen2.5-VL model trained on only 5,240 samples with these fixes reaches 49.2 gIoU on ReasonSeg-X test and 69.7 on ReasonSeg-R, and that applying a mask-clustering majority-voting procedure at inference pushes those numbers to 50.3 and 70.7. The same 7B model with voting achieves overall ReasonSeg-X test gIoU comparable to a 72B sequential agent (64.1 vs 64.1 wit

What carries the argument

Rollout-Expanded Selective-Tuning (REST): a drop-in GRPO modification that decouples exploration from learning by sampling a large pool (n=128-256) of rollouts per prompt, then updating only on the m/2 highest-advantage and m/2 lowest-advantage trajectories (m=16). This increases exposure to rare successful reasoning paths and sharpens contrastive signal, while keeping wall-clock cost modest because rollout generation is parallel and cheap relative to policy updates. Two other load-bearing pieces are the tiered mask-IoU reward (a six-level function that gives graded feedback directly on mask quality) and label prediction, which inserts a short semantic label before bbox/point coordinates to

Load-bearing premise

The headline gains rest on the fairness of the authors' own baseline evaluations on their newly constructed ReasonSeg-X/R benchmarks, where they run the baselines themselves, tune hyperparameters on the test set, and grant one competitor a modified retry protocol—if those baselines are run pessimistically or the queries/annotations are shaped toward StAR's label-first format, the reported improvements shrink.

What would settle it

A neutral third party re-running VisionReasoner, SAM 3 Agent, and other baselines on the released ReasonSeg-X/R with identical prompts, retry rules, and hyperparameters, and checking whether the +7.0 gIoU gap persists; or a re-annotation study that measures whether ReasonSeg-X queries are implicitly biased toward label-first answers (e.g., by checking if answer-format-agnostic models perform anomalously poorly on samples where the query contains a natural label).

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim is correct, reasoning segmentation models can be improved substantially without scaling model size or using large curated reasoning datasets; 5k samples suffice for a 7B model.
  • Parallel test-time scaling can be extended to pixel-level prediction tasks, offering a cheaper alternative to sequential refinement agents.
  • The finding that label prediction before coordinates increases visual attention (5.3% to 6.3% attention mass over visual tokens) suggests that simple format changes can mitigate hallucination in multimodal chain-of-thought.
  • The proposed ReasonSeg-X benchmark with four reasoning types provides a more discriminative evaluation than existing ReasonSeg, and could become a standard diagnostic for reasoning segmentation.
  • REST's principle of sampling many rollouts but updating on an informative subset is transferable to other RLVR tasks beyond segmentation, such as general visual question answering.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The +7.0 gIoU gap likely underestimates the benefit of the combined retrofit if the baseline was not given equivalent hyperparameter tuning or retry allowances; conversely, if baseline evaluations are pessimistic, the headline gain may shrink—so the true effect size is best established by an independent replication.
  • A natural testable extension is to apply REST and label-prediction to other dense prediction tasks (detection, panoptic segmentation, depth) where semantic-first answers might yield similar anchoring benefits.
  • The mask-level voting scheme could be combined with token-level selective tuning or negative-gradient masking, which the paper itself suggests as future work; the synergy might further close the gap to much larger models.
  • The paper's claim that 'dormant reasoning' is surfaced implies that larger base models (e.g., a 32B) benefit disproportionately from REST, which the data partially support but which deserves dedicated scaling-law analysis.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes StAR, a staged RLVR framework for reasoning segmentation that retrofits four components of prior pipelines: LoRA with high rank/large learning rate, a tiered SAM-level mask-IoU reward added to MLLM-level accuracy rewards, rollout-expanded selective tuning (REST), and label-before-coordinate answer format. Starting from VisionReasoner-7B, the authors report gains of +7.0 gIoU on their self-built ReasonSeg-X test set (42.2→49.2) and +4.9 gIoU on ReasonSeg-R (64.8→69.7), with further gains from a mask-clustering majority-voting strategy. They also introduce ReasonSeg-X, a 1,169-sample benchmark with four reasoning types, and ReasonSeg-R, a refined version of the existing ReasonSeg dataset. The paper claims that StAR-7B with majority voting roughly matches a 72B SAM 3 Agent, and that REST surfaces 'dormant' reasoning capabilities.

Significance. If the central empirical claims are reliable, the paper makes a substantive contribution: it provides a systematic component-wise analysis of RLVR for reasoning segmentation, proposes a practical selective-tuning strategy that decouples exploration from policy updates, and adapts test-time scaling to pixel-level prediction. The external-benchmark results on ReasonSeg, MMR, MUSE, and RES give partial support that the gains are not purely an artifact of the self-built benchmark. The efficiency comparisons are also useful, showing that LoRA-based RLVR can be cheaper than full fine-tuning. However, the headline comparison on ReasonSeg-X/R depends on the authors' own annotations, their own baseline runs, and hyperparameters selected on the test set; the paper currently lacks the statistical and procedural safeguards needed to make the central claim quantitatively robust. The originality of REST and the majority-voting scheme is real, but their quantitative quantification needs revision.

major comments (4)
  1. [§4.3, Tab. 3 left; Supp. Tab. S.5; Supp. Fig. S.4] Key hyperparameters are selected directly on the ReasonSeg-X test set. Tab. 3 left chooses REST rollout count n=256 by comparing RS-X test gIoU over n=16/64/128/256; Supp. Tab. S.5 selects LoRA rank 64 on RS-X test; Supp. Fig. S.4 tunes τ_IoU, τ_vote, no-target threshold, and sampling temperature T on the same test set. With only 773 test samples and single runs, even a small optimistic-selection bias (1–2 gIoU) can inflate the headline +7.0 gIoU. The authors must either hold out a validation split for parameter selection, report cross-validated selection, or provide multiple-seed error bars and a selection-bias analysis.
  2. [Supp. A, 'SAM 3 Agent evaluation'] The SAM 3 Agent baseline is evaluated with a modified protocol: a single retry is allowed for certain failures, and Qwen3-VL runs are excluded 'due to substantially more failures... and inconsistencies with reported results.' This is an ad-hoc deviation from the official code and the exclusion is asymmetric. The claim that StAR-7B+MV is 'comparable' to SAM 3 Agent-72B (Tab. 1) rests on these choices. Please report failure rates for all baselines, run the official evaluation protocol without retries as a sensitivity check, and release the evaluation harness so the baseline numbers are independently auditable.
  3. [§2 and Tab. S.2] ReasonSeg-X is constructed by the authors, and the prompt template used for StAR (Supp. Tab. S.2) includes label-first, stepwise-verification instructions that are also embedded in Stage-2 training and inference. This creates a risk that the benchmark and the prompt template are jointly shaped to favor StAR's output schema. External benchmarks mitigate this concern, but the ReasonSeg-X-specific claim needs a neutrality check. Please report inter-annotator agreement statistics (e.g., Cohen's κ or pairwise IoU agreement) for the 1,169 samples and provide a public annotation protocol and dataset release. Independent re-annotation of a random subset by third-party annotators would be a strong addition.
  4. [§3.2, 'RL strategy'; Fig. 6; Tab. 3 left] The design of REST does not isolate the selective-update mechanism from the large rollout count. The ablation in Tab. 3 left varies n only for REST (updating on m=16 selected rollouts), while Fig. 6 compares 'with REST' vs 'without REST' but does not report the vanilla-GRPO n=256, m=256 condition. If simply increasing rollouts from 16 to 256 (with uniform updates) yields a similar gain, the selective-tuning claim is unsupported. Please add ablations: vanilla GRPO with n=16, vanilla GRPO with n=256 (m=256), REST with n=256/m=16, and REST with n=256/m=256, all under the same compute budget, with at least 2–3 seeds.
minor comments (8)
  1. [General] All quantitative results are reported as single runs without error bars or significance tests; please add per-seed summaries for the main tables (Tab. 1 and Tab. 3).
  2. [Supp. G.5] The majority voting strategy has several hyperparameters (τ_IoU=0.85, τ_vote=0.2, no-target=0.5, N=32). The paper claims robustness via Fig. S.4, but the default values are chosen on the test set; clarify which decisions were made on validation versus test.
  3. [§4.2 / Tab. 1] In Tab. 1, the column headers 'RS-X test' and 'overall' are ambiguous: 'overall' appears to be the aggregate over the four reasoning types on the RS-X test set. Please clarify the grouping and define gIoU/cIoU aggregation for multi-type and multi-target cases.
  4. [Supp. Tab. S.4] The MUSE evaluation replaces PixelLM's GPT-scored metric with standard IoU. This is reasonable, but the abstract and intro claim 'comparable performance' with prior methods; please explicitly state that the metric change makes direct comparison to published MUSE scores non-apples-to-apples.
  5. [Supp. A and Fig. 4(a)] The training time comparison (33h vs 29h) is reported for a single run; please state the hardware configuration for both runs and whether the 2× speedup from LoRA is already included in these numbers.
  6. [§3.1, Eq. (3)] The KL penalty term is written as βD_KL but the text says the KL penalty is disabled for LoRA training. Clarify whether Eq. (3) is the general objective and the KL term is set to zero in the actual runs, or whether the KL is still applied in some experiments.
  7. [Supp. G.1, 'LoRA Finetuing'] Typo in the section title: 'Finetuing' should be 'Finetuning.'
  8. [§2 / Supp. C] The paper says GPT-5 and Gemini 3 are used for 'verification' of annotations, but no quantitative verification results are reported (e.g., agreement rates, number of corrections). Please include a short analysis of the verification outcomes.

Circularity Check

2 steps flagged

Headline ReasonSeg-X/R gains are partly self-measured: key hyperparameters are selected on the ReasonSeg-X test set and then reported as improvements, although external benchmarks provide independent support for the core method.

specific steps
  1. fitted input called prediction [Table 3 left / Section 4.3]
    "We evaluate performance under varying degrees of RESTexploration (the number of rollout generationsn), focusing on the efficacy of theinference-time strategy; the per-step training time for StAR-7B is reported below.Default settings are marked inblue."

    The default n=256 (blue) is chosen by comparing RS-X test gIoU across n=16/64/128/256 (StAR-7B +MV: 49.3/49.5/49.8/50.3). The paper then credits REST with the Stage-2 boost ('boosts performance on ReasonSeg-X/R by 1.5/1.6%') and reports final RS-X test numbers. The reported gain is the selected maximum over the grid, so the 'prediction' on RS-X test is statistically forced by the selection criterion rather than an independent estimate.

  2. fitted input called prediction [Table S.5 / Supp. G.1]
    "Performance comparison between LoRA rank 16 vs. 64 on ReasonSeg-R/X."

    Rank 64 is adopted as the default for all models after observing higher RS-X test gIoU (49.2 vs 46.8 for rank 16). The subsequent RS-X/R headline improvements are therefore not out-of-sample; the LoRA configuration was itself selected on the same test set that is later reported as the result.

full rationale

The core RLVR method (LoRA + REST + mask-IoU reward + label-first format) is not definitionally tied to its evaluation: the components are specified independently of the benchmarks, and gains also appear on external ReasonSeg, MMR, and MUSE, so the central derivation is not an identity. The circularity that exists is confined to the self-built ReasonSeg-X/R evaluation: several design choices (REST rollout count, LoRA rank, voting settings) are selected on the RS-X test split and then presented as measured improvements, which makes those particular numbers statistically forced rather than independent. There is no self-citation chain or uniqueness theorem imported from the authors, and no ansatz smuggled via self-citation. The external benchmark results keep the central claim partially independent, so the overall circularity is moderate rather than total.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 2 invented entities

The central claim is empirical, so the ledger is dominated by tuned hyperparameters (LoRA rank, learning rates, reward tiers, REST counts, voting thresholds) rather than by derivation constants. The key domain assumptions are that GRPO+LoRA training is stable, that SAM-2-mask IoU is the right objective, and that the self-built benchmark is unbiased. The invented entity is a narrative construct ('dormant reasoning') plus a new unreleased benchmark family.

free parameters (8)
  • LoRA rank = 64
    Rank 16→64 chosen by ablation on the ReasonSeg-X/R test sets (Tab. S.5); higher rank argued necessary for complex reasoning.
  • Learning rate = 1e-5 (stage 1), 5e-6 (stage 2)
    Hand-set; KL penalty disabled (Supp. A).
  • Mask reward tiers = IoU buckets 0.3/0.5/0.7/0.8/0.9 -> rewards 1-5
    Hand-designed reward shaping (Eq. 5); the 0.3-0.5 tier is the paper's addition to SAM-R1.
  • bbox/point reward thresholds = bbox IoU>0.5; L1<10/30 px; divide by max(Npred,NGT)
    Carried over from VisionReasoner [30] without re-justification.
  • REST rollout/update counts = n=256, m=16 (n=128 for 32B)
    Selected by sweeping n in {16,64,128,256} on the ReasonSeg-X test set (Tab. 3 left).
  • Majority voting parameters = N=32, tau_IoU=0.85, tau_vote=0.2, no-target 0.5, T=1.0, top-p 0.9
    Hand-set; T and N further tuned on ReasonSeg-X test (Fig. S.4).
  • Stage-2 training epochs = 10
    Hand-set (Supp. A).
  • 'Correct' threshold for REST analysis = IoU > 0.5
    Definition used to classify samples into correct/incorrect in Fig. 6.
axioms (6)
  • domain assumption GRPO (Eqs. 3-4) with LoRA and disabled KL is a stable, effective optimizer for MLLM policy learning on this task.
    Invoked throughout Sections 3.1-3.2; no convergence analysis or learning curves are shown in the main text.
  • domain assumption Frozen SAM 2 converts bbox/point prompts into masks whose IoU with GT measures task success.
    Used in Eqs. 2 and 5 and in all evaluation; Supp. A's SAM 3 Agent tag-omission failures show mask generation is not trivial.
  • ad hoc to paper The four-type reasoning taxonomy (P/F, C/KI, C/R, C/MH) is meaningful and annotation validity is confirmed by GPT-5/Gemini-3 verification.
    Section 2; the paper itself concedes type boundaries are blurred, so the taxonomy is a design choice rather than independent ground truth.
  • domain assumption Higher LoRA rank (64) is necessary for complex reasoning, following HiRA [15].
    Supp. G.1/Tab. S.5; adopted as a prior claim, consistent with the paper's own ablation.
  • domain assumption Test-time sampling distribution (T, top-p, N) is compatible with the large-n training distribution, making majority voting beneficial.
    Supp. G.3/G.5; Fig. S.4 shows gains vary with T and N, so this is an empirical tuning assumption.
  • ad hoc to paper Manual annotation of 1,169 ReasonSeg-X samples is free of systematic bias toward the authors' method.
    Section 2/Supp. C; queries were authored by the same team that built and tuned StAR, so query style may implicitly match the label-first answer format.
invented entities (2)
  • Dormant/latent reasoning capability no independent evidence
    purpose: Explains why REST improves hard samples: the model supposedly possesses reasoning it cannot express before broadened exploration; the contribution is framed as 'surfacing' it.
    Introduced in the abstract and Fig. 6; the supporting evidence (hard-sample gIoU improvement with REST) is correlational and measured on the paper's own benchmark; no independent falsifiable handle.
  • ReasonSeg-X / ReasonSeg-R datasets no independent evidence
    purpose: New evaluation and training resources for reasoning segmentation with four reasoning types and no-target samples.
    Not released with the paper (no dataset link or hash), and the model is trained and hyperparameter-tuned on the same benchmark family it is prominently evaluated on; independent verification is currently impossible.

pith-pipeline@v1.3.0-alltime-deepseek · 29655 in / 18648 out tokens · 170328 ms · 2026-08-02T18:10:14.970983+00:00 · methodology

0 comments
read the original abstract

As AI systems are being integrated more rapidly into diverse and complex real-world environments, the ability to perform holistic reasoning over an implicit query and an image to localize a target is becoming increasingly important. However, recent reasoning segmentation methods fail to sufficiently elicit the visual reasoning capabilities of the base mode. In this work, we present Segment Anything Reasoner (StAR), a comprehensive framework that refines the design space from multiple perspectives-including parameter-tuning scheme, reward functions, learning strategies and answer format-and achieves substantial improvements over recent baselines. In addition, for the first time, we successfully introduce parallel test-time scaling to the segmentation task, pushing the performance boundary even further. To extend the scope and depth of reasoning covered by existing benchmark, we also construct the ReasonSeg-X, which compactly defines reasoning types and includes samples that require deeper reasoning. Leveraging this dataset, we train StAR with a rollout-expanded selective-tuning approach to activate the base model's latent reasoning capabilities, and establish a rigorous benchmark for systematic, fine-grained evaluation of advanced methods. With only 5k training samples, StAR achieves significant gains over its base counterparts across extensive benchmarks, demonstrating that our method effectively brings dormant reasoning competence to the surface.

Figures

Figures reproduced from arXiv: 2603.14382 by Chanseul Cho, Dongheon Lee, Jaesung Jun, Noori Bae, Seokju Yun, Youngmin Ro.

Figure 1
Figure 1. Figure 1: We establish a reasoning segmentation benchmark, ReasonSeg-X, that demands a broad range of reasoning skills. Our dataset addresses goal-oriented reasoning and the ability to flexibly invoke world knowledge, while also covering complex relational reasoning and step-by-step reasoning capabilities. Our model, StAR (equipped with Qwen3-VL 32B [1]), demonstrates remarkable performance across all these aspects … view at source ↗
Figure 2
Figure 2. Figure 2: ReasonSeg-R examples. We correct masks for cases that are mismatched to the query and also refine mask quality. In addition, we modify query expressions that may inadvertently include regions outside the mask (e.g., the video player monitor). To generate our dataset, we curate diverse and context-rich images from OpenImages [22] and, based on the reasoning type design above, meticulously annotate them with… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the StAR framework pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: (a) We progressively retrofit all pillars of the standard RLVR framework (Vi￾sionReasoner) toward our performant and robust model (StAR), without introduc￾ing significant computational overhead. The foreground bars show performance on ReasonSeg-X test, while results on ReasonSeg-R are shown with gray bars. The perfor￾mance values in parentheses are obtained with our majority voting strategy (Sec. 3.3). Gra… view at source ↗
Figure 5
Figure 5. Figure 5: Motivation for the mask IoU reward. Left: Misalignment between the MLLM-level reward function and the task’s final objective potentially induces confusing learning signals. Right: Distinguishing between diverse mask defects via a fine-grained tiered mask reward promotes stable and stepwise optimization. \scalebox {0.6}{$ \text {mask reward} = \begin {cases} \textcolor {red}{5}, & \textcolor {red}{\text {Io… view at source ↗
Figure 6
Figure 6. Figure 6: Left: Rollout-expanded selective-tuning (REST) substantially improves sample utilization. In particular, for hard samples, it develops the model’s intricate reasoning by exploring a vast space of reasoning paths. “Correct” is defined as IoU > 0.5, and results are reported with the Qwen3-VL 8B model. Right: REST effectively boosts performance on problems that require multi-step reasoning; moreover, its gain… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative results on our ReasonSeg-X test set demonstrate that StAR mod￾els, via multi-step reasoning, can effectively tackle complex segmentation scenarios ne￾cessitating various capabilities, such as comparative/relational reasoning, physics-based 3D awareness, comprehensive pattern analysis, and multi-hop algebraic expansion. size to 16 and the learning rate to 1×10−5 (5×10−6 for stage 2). We use LoRA… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Vision Harnessing Agent for Open Ad-hoc Segmentation

    cs.CV 2026-05 unverdicted novelty 7.0

    VASA is a vision-guided agent for open ad-hoc segmentation that creates and validates masks through planning, tool use, and error recovery, outperforming baselines on the new PARS benchmark and RefCOCOm.

  2. B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation

    cs.CV 2026-05 unverdicted novelty 6.0

    B-GRTO extends GRPO by reusing rollouts to optimize auxiliary segmentation decoder objectives, yielding substantial gains over plain GRPO on referring segmentation tasks.

  3. B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation

    cs.CV 2026-05 unverdicted novelty 4.0

    B-GRTO pre-trains a segmentation tool via bootstrapped group relative optimization on GRPO rollouts, yielding substantial gains over plain GRPO on referring segmentation benchmarks.

Reference graph

Works this paper leans on

69 extracted references · 13 linked inside Pith · cited by 2 Pith papers

  1. [1]

    arXiv preprint arXiv:2511.21631 (2025) 2, 3, 12, 23

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, ...

    Pith/arXiv arXiv 2025

  2. [2]

    5-vl technical report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025) 2, 12

    Pith/arXiv arXiv 2025

  3. [3]

    In: ECCV (2024) 2, 12, 21

    Bao, X., Sun, S., Ma, S., Zheng, K., Guo, Y., Zhao, G., Zheng, Y., Wang, X.: Cores: Orchestrating the dance of reasoning and segmentation. In: ECCV (2024) 2, 12, 21

    2024

  4. [4]

    TMLR (2024) 8

    Biderman, D., Ortiz, J.G., Portes, J., Paul, M., Greengard, P., Jennings, C., King, D., Havens, S., Chiley, V., Frankle, J., et al.: Lora learns less and forgets less. TMLR (2024) 8

    2024

  5. [5]

    In: ICLR (2026) 2, 3, 12, 17, 22, 23

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala,K.V.,Khedr,H.,Huang,A.,etal.:Sam3:Segmentanythingwithconcepts. In: ICLR (2026) 2, 3, 12, 17, 22, 23

    2026

  6. [6]

    In: ICLR (2025) 28

    Chow, Y., Tennenholtz, G., Gur, I., Zhuang, V., Dai, B., Kumar, A., Agarwal, R., Thiagarajan, S., Boutilier, C., Faust, A.: Inference-aware fine-tuning for best-of-n sampling in large language models. In: ICLR (2025) 28

    2025

  7. [7]

    In: NeurIPS (2025) 15

    Deng, W., Ren, Y., Li, M., Sutherland, D.J., Li, X., Thrampoulidis, C.: On the effect of negative gradient in group relative deep reinforcement optimization. In: NeurIPS (2025) 15

    2025

  8. [8]

    In: ICLR (2026) 2, 12, 22, 24

    Du, T., Li, H., Fan, Z., Zhang, J., Pan, P., Zhang, Y.: SAM-veteran: An MLLM- based human-like SAM agent for reasoning segmentation. In: ICLR (2026) 2, 12, 22, 24

    2026

  9. [9]

    Gemini Team, Google DeepMind: Gemini 3: A new era of intelligence with gemini

  10. [10]

    Tech. rep. (2025),https://blog.google/products-and-platforms/products/ gemini/gemini-3/, technical Report 5

    2025

  11. [11]

    In: NeurIPS (2025) 22

    Ghosal, S.S., Chakraborty, S., Reddy, A., Lu, Y., Wang, M., Manocha, D., Huang, F., Ghavamzadeh, M., Bedi, A.S.: Does thinking more always help? mirage of test- time scaling in reasoning models. In: NeurIPS (2025) 22

    2025

  12. [12]

    arXiv preprint arXiv:2501.12948 (2025) 2, 21

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025) 2, 21

    Pith/arXiv arXiv 2025

  13. [13]

    In: CVPR (2019) 7, 12

    Gupta, A., Dollar, P., Girshick, R.: Lvis: A dataset for large vocabulary instance segmentation. In: CVPR (2019) 7, 12

    2019

  14. [14]

    In: ICLR (2022) 2, 8, 23, 24

    Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: ICLR (2022) 2, 8, 23, 24

    2022

  15. [15]

    In: NeurIPS (2025) 2, 8, 12, 21, 24, 25 StAR: Segment Anything Reasoner 35

    Huang, J., Xu, Z., Zhou, J., Liu, T., Xiao, Y., Ou, M., Ji, B., Li, X., Yuan, K.: SAM-r1: Leveraging SAM for reward feedback in multimodal segmentation via reinforcement learning. In: NeurIPS (2025) 2, 8, 12, 21, 24, 25 StAR: Segment Anything Reasoner 35

    2025

  16. [16]

    In: ICLR (2025) 8, 24

    Huang, Q., Ko, T., Zhuang, Z., Tang, L., Zhang, Y.: HiRA: Parameter-efficient hadamard high-rank adaptation for large language models. In: ICLR (2025) 8, 24

    2025

  17. [17]

    In: ICLR (2026) 11

    Huang, S., Qu, X., Li, Y., Luo, Y., He, Z., Liu, D., Cheng, Y.: Spotlight on token perception for multimodal reinforcement learning. In: ICLR (2026) 11

    2026

  18. [18]

    In: ICLR (2025) 2, 4, 12, 14, 21

    Jang, D., Cho, Y., Lee, S., Kim, T., Kim, D.: MMR: A large-scale benchmark dataset for multi-target and multi-granularity reasoning segmentation. In: ICLR (2025) 2, 4, 12, 14, 21

    2025

  19. [19]

    In: ICML (2025) 11

    Jung, M., Lee, S., Kim, E., Yoon, S.: Visual attention never fades: Selective pro- gressive attention recalibration for detailed image captioning in multimodal large language models. In: ICML (2025) 11

    2025

  20. [20]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https://openreview.net/forum?id=29FRqmVQK822

    Kang, Z., Zhao, X., Song, D.: Scalable best-of-n selection for large language models via self-certainty. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https://openreview.net/forum?id=29FRqmVQK822

    2025

  21. [21]

    In: ICCV (2023) 2, 21

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: ICCV (2023) 2, 21

    2023

  22. [22]

    In: ICLR (2025) 22

    Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J.D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., Zhang, L.M., McKinney, K., Shrivastava, D., Paduraru, C., Tucker, G., Precup, D., Behbahani, F., Faust, A.: Training language models to self-correct via reinforcement learning. In: ICLR (2025) 22

    2025

  23. [23]

    IJCV (2020) 5

    Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Ka- mali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., Ferrari, V.: The open images dataset v4: Unified image classification, object detection, and visual rela- tionship detection at scale. IJCV (2020) 5

    2020

  24. [24]

    In: CVPR (2024) 1, 2, 3, 4, 12, 13, 21

    Lai, X., Tian, Z., Chen, Y., Li, Y., Yuan, Y., Liu, S., Jia, J.: Lisa: Reasoning segmentation via large language model. In: CVPR (2024) 1, 2, 3, 4, 12, 13, 21

    2024

  25. [25]

    arXiv preprint arXiv:2411.15124 (2024) 2

    Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Mi- randa, L.J.V., Liu, A., Dziri, N., Lyu, S., et al.: Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124 (2024) 2

    Pith/arXiv arXiv 2024

  26. [26]

    In: CVPR (2025) 8

    Liang, J., Huang, W., Wan, G., Yang, Q., Ye, M.: Lorasculpt: Sculpting lora for harmonizing general and specialized knowledge in multimodal large language mod- els. In: CVPR (2025) 8

    2025

  27. [27]

    In: CVPR (2023) 7, 12

    Liu, C., Ding, H., Jiang, X.: Gres: Generalized referring expression segmentation. In: CVPR (2023) 7, 12

    2023

  28. [28]

    Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J.: Llava-next: Improved reasoning, ocr, and world knowledge (2024),https://llava-vl.github.io/blog/ 2024-01-30-llava-next/2

    2024

  29. [29]

    In: NeurIPS (2023) 2

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: NeurIPS (2023) 2

    2023

  30. [30]

    arXiv preprint arXiv:2503.06520 (2025) 2, 5, 21

    Liu, Y., Peng, B., Zhong, Z., Yue, Z., Lu, F., Yu, B., Jia, J.: Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520 (2025) 2, 5, 21

    Pith/arXiv arXiv 2025

  31. [31]

    In: ICLR (2026) 2, 5, 7, 8, 9, 12, 13, 21, 22, 23

    Liu, Y., Qu, T., Zhong, Z., Peng, B., Liu, S., Yu, B., Jia, J.: Visionreasoner: Unified reasoning-integrated visual perception via reinforcement learning. In: ICLR (2026) 2, 5, 7, 8, 9, 12, 13, 21, 22, 23

    2026

  32. [32]

    In: ACL (2025) 2, 12, 22

    Lu, Y., Cao, J., Wu, Y., Li, B., Tang, L., Ji, Y., Wu, C., Wu, J., Zhu, W.: RSVP: Reasoning segmentation via visual prompting and multi-modal chain-of-thought. In: ACL (2025) 2, 12, 22

    2025

  33. [33]

    In: ICLR (2026) 2, 12, 21 36 S

    Lu, Z., Li, L., Wang, J., Feng, Y., Chen, B., Chen, K., Wang, Y.: Coprs: Learning positional prior from chain-of-thought for reasoning segmentation. In: ICLR (2026) 2, 12, 21 36 S. Yun et al

    2026

  34. [34]

    In: NeurIPS (2023) 22

    Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B.P., Hermann, K., Welleck, S., Yazdanbakhsh, A., Clark, P.: Self-refine: Iterative refinement with self-feedback. In: NeurIPS (2023) 22

    2023

  35. [35]

    In: EMNLP (2025) 8

    Muennighoff, N., Yang, Z., Shi, W., Li, X.L., Fei-Fei, L., Hajishirzi, H., Zettle- moyer, L., Liang, P., Candès, E., Hashimoto, T.B.: s1: Simple test-time scaling. In: EMNLP (2025) 8

    2025

  36. [36]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Qian, R., Yin, X., Dou, D.: Reasoning to attend: Try to understand how< seg> token works. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 24722–24731 (2025) 2, 12, 21

    2025

  37. [37]

    In: NeurIPS (2024) 22

    Qu, Y., Zhang, T., Garg, N., Kumar, A.: Recursive introspection: Teaching lan- guage model agents how to self-improve. In: NeurIPS (2024) 22

    2024

  38. [38]

    In: ICML (2021) 21

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021) 21

    2021

  39. [39]

    In: ICLR (2025) 2, 5, 11

    Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K.V., Carion, N., Wu, C.Y., Girshick, R., Dollar, P., Feichtenhofer, C.: SAM 2: Segment anything in images and videos. In: ICLR (2025) 2, 5, 11

    2025

  40. [40]

    In: NeurIPS (2025) 10

    Razin, N., Wang, Z., Strauss, H., Wei, S., Lee, J.D., Arora, S.: What makes a reward model a good teacher? an optimization perspective. In: NeurIPS (2025) 10

    2025

  41. [41]

    In: CVPR (2024) 2, 4, 12, 23

    Ren, Z., Huang, Z., Wei, Y., Zhao, Y., Fu, D., Feng, J., Jin, X.: Pixellm: Pixel reasoning with large multimodal model. In: CVPR (2024) 2, 4, 12, 23

    2024

  42. [42]

    arXiv preprint arXiv:1707.06347 (2017) 6

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017) 6

    Pith/arXiv arXiv 2017

  43. [43]

    In: ICML (2025) 15

    Setlur, A., Rajaraman, N., Levine, S., Kumar, A.: Scaling test-time compute with- out verification or RL is suboptimal. In: ICML (2025) 15

    2025

  44. [44]

    arXiv preprint arXiv:2402.03300 (2024) 2, 6, 21

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024) 2, 6, 21

    Pith/arXiv arXiv 2024

  45. [45]

    arXiv preprint arXiv: 2409.19256 (2024) 17

    Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., Wu, C.: Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256 (2024) 17

    Pith/arXiv arXiv 2024

  46. [46]

    arXiv preprint arXiv:2601.03267 (2025) 5

    Singh, A., Fry, A., Perelman, A., Tart, A., Ganesh, A., El-Kishky, A., McLaughlin, A., Low, A., Ostrow, A., Ananthram, A., et al.: Openai gpt-5 system card. arXiv preprint arXiv:2601.03267 (2025) 5

    Pith/arXiv arXiv 2025

  47. [47]

    In: ICLR (2025) 2, 15, 22

    Snell, C.V., Lee, J., Xu, K., Kumar, A.: Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In: ICLR (2025) 2, 15, 22

    2025

  48. [48]

    In: ICLR (2026) 11, 22

    Tian, X., Zou, S., Yang, Z., He, M., Waschkowski, F., Wesemann, L., Tu, P.H., Zhang, J.: More thought, less accuracy? on the dual nature of reasoning in vision- language models. In: ICLR (2026) 11, 22

    2026

  49. [49]

    In: NAACL (2025) 8

    Wang, H., Li, Y., Wang, S., Chen, G., Chen, Y.: Milora: Harnessing minor singular components for parameter-efficient llm finetuning. In: NAACL (2025) 8

    2025

  50. [50]

    In: ICLR (2026) 22

    Wang, H., Du, C., Kawaguchi, K., Pang, T.: Think in parallel, answer as one: Logit averaging for open-ended reasoning. In: ICLR (2026) 22

    2026

  51. [51]

    In: NeurIPS (2025) 8 StAR: Segment Anything Reasoner 37

    Wang, H., Qu, C., Huang, Z., Chu, W., Lin, F., Chen, W.: VL-rethinker: Incen- tivizing self-reflection of vision-language models with reinforcement learning. In: NeurIPS (2025) 8 StAR: Segment Anything Reasoner 37

    2025

  52. [52]

    In: ACL (2023) 18

    Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R.K.W., Lim, E.P.: Plan-and- solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In: ACL (2023) 18

    2023

  53. [53]

    arXiv preprint arXiv:2409.12191 (2024) 2

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024) 2

    Pith/arXiv arXiv 2024

  54. [54]

    In: ICLR (2026) 8

    Wang, S., Asilis, J., Akgül, Ö.F., Bilgin, E.B., Liu, O., Neiswanger, W.: Tina: Tiny reasoning models via loRA. In: ICLR (2026) 8

    2026

  55. [55]

    In: NeurIPS (2025) 15

    Wang, S., Yu, L., Gao, C., Zheng, C., Liu, S., Lu, R., Dang, K., Chen, X.H., Yang, J., Zhang, Z., Liu, Y., Yang, A., Zhao, A., Yue, Y., Song, S., Yu, B., Huang, G., Lin, J.: Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for LLM reasoning. In: NeurIPS (2025) 15

    2025

  56. [56]

    In: ICLR (2025) 2, 12, 21, 24

    Wang, X., Zhang, S., Li, S., Li, K., Kallidromitis, K., Kato, Y., Kozuka, K., Darrell, T.: SegLLM: Multi-round reasoning segmentation with large language models. In: ICLR (2025) 2, 12, 21, 24

    2025

  57. [57]

    In: ICLR (2023) 2, 11, 22

    Wang, X., Wei, J., Schuurmans, D., Le, Q.V., Chi, E.H., Narang, S., Chowdhery, A., Zhou, D.: Self-consistency improves chain of thought reasoning in language models. In: ICLR (2023) 2, 11, 22

    2023

  58. [58]

    In: NeurIPS (2022) 6, 22

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., brian ichter, Xia, F., Chi, E.H., Le, Q.V., Zhou, D.: Chain of thought prompting elicits reasoning in large language models. In: NeurIPS (2022) 6, 22

    2022

  59. [59]

    In: ICCV (2025) 8

    Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., Sun, L., Yuan, L.: Llava-cot: Let vision language models reason step-by-step. In: ICCV (2025) 8

    2025

  60. [60]

    arXiv preprint arXiv:2504.13818 (2025) 10, 25

    Xu, Y.E., Savani, Y., Fang, F., Kolter, J.Z.: Not all rollouts are useful: Down- sampling rollouts in llm reinforcement learning. arXiv preprint arXiv:2504.13818 (2025) 10, 25

    Pith/arXiv arXiv 2025

  61. [61]

    In: NeurIPS (2025) 11, 22

    Xu, Z., Liu, C., Wei, Q., Wu, J., Zou, J., Wang, X.E., Zhou, Y., Liu, S.: More thinking, less seeing? assessing amplified hallucination in multimodal reasoning models. In: NeurIPS (2025) 11, 22

    2025

  62. [62]

    arXiv preprint arXiv:2312.17240 (2023) 7

    Yang, S., Qu, T., Lai, X., Tian, Z., Peng, B., Liu, S., Jia, J.: Lisa++: An improved baseline for reasoning segmentation with large language model. arXiv preprint arXiv:2312.17240 (2023) 7

    Pith/arXiv arXiv 2023

  63. [63]

    In: CVPR (2026) 21

    Yang, T., Zhou, Q., Li, Y., Wang, Q.: Discriminative perception via anchored description for reasoning segmentation. In: CVPR (2026) 21

    2026

  64. [64]

    In: ECCV (2016) 7, 12, 24

    Yu, L., Poirson, P., Yang, S., Berg, A.C., Berg, T.L.: Modeling context in referring expressions. In: ECCV (2016) 7, 12, 24

    2016

  65. [65]

    arXiv preprint arXiv:2503.14476 (2025) 8

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., et al.: Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476 (2025) 8

    Pith/arXiv arXiv 2025

  66. [66]

    In: CVPR (2025) 8

    Yun, S., Chae, S., Lee, D., Ro, Y.: Soma: Singular value decomposed minor com- ponents adaptation for domain generalizable representation learning. In: CVPR (2025) 8

    2025

  67. [67]

    In: ICLR (2023) 18

    Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q.V., Chi, E.H.: Least-to-most prompting enables complex reasoning in large language models. In: ICLR (2023) 18

    2023

  68. [68]

    In: CVPR (2025) 2, 21

    Zhu, L., Chen, T., Xu, Q., Liu, X., Ji, D., Wu, H., Soh, D.W., Liu, J.: Popen: Preference-based optimization and ensemble for lvlm-based reasoning segmenta- tion. In: CVPR (2025) 2, 21

    2025

  69. [69]

    In: NeurIPS (2025) 26

    Zhu, X., Xia, M., Wei, Z., Chen, W.L., Chen, D., Meng, Y.: The surprising effec- tiveness of negative reinforcement in LLM reasoning. In: NeurIPS (2025) 26

    2025