{"id":"9cfbbffc-1045-4f5a-9783-8c096582aca3","arxiv_id":"2603.14382","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"StAR, a low-rank RL fine-tuning recipe with mask-aware rewards, selective rollout expansion, and label-before-box answers, lifts reasoning-segmentation gIoU by roughly 7 points and adds a new four-type benchmark, ReasonSeg-X.","lead":"StAR retrofits each piece of a reinforcement-learning training pipeline for reasoning segmentation and reports large gains over prior baselines on both its own new benchmark and existing ones. The paper also adapts majority voting to pixel-level masks, sampling many candidate segmentations and keeping the most consistent ones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ReasonSeg-X/R gap (+7.0/+4.9 gIoU) rests on a self-constructed benchmark where hyperparameters are selected on the test set and baselines are evaluated without released artifacts or variance.","rationale":"The reader's weakest assumption correctly identifies the central epistemic risk: the headline numbers are measured on a benchmark the authors built and evaluated themselves, with test-set-driven hyperparameter selection, author-configured baseline protocols, and no variance reporting. This is exactly the load-bearing concern I find. StAR is nonetheless a plausible engineering contribution: the ablations are systematic, the improvements extend to external benchmarks, and the REST/voting designs are clearly described. The concern is not that the method is invalid, but that the quantitative strength of the central claim—especially '+7.0 gIoU' and 'matches a 72B agent'—is not yet independently verifiable. Because the reader already conditioned the verdict on this issue, my stress-test does not move the verdict; it reinforces CONDITIONAL until the authors release artifacts and validation-selection evidence.","tokens_in":30205,"tokens_out":7026,"duration_ms":80870,"concrete_test":"Release ReasonSeg-X/R with original splits and all baseline evaluation scripts, then re-run the complete StAR pipeline with all hyperparameters (LoRA rank, REST n/m, voting thresholds, answer-order prompt) selected on the validation split or a fresh held-out split, not on the test set. Run StAR-7B and VisionReasoner with 3 training seeds and 5 sampling seeds, and also run VisionReasoner under StAR's stage-2 prompt template and StAR under VisionReasoner's prompt template, reporting mean±std. If the test-selected configuration changes, if the gap falls below ~5 gIoU, or if the prompt swap closes more than ~2 gIoU of the baseline gap, the headline claim should be revised downward.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires the +7.0/+4.9 gIoU improvement over VisionReasoner to be a genuine causal effect of StAR's four design changes. The weakest load-bearing link is the measurement itself. The authors construct ReasonSeg-X/R, run all baselines themselves, and do not release data, code, or evaluation scripts with a hash. Multiple design choices are made directly on the ReasonSeg-X test set: Tab. 3 left selects REST rollout count n=256 by comparing n=16/64/128/256 on RS-X test; Tab. S.5 selects LoRA rank 64 on RS-X test; Tab. 3 right and Supp. G.5 similarly use test-set results for answer-order and voting-threshold decisions. With only 773 test samples and single runs/no error bars, even a small optimistic-selection bias (1–2 gIoU) is plausible. The baseline evaluations are also author-configured: Supp. A grants SAM 3 Agent a retry protocol and excludes Qwen3-VL runs that produced 'substantially more failures,' directly affecting the 'comparable to a 72B agent' claim. Additionally, ReasonSeg-X queries and prompts were designed with StAR's label-first, stepwise-verification answer format in mind, so the benchmark may be implicitly shaped toward StAR's output schema. This does not mean the results are fabricated; it means the headline gap is not yet independently or statistically secured. External benchmarks (ReasonSeg, MMR, MUSE, RES) give partial support that StAR improves beyond the self-built benchmark, but those are also single-run self-evaluations and do not reproduce the exact +7.0 headline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes StAR, a staged RLVR framework for reasoning segmentation that retrofits four components of prior pipelines: LoRA with high rank/large learning rate, a tiered SAM-level mask-IoU reward added to MLLM-level accuracy rewards, rollout-expanded selective tuning (REST), and label-before-coordinate answer format. Starting from VisionReasoner-7B, the authors report gains of +7.0 gIoU on their self-built ReasonSeg-X test set (42.2→49.2) and +4.9 gIoU on ReasonSeg-R (64.8→69.7), with further gains from a mask-clustering majority-voting strategy. They also introduce ReasonSeg-X, a 1,169-sample benchmark with four reasoning types, and ReasonSeg-R, a refined version of the existing ReasonSeg dataset. The paper claims that StAR-7B with majority voting roughly matches a 72B SAM 3 Agent, and that REST surfaces 'dormant' reasoning capabilities.","tokens_in":30618,"tokens_out":3766,"duration_ms":40050,"significance":"If the central empirical claims are reliable, the paper makes a substantive contribution: it provides a systematic component-wise analysis of RLVR for reasoning segmentation, proposes a practical selective-tuning strategy that decouples exploration from policy updates, and adapts test-time scaling to pixel-level prediction. The external-benchmark results on ReasonSeg, MMR, MUSE, and RES give partial support that the gains are not purely an artifact of the self-built benchmark. The efficiency comparisons are also useful, showing that LoRA-based RLVR can be cheaper than full fine-tuning. However, the headline comparison on ReasonSeg-X/R depends on the authors' own annotations, their own baseline runs, and hyperparameters selected on the test set; the paper currently lacks the statistical and procedural safeguards needed to make the central claim quantitatively robust. The originality of REST and the majority-voting scheme is real, but their quantitative quantification needs revision.","major_comments":[{"comment":"Key hyperparameters are selected directly on the ReasonSeg-X test set. Tab. 3 left chooses REST rollout count n=256 by comparing RS-X test gIoU over n=16/64/128/256; Supp. Tab. S.5 selects LoRA rank 64 on RS-X test; Supp. Fig. S.4 tunes τ_IoU, τ_vote, no-target threshold, and sampling temperature T on the same test set. With only 773 test samples and single runs, even a small optimistic-selection bias (1–2 gIoU) can inflate the headline +7.0 gIoU. The authors must either hold out a validation split for parameter selection, report cross-validated selection, or provide multiple-seed error bars and a selection-bias analysis.","section":"§4.3, Tab. 3 left; Supp. Tab. S.5; Supp. Fig. S.4"},{"comment":"The SAM 3 Agent baseline is evaluated with a modified protocol: a single retry is allowed for certain failures, and Qwen3-VL runs are excluded 'due to substantially more failures... and inconsistencies with reported results.' This is an ad-hoc deviation from the official code and the exclusion is asymmetric. The claim that StAR-7B+MV is 'comparable' to SAM 3 Agent-72B (Tab. 1) rests on these choices. Please report failure rates for all baselines, run the official evaluation protocol without retries as a sensitivity check, and release the evaluation harness so the baseline numbers are independently auditable.","section":"Supp. A, 'SAM 3 Agent evaluation'"},{"comment":"ReasonSeg-X is constructed by the authors, and the prompt template used for StAR (Supp. Tab. S.2) includes label-first, stepwise-verification instructions that are also embedded in Stage-2 training and inference. This creates a risk that the benchmark and the prompt template are jointly shaped to favor StAR's output schema. External benchmarks mitigate this concern, but the ReasonSeg-X-specific claim needs a neutrality check. Please report inter-annotator agreement statistics (e.g., Cohen's κ or pairwise IoU agreement) for the 1,169 samples and provide a public annotation protocol and dataset release. Independent re-annotation of a random subset by third-party annotators would be a strong addition.","section":"§2 and Tab. S.2"},{"comment":"The design of REST does not isolate the selective-update mechanism from the large rollout count. The ablation in Tab. 3 left varies n only for REST (updating on m=16 selected rollouts), while Fig. 6 compares 'with REST' vs 'without REST' but does not report the vanilla-GRPO n=256, m=256 condition. If simply increasing rollouts from 16 to 256 (with uniform updates) yields a similar gain, the selective-tuning claim is unsupported. Please add ablations: vanilla GRPO with n=16, vanilla GRPO with n=256 (m=256), REST with n=256/m=16, and REST with n=256/m=256, all under the same compute budget, with at least 2–3 seeds.","section":"§3.2, 'RL strategy'; Fig. 6; Tab. 3 left"}],"minor_comments":[{"comment":"All quantitative results are reported as single runs without error bars or significance tests; please add per-seed summaries for the main tables (Tab. 1 and Tab. 3).","section":"General"},{"comment":"The majority voting strategy has several hyperparameters (τ_IoU=0.85, τ_vote=0.2, no-target=0.5, N=32). The paper claims robustness via Fig. S.4, but the default values are chosen on the test set; clarify which decisions were made on validation versus test.","section":"Supp. G.5"},{"comment":"In Tab. 1, the column headers 'RS-X test' and 'overall' are ambiguous: 'overall' appears to be the aggregate over the four reasoning types on the RS-X test set. Please clarify the grouping and define gIoU/cIoU aggregation for multi-type and multi-target cases.","section":"§4.2 / Tab. 1"},{"comment":"The MUSE evaluation replaces PixelLM's GPT-scored metric with standard IoU. This is reasonable, but the abstract and intro claim 'comparable performance' with prior methods; please explicitly state that the metric change makes direct comparison to published MUSE scores non-apples-to-apples.","section":"Supp. Tab. S.4"},{"comment":"The training time comparison (33h vs 29h) is reported for a single run; please state the hardware configuration for both runs and whether the 2× speedup from LoRA is already included in these numbers.","section":"Supp. A and Fig. 4(a)"},{"comment":"The KL penalty term is written as βD_KL but the text says the KL penalty is disabled for LoRA training. Clarify whether Eq. (3) is the general objective and the KL term is set to zero in the actual runs, or whether the KL is still applied in some experiments.","section":"§3.1, Eq. (3)"},{"comment":"Typo in the section title: 'Finetuing' should be 'Finetuning.'","section":"Supp. G.1, 'LoRA Finetuing'"},{"comment":"The paper says GPT-5 and Gemini 3 are used for 'verification' of annotations, but no quantitative verification results are reported (e.g., agreement rates, number of corrections). Please include a short analysis of the verification outcomes.","section":"§2 / Supp. C"}],"recommendation":"major_revision","confidential_remarks":"The central method (REST plus mask-level voting) is interesting and the external-benchmark evidence suggests the gains are not entirely fabricated. However, the paper's own text and supplementary describe the exact practices that undermine the headline numbers: test-set hyperparameter tuning, ad-hoc baseline protocol changes, and exclusive reliance on a self-built benchmark for the main comparison. These issues are fixable in a revision, so I recommend major revision rather than rejection. I would ask the editor to strongly encourage the authors to release data, code, and evaluation scripts at the earliest stage, and to require at least one round of re-analysis with proper validation/test separation and multiple seeds."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading. The authors do something unusual for this subfield: they run a clean, step-by-step retrofit of an existing RLVR pipeline, ablate each change, and report where they borrowed from prior work (SAM-R1's mask reward, rollout downsampling, DPAD-style label prediction). That honesty earns real credit. The four changes — high-rank/high-LR LoRA, tiered mask-IoU reward, rollout-expanded selective tuning (REST), and label-before-coordinate answers — are individually plausible and the trajectory in Fig. 4a is internally consistent. The mask-level majority voting adaptation is a genuinely new piece, and the construction of ReasonSeg-X/R as a benchmark with explicit reasoning types is a useful contribution, even if the taxonomy boundaries are admittedly blurry. The strongest evidence is not the headline numbers on ReasonSeg-X but the consistent gains over VisionReasoner on external benchmarks: ReasonSeg, MMR, MUSE, and RES. That external support is what keeps the paper's central claim credible.\n\nThe soft spots are real but proportionate. Hyperparameters are selected directly on the ReasonSeg-X test set: Tab. 3 left chooses REST n=256 on RS-X test, Tab. S.5 chooses LoRA rank 64 on test, and answer order and voting thresholds are also tuned on test. With 773 test samples and single runs without error bars, even a small optimistic-selection bias is plausible. The SAM 3 Agent baseline is evaluated with a custom retry protocol and with Qwen3-VL runs excluded; that affects the 'comparable to a 72B agent' claim. And the paper does not release code, data, or evaluation scripts with a hash, so the measurements cannot yet be independently audited. None of this makes me think the results are fabricated; it makes me think the +7.0/+4.9 should be read as 'in this paper's setup' until someone else runs it. The 'dormant reasoning' language is a framing device, not a measured quantity — fine, as long as it is not over-interpreted as a mechanistic claim.\n\nBottom line: this deserves a serious referee. The paper is competent, transparent, and likely influential in the reasoning-segmentation niche. A referee should push for released artifacts and variance estimates; the authors have the ablations in place to respond. I would bring it to a reading group and would cite it for the method combination and the benchmark.","headline":"A solid, honestly-ablated engineering paper whose main uncertainty is measurement, not method: the +7.0/+4.9 gIoU gains on the authors' own benchmark rest on test-set-tuned hyperparameters and author-run baselines, but the external-benchmark gains suggest the core direction is real.","tokens_in":31172,"tokens_out":1496,"would_cite":true,"duration_ms":18554,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"StAR claims a ~7B segmentation model can match a 72B agent by retrofitting every pillar of RLVR and adding mask-level voting.","keywords":["reasoning segmentation","reinforcement learning with verifiable rewards","GRPO","LoRA","test-time scaling","majority voting","multimodal LLM","benchmark construction"],"falsifier":"A neutral third party re-running VisionReasoner, SAM 3 Agent, and other baselines on the released ReasonSeg-X/R with identical prompts, retry rules, and hyperparameters, and checking whether the +7.0 gIoU gap persists; or a re-annotation study that measures whether ReasonSeg-X queries are implicitly biased toward label-first answers (e.g., by checking if answer-format-agnostic models perform anomalously poorly on samples where the query contains a natural label).","tokens_in":29993,"feed_emoji":"🎯","tokens_out":3489,"duration_ms":34920,"temperature":0.7,"pith_summary":"The paper argues that current reinforcement-learning-with-verifiable-rewards pipelines for reasoning segmentation leave most of the base model's visual reasoning capability untapped. It proposes four coordinated design changes—high-rank LoRA with a large learning rate, a fine-grained mask-IoU reward added to MLLM-level accuracy rewards, rollout-expanded selective tuning that samples 256 rollouts and updates on 16 extreme-advantage ones, and a label-before-coordinates answer format—and reports that these changes raise gIoU on its new ReasonSeg-X test set from 42.2 to 49.2 and on ReasonSeg-R from 64.8 to 69.7 over the existing 7B baseline. It also introduces a mask-level majority-voting scheme that lets a 7B model approach the performance of a 72B sequential agent, and constructs a benchmark organized around four reasoning types to expose these gains. If the comparisons are fair, the core truth is that reasoning ability already dormant in the model can be elicited by training-time exploration and a semantic anchor, rather than by scaling parameters.","feed_headline":"A 7B model matches a 72B agent by retrofitting RLVR","feed_subtitle":"Four design changes raise segmentation gIoU by up to 7 points and enable parallel test-time scaling.","key_machinery":"Rollout-Expanded Selective-Tuning (REST): a drop-in GRPO modification that decouples exploration from learning by sampling a large pool (n=128-256) of rollouts per prompt, then updating only on the m/2 highest-advantage and m/2 lowest-advantage trajectories (m=16). This increases exposure to rare successful reasoning paths and sharpens contrastive signal, while keeping wall-clock cost modest because rollout generation is parallel and cheap relative to policy updates. Two other load-bearing pieces are the tiered mask-IoU reward (a six-level function that gives graded feedback directly on mask quality) and label prediction, which inserts a short semantic label before bbox/point coordinates to","core_discovery":"The central claim is that the reasoning bottleneck in current RLVR-based reasoning segmentation is not the base MLLM's capacity but the design of every RLVR component, and that simultaneously fixing parameter tuning, reward granularity, rollout strategy, and answer format 'surfaces dormant reasoning.' Concretely, the paper shows a 7B Qwen2.5-VL model trained on only 5,240 samples with these fixes reaches 49.2 gIoU on ReasonSeg-X test and 69.7 on ReasonSeg-R, and that applying a mask-clustering majority-voting procedure at inference pushes those numbers to 50.3 and 70.7. The same 7B model with voting achieves overall ReasonSeg-X test gIoU comparable to a 72B sequential agent (64.1 vs 64.1 wit","pith_inferences":["The +7.0 gIoU gap likely underestimates the benefit of the combined retrofit if the baseline was not given equivalent hyperparameter tuning or retry allowances; conversely, if baseline evaluations are pessimistic, the headline gain may shrink—so the true effect size is best established by an independent replication.","A natural testable extension is to apply REST and label-prediction to other dense prediction tasks (detection, panoptic segmentation, depth) where semantic-first answers might yield similar anchoring benefits.","The mask-level voting scheme could be combined with token-level selective tuning or negative-gradient masking, which the paper itself suggests as future work; the synergy might further close the gap to much larger models.","The paper's claim that 'dormant reasoning' is surfaced implies that larger base models (e.g., a 32B) benefit disproportionately from REST, which the data partially support but which deserves dedicated scaling-law analysis."],"forward_implications":["If the central claim is correct, reasoning segmentation models can be improved substantially without scaling model size or using large curated reasoning datasets; 5k samples suffice for a 7B model.","Parallel test-time scaling can be extended to pixel-level prediction tasks, offering a cheaper alternative to sequential refinement agents.","The finding that label prediction before coordinates increases visual attention (5.3% to 6.3% attention mass over visual tokens) suggests that simple format changes can mitigate hallucination in multimodal chain-of-thought.","The proposed ReasonSeg-X benchmark with four reasoning types provides a more discriminative evaluation than existing ReasonSeg, and could become a standard diagnostic for reasoning segmentation.","REST's principle of sampling many rollouts but updating on an informative subset is transferable to other RLVR tasks beyond segmentation, such as general visual question answering."],"fun_headline_variants":["7B matches 72B after RLVR overhaul","Four RLVR fixes make 7B competitive with 72B","Test-time scaling helps 7B tie 72B on segmentation","StAR: 5k samples, RLVR fixes, 7B equals 72B"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline gains rest on the fairness of the authors' own baseline evaluations on their newly constructed ReasonSeg-X/R benchmarks, where they run the baselines themselves, tune hyperparameters on the test set, and grant one competitor a modified retry protocol—if those baselines are run pessimistically or the queries/annotations are shaped toward StAR's label-first format, the reported improvements shrink.","fun_headline_variants_meta":{"raw":{"variants":["7B matches 72B after RLVR overhaul","Four RLVR fixes make 7B competitive with 72B","Test-time scaling helps 7B tie 72B on segmentation","StAR: 5k samples, RLVR fixes, 7B equals 72B"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000392,"raw_usage":{"total_tokens":1910,"prompt_tokens":771,"completion_tokens":1139,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":1061}},"tokens_in":515,"tokens_out":1139,"duration_ms":10580,"temperature":1.0,"reasoning_tokens":1061,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T18:10:14.970983+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A neutral third party re-running VisionReasoner, SAM 3 Agent, and other baselines on the released ReasonSeg-X/R with identical prompts, retry rules, and hyperparameters, and checking whether the +7.0 gIoU gap persists; or a re-annotation study that measures whether ReasonSeg-X queries are implicitly biased toward label-first answers (e.g., by checking if answer-format-agnostic models perform anomalously poorly on samples where the query contains a natural label).","supporting_citations":[],"review_version":1}