Pith. sign in

REVIEW 3 major objections 5 minor 61 references

Multimodal LLMs ground objects but not their parts; this paper claims the fix is object-first reasoning plus part-aware reinforcement rewards, and a 4B model trained this way outscores 7B rivals and SAM3 on part benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 23:33 UTC pith:5TLEUUVK

load-bearing objection Honestly useful paper, but the headline overstates what the experiments support: the fair comparison is the plain-prompt baseline in H.2, not the structured-prompt row in Table 1, and the true margins are a few gIoU points. the 3 major comments →

arxiv 2607.15374 v1 pith:5TLEUUVK submitted 2026-07-16 cs.CV

Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning

Kazi Sajeed Mehrab , Hani Alomari , Najibul Haque Sarker , Chia-Wei Tang , Zaber Ibn Abdul Hakim , Anuj Karpatne , Chris Thomas This is my paper
classification cs.CV
keywords visual groundingobject partsmultimodal large language modelsreinforcement learningGRPOhierarchical reasoningpart-aware rewardszero-shot transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that multimodal LLMs fail at part-level visual grounding because they localize a part in the same single step they use for a whole object, with no notion that a part lives inside a parent. Its proposed system, Object-Part Hierarchical Reflective Grounding (OP-HRG), forces the model to first decide whether the query names an object or a part, then localize the parent object, then the part within it, then critique its own box and optionally re-encode the predicted crop before finalizing. Training under a group-relative policy-optimization objective with stage-wise part-aware rewards — object-hint accuracy, part-in-object containment, refinement improvement, adjustment consistency — produces, from a 4-billion-parameter backbone, part-grounding scores above 7B grounding models and above text-promptable SAM3 on PascalPart, PartImageNet, and InstructPart, while roughly preserving whole-object referring. The implication the authors draw: the capability for fine-grained spatial reasoning already exists in pretrained MLLMs and is dormant; structured prompting plus targeted reward signals draw it out, so scale is not the main lever for part grounding.

Core claim

The central claim is that the object-part hierarchy is the missing inductive bias in MLLM grounding. If a query names a part, the right computation is coarse-to-fine: anchor on the parent object, then predict a tight box and interior point for the part, verify by self-reflection (strengthened by re-encoding the predicted crop), and finalize. The paper shows this chain can be taught entirely with verifiable rewards — no part-specific supervised fine-tuning — and that the resulting 4B model reaches 75.56 gIoU on InstructPart, 38.59 on PascalPart, 56.87 on PartImageNet, 87.50 on Pascal objects, and 69.6 on reasoning segmentation, surpassing larger baselines.

What carries the argument

The OP-HRG output protocol: a fixed sequence of tagged reasoning states (<locate>, <target>, <object_hint>, <first_answer>, <criticism>, <answer>) that exposes intermediate decisions so each can be rewarded. It is paired with a composite reward whose parts are base localization rewards (IoU, L1, point, compactness, format, non-repetition), hierarchical rewards active only for part queries (object-hint IoU, part containment), and reflective-refinement rewards (improvement over the stronger of the model's first answer or a precomputed baseline, plus an adjustment-consistency penalty). The active-perception variant re-encodes the predicted crops and injects them as fresh visual tokens before th

Load-bearing premise

The cross-dataset gains rest on 1,200 part-mask training images whose mask-to-box and deepest-point labels are faithful for thin, curved, or repeated parts, and on those images not overlapping the PascalPart and PartImageNet test sets.

What would settle it

A provenance audit comparing image hashes between the 1,200-image training set and the PascalPart/PartImageNet test sets, plus a per-part-type breakdown of gIoU for thin or curved parts (tails, handles, rims), would settle the transfer claim: leakage or label unfaithfulness on those parts would show the headline 38.59/56.87 scores are inflated.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Part grounding in MLLMs can be improved substantially without scale or part-specific supervised fine-tuning; a 4B model suffices.
  • The reflective step's benefit is largely internalized during training, so at inference the model can often be run with a simpler prompt at nearly the same accuracy, saving tokens and time.
  • The same reasoning-guided pipeline transfers to reasoning segmentation, suggesting the hierarchy and reward design generalize beyond part boxes.
  • Object-level referring degrades only modestly (about 3 points), so part-centric reinforcement does not destroy general grounding competence.
  • Decoder-agnostic: swapping the frozen mask decoder barely changes results, meaning the gains live in the language model's box and point predictions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One testable extension: apply the same object-first plus reward framework to fine-grained attributes (color, texture, keypoints), where the same single-step bias may exist.
  • The mask-derived training labels (connected-component boxes, deepest-interior points) plausibly cap performance on thin or curved parts; using polygon-aware or rotated labels could push the reported numbers higher, and that should be measured before adopting the approach as-is.
  • The part-containment reward's tolerance of any matching parent object suggests multi-instance training data would be the next lever; the paper itself flags this limitation.
  • Because the refinement step converges to verification, the active-perception crop loop may shift from correction to confidence calibration; testing on out-of-distribution images where first answers remain imperfect would reveal when crops still matter.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Object-Part Hierarchical Reflective Grounding (OP-HRG), a structured prompting protocol for part-level visual grounding in which the model first decides whether the query is an object or a part, localizes the parent object for part queries, produces an initial part localization, self-critiques, and optionally re-encodes crops before giving a final answer. This is trained with a part-aware GRPO framework using stage-wise rewards: base localization rewards, hierarchical rewards (object hint, part containment), and reflective refinement rewards (improvement over the initial answer and an external baseline). Using Qwen3-VL-Instruct-4B with a frozen SAM2 decoder, trained on InstructPart train plus the VisionReasoner 7k set, the paper reports gIoU 75.56 on InstructPart, 38.59 on PascalPart, 56.87 on PartImageNet, 87.50 on Pascal-Obj, and 69.6 on ReasonSeg, surpassing 7B grounding MLLMs and SAM3 on the part benchmarks. Ablations show both the hierarchical structure and the reflective refinement contribute, and that the reflective step mostly acts as a train-time regularizer.

Significance. If the results hold, the paper makes a strong practical contribution: a 4B model, with a frozen mask decoder and no SFT cold-start, outperforms larger grounding MLLMs and a specialized text-promptable segmenter on fine-grained part grounding, with only a modest trade-off on RefCOCO. The paper is unusually transparent in several respects: the reward design is validated against reward exploitation (App. C.6, Fig. 6), the mask decoder is swapped out as a control (Table 2), a plain-prompt baseline is reported in App. H.2, and all baseline numbers are said to be recomputed under a single protocol (App. I). These are genuine strengths. The central claim, however, depends on two things that need attention: an unverified disjointness assertion for the zero-shot benchmarks, and a main-table comparison that uses a poorly parsed structured-prompt baseline rather than the model's actual plain-prompt capability.

major comments (3)
  1. [§4.2–4.3 (zero-shot claim)] The assertion that the model 'has seen neither images nor annotations from either benchmark' (PascalPart/PartImageNet) is load-bearing for the headline cross-dataset numbers (38.59/56.87 gIoU). The paper provides no provenance audit: no image-ID overlap check, no near-duplicate/hash verification, and no account of how the InstructPart train split and VisionReasoner 7k set were sourced relative to PASCAL-VOC and ImageNet. Since all three datasets are natural images, disjointness cannot be assumed. Please provide a concrete audit; without it, the zero-shot interpretation of these results is not established.
  2. [Table 1 vs App H.2] The main-table 'zero-shot OP-HRG prompt' baseline (31.45/21.95) is not a fair reference for the RL contribution: App. H.2 shows the same base model under a plain single-step prompt reaches 72.39/50.20, while the structured prompt parses only 52.7%/56.6% of the time without RL. Reporting only the structured-prompt baseline overstates the gain of RL (+44.11 vs +3.17 on InstructPart). The fair comparison still supports the method (+3.17 in-domain, +6.67 zero-shot PartImageNet), but the main text and abstract should be reframed around the plain-prompt baseline, and App. H.2 should be moved into the main results.
  3. [§4.2 and §3.5 (object-box supervision)] The hierarchical rewards require ground-truth object boxes for the object hint reward and part containment reward, but §4.2 only describes deriving part boxes and points from InstructPart part masks. It is not explained how object-level ground truth is obtained for the InstructPart training samples. Please specify the derivation (e.g., union of part masks per object, or a separate object-annotation stream) and validate that these boxes are sufficiently accurate to serve as reward targets. Without this, the hierarchical rewards are underspecified.
minor comments (5)
  1. [Eq. (2) / App. C.3, C.5] State explicitly in the main text that IoU_baseline is precomputed on the InstructPart training split only and is a fixed per-sample scalar, independent of the test evaluations in Table 1. This will preempt the appearance of a circular use of SAM3, which is also a baseline in Table 1.
  2. [Fig. 3] The <object_hint> example shows only a bbox field, but the prompt in Appendix B requires both bbox_2d and point_2d in that block. Make the figure consistent with the prompt.
  3. [Table 7 / H.2] The ablation 'Ours w/o hierarchy & refinement' (70.32) is below the plain-prompt base (72.39) reported in H.2. The paper should acknowledge this and explain why standard-reward RL under the OP-HRG prompt does not recover plain-prompt performance; otherwise it invites an interpretation that the structure itself is harmful without the hierarchical rewards.
  4. [§4.5 / Table 3] The analysis says the reflective step's benefit is 'internalized into the weights' and a plain single-answer prompt retains 75.40, yet the active-perception variant (Table 3) applies refinement at inference with additional gains. Clarify how these two findings are reconciled—specifically, whether the AP evaluation still uses the two-pass crop-refinement protocol.
  5. [Table 6] Report the inference time and token count for the plain-prompt base model as well, so readers can separate the cost of the structured protocol from the cost of RL-induced output compaction.

Circularity Check

0 steps flagged

No significant circularity: held-out benchmark evaluations, disclosed training signals, and no self-citation chain.

full rationale

Potential circularity candidates were checked and rejected. (1) The SAM3 reference in the improvement reward (Eq. 2 / App. C.3, C.5) is a per-sample IoU threshold computed on the InstructPart training split and is disclosed in Sec. 4.2. The headline comparison to SAM3 is on the InstructPart test split plus cross-dataset PascalPart/PartImageNet; the latter never saw the SAM3 reward, and the former is a held-out split. Optimizing against a baseline on train and evaluating on test is standard supervised/RL evaluation, not a prediction that is equal to its input by construction. (2) The paper contains no self-citation chain: closest prior pipelines (Seg-Zero, VisionReasoner) are external works, and no "uniqueness" theorem is invoked to force the design. (3) The object-part hierarchy is encoded both in the task definition and in rewards (object hint, containment), but the reported masks come from a frozen SAM2 decoder on images not used for the reward computation; there is no equation in which the output metric is defined as the optimized reward. (4) The H.2 plain-prompt baseline (72.39/50.20) exceeds the zero-shot OP-HRG prompt (31.45/21.95), and the paper analyzes this as a reasoning-execution gap; the final +3.17/+6.67 over the plain baseline is an empirical ablation, not a fitted artifact. (5) The zero-shot imageset disjointness is asserted but un-audited; that is an external validity/data-provenance risk, not a circularity. The paper's own limitations (reflective step converges to verification; containment can match the wrong same-category object) undercut strength but are not circular. No step reduces to its own input by definition; accordingly no circular steps are reported.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The method rests on the premise that pretrained MLLMs already contain part-reasoning capacity and that structured prompts + rewards can elicit it; on the fidelity of mask-derived box/point supervision from 1200 InstructPart images; on non-overlap with the zero-shot benchmarks; and on the faithfulness of box-level rewards to mask-level gIoU. No new entities are postulated. Free parameters are reward thresholds/weights and training hyperparameters, all hand-set with no per-benchmark tuning claimed.

free parameters (6)
  • α_l1 (adaptive L1 box threshold scaling) = 0.10
    Hand-set; controls how tightly predicted boxes must match GT; chosen to balance reward ambiguity vs sparsity (App C.1, Table 8).
  • α_p (point distance threshold scaling) = 0.20
    Hand-set; point tolerance relative to GT box diagonal; looser than α_l1 because points need only fall inside the region (App C.1).
  • τ clamps (L1 and point thresholds) = ℓ1: 3–10 px; point: 5–30 px
    Min/max caps on the adaptive tolerances; hand-set to avoid degenerate thresholds for very small/large regions (Table 8).
  • Reward weights (λ_iou = 2.0, others 1.0) = 2.0 for IoU; 1.0 otherwise
    Hand-set; λ_iou also re-scales the SAM3 baseline in the improvement reward (Eq. 2, C.2, C.3), which embeds the opponent's score into the training signal.
  • External baseline IoU_baseline = SAM3 box IoU per InstructPart train sample
    The improvement reward credits only surpassing the stronger of the model's own first answer and 2×SAM3 IoU (C.3, C.5); this is the main calibration point for the in-domain comparison.
  • Training hyperparameters = lr 1e-6, β_KL 1e-2, ϵ 0.2/0.3, temp 1.2, group 4, 1300 steps, MaxCrops 4
    Hand-set GRPO/stability choices (Table 9); authors state no per-benchmark tuning, but they are still free parameters of the pipeline.
axioms (6)
  • domain assumption Pretrained MLLMs already contain latent hierarchical part-reasoning capacity that structured prompting + RL can activate.
    The entire premise for why RL-on-prompt-structure works without an SFT cold-start; stated in §1 and §6.
  • domain assumption Frozen SAM2/SAM3 decoders turn box+point prompts into accurate masks.
    The decoupled design assumes the mask decoder is not the bottleneck; Table 2 decoder-swap supports this.
  • domain assumption InstructPart train masks converted to boxes + deepest-interior points are faithful part supervision.
    Connected-component axis-aligned boxes can be loose for thin or curved parts (tails, rims, handles), mis-calibrating compactness and localization rewards (§4.2).
  • domain assumption No image-level overlap between InstructPart train images and PascalPart/PartImageNet test sets.
    The cross-dataset zero-shot claim depends on this; asserted in §4.3 but not audited.
  • domain assumption Hungarian-matched IoU/L1/point rewards monotonically track downstream mask gIoU.
    Reward design assumes box-level optimization transfers to the mask-level metric through the frozen decoder (§3.5).
  • standard math Standard part-grounding evaluation: query only objects/parts known present, report gIoU.
    Field-standard protocol (App E); excludes true-negative and open-vocabulary detection scenarios.

pith-pipeline@v1.3.0-alltime-deepseek · 28433 in / 22555 out tokens · 199411 ms · 2026-08-01T23:33:14.781946+00:00 · methodology

0 comments
read the original abstract

Multimodal large language models (MLLMs) ground whole objects well from free-form language queries, but they struggle when the query names a part rather than the object. We trace this to a missing object-part hierarchy, since parts are localized in the same single step used for objects. We propose Object-Part Hierarchical Reflective Grounding (OP-HRG), a coarse-to-fine reasoning-guided grounding strategy that first localizes the parent object and then the part within it. A self-check then reflects on the result, with an extension to re-encode the predicted crop to inspect the region it is correcting. We introduce a part-aware GRPO framework to train our pipeline with stage-wise rewards. A 4B model trained this way outperforms 7B grounding LLMs and SAM3 across PascalPart, PartImageNet, and InstructPart, and transfers to reasoning segmentation.

Figures

Figures reproduced from arXiv: 2607.15374 by Anuj Karpatne, Chia-Wei Tang, Chris Thomas, Hani Alomari, Kazi Sajeed Mehrab, Najibul Haque Sarker, Zaber Ibn Abdul Hakim.

Figure 1
Figure 1. Figure 1: Our method grounds parts coarse-to-fine: locate the object, then the part within it, then self-checks, re-encoding the predicted crop to refine. Abstract. Multimodal large language models (MLLMs) ground whole objects well from free-form language queries, but they struggle when the query names a part rather than the object. We trace this to a missing object-part hierarchy, since parts are localized in the s… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the action steps and reward evaluation for OP-HRG. ring frameworks [53, 56] and open-set detectors such as Grounding DINO [28] and Grounded SAM [40] further advance visual grounding. More recently, rein￾forcement learning has emerged as an effective alignment strategy for grounding MLLMs. GRPO [42] removes the critic network to improve training efficiency and has been widely adopted in visual r… view at source ↗
Figure 3
Figure 3. Figure 3: High-level structure of OP-HRG prompt and corresponding output format. 3.1 Problem Formulation We consider the task of visual grounding of objects and object parts. Our system receives an RGB image I ∈ R H×W×3 and a natural language query q. The query may denote a whole object category (e.g., “cat”) or a semantic part of an object (e.g., “cat’s tail”). During inference, the model must interpret the image a… view at source ↗
Figure 4
Figure 4. Figure 4: Examples of the reflective step from an intermediate model checkpoint. Are both object-part hierarchy and the reflective refinement com￾ponents necessary? We ablate the two core mechanisms of OP-HRG indepen￾dently. Training with only standard localization rewards (IoU, L1, point accu￾racy) and no OP-HRG structure reaches 70.32 gIoU, which confirms that GRPO alignment helps but trails our full method. Remov… view at source ↗
Figure 5
Figure 5. Figure 5: Refinement behavior over training. Top: revision rate and mean IoU gain. Bot￾tom: initial vs. final gIoU. We observe that as training pro￾gresses over many steps, the reason￾ing model converges toward produc￾ing higher quality predictions at the initial stage, which in turn leads to the reflective mechanism consistently declining to adjust. This is a natu￾ral consequence of optimization: as the model’s fir… view at source ↗
Figure 6
Figure 6. Figure 6: Training curves for the ablated improvement reward variant in which the exter￾nal baseline IoU is excluded, reducing the improvement term to max(0, Rfinal IoU −R initial IoU ). The x-axis shows training steps. Panels show (left) initial-answer IoU (initial_iou), (center) improvement reward (reward/improvement), and (right) final-answer IoU (reward/final_iou). After an initial rise, the initial-answer IoU c… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative example on InstructPart [PITH_FULL_IMAGE:figures/full_fig_p036_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative example on InstructPart [PITH_FULL_IMAGE:figures/full_fig_p037_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative example on InstructPart [PITH_FULL_IMAGE:figures/full_fig_p038_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative example on PascalPart [PITH_FULL_IMAGE:figures/full_fig_p039_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative example on PascalPart [PITH_FULL_IMAGE:figures/full_fig_p040_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative example on PartImageNet [PITH_FULL_IMAGE:figures/full_fig_p041_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative example on PartImageNet [PITH_FULL_IMAGE:figures/full_fig_p042_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 15 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2511.21631 (2025)

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-VL technical report. arXiv preprint arXiv:2511.21631 (2025)

    Pith/arXiv arXiv 2025

  2. [2]

    arXiv preprint arXiv:2502.13923 (2025).https: //doi.org/10.48550/arXiv.2502.13923,https://arxiv.org/abs/2502.13923

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., Lin, J.: Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923 (2025).https: //doi.org/10.48550/arXiv.2...

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.13923 2025

  3. [3]

    arXiv preprint arXiv:2505.14231 (2025)

    Bai, S., Li, M., Liu, Y., Tang, J., Zhang, H., Sun, L., Chu, X., Tang, Y.: UniVG-R1: Reasoning guided universal visual grounding with reinforcement learning. arXiv preprint arXiv:2505.14231 (2025)

    Pith/arXiv arXiv 2025

  4. [4]

    arXiv preprint arXiv:2505.20272 (2025)

    Cao, M., Zhao, H., Zhang, C., Chang, X., Reid, I., Liang, X.: Ground-R1: In- centivizing grounded visual reasoning via reinforcement learning. arXiv preprint arXiv:2505.20272 (2025)

    arXiv 2025

  5. [5]

    In: International Conference on Learning Representations (2026)

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Al- wala, K.V., Khedr, H., Huang, A., et al.: SAM 3: Segment anything with concepts. In: International Conference on Learning Representations (2026)

    2026

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR)

    Chen, B., Xu, Z., Kirmani, S., Ichter, B., Sadigh, D., Guibas, L., Xia, F.: Spa- tialVLM: Endowing vision-language models with spatial reasoning capabilities. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR). pp. 14455–14465 (June 2024)

    2024

  7. [7]

    arXiv preprint arXiv:2306.15195 (2023)

    Chen, K., Zhang, Z., Zeng, W., Zhang, R., Zhu, F., Zhao, R.: Shikra: Unleash- ing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195 (2023)

    Pith/arXiv arXiv 2023

  8. [8]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., Yuille, A.: Detect what you can: Detecting and representing objects using holistic models and body parts. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1971–1978 (2014)

    1971

  9. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops

    Chen, X., Li, W., Liu, C., Xie, C., Hu, X., Ma, C., Zhu, F., Zhao, R.: On the suitability of reinforcement fine-tuning to visual tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 3382–3386 (2025)

    2025

  10. [10]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., Li, B., Luo, P., Lu, T., Qiao, Y., Dai, J.: Internvl: Scaling up vi- sion foundation models and aligning for generic visual-linguistic tasks. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 24185–24198 (June 2024)

    2024

  11. [11]

    In: Proceed- ings of the Computer Vision and Pattern Recognition Conference

    Choi, J., Lee, S., Lee, M., Lee, S., Shim, H.: Fine-grained image-text correspon- dence with cost aggregation for open-vocabulary part segmentation. In: Proceed- ings of the Computer Vision and Pattern Recognition Conference. pp. 9782–9793 (2025) Reasoning-Guided Part-Level Visual Grounding 17

    2025

  12. [12]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Deitke, M., Clark, C., Lee, S., Tripathi, R., Yang, Y., Park, J.S., Salehi, M., Muen- nighoff, N., Lo, K., Soldaini, L., et al.: Molmo and PixMo: Open weights and open data for state-of-the-art vision-language models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 91–104 (2025)

    2025

  13. [13]

    Fu, X., Liu, M., Yang, Z., Corring, J., Lu, Y., Yang, J., Roth, D., Florencio, D., Zhang, C.: ReFocus: Visual editing as a chain of thought for structured image understanding.In:InternationalConferenceonMachineLearning.pp.17783–17805 (2025)

    2025

  14. [14]

    In: European Conference on Computer Vision

    He, J., Yang, S., Yang, S., Kortylewski, A., Yuan, X., Chen, J.N., Liu, S., Yang, C., Yu, Q., Yuille, A.: PartImageNet: A large, high-quality dataset of parts. In: European Conference on Computer Vision. pp. 128–145. Springer (2022)

    2022

  15. [15]

    arXiv preprint arXiv:2601.09981 (2026)

    He, Y., Chen, W., Jian, Z., Guo, T., Zhou, W., Li, M.: DR2Seg: Decomposed two- stage rollouts for efficient reasoning segmentation in multimodal large language models. arXiv preprint arXiv:2601.09981 (2026)

    arXiv 2026

  16. [16]

    arXiv preprint arXiv:2602.09701 (2026)

    Hegde, S., Chacko, J.S., Banerjee, D., Mahesh, U.: Genseg-r1: Rl-driven vision- language grounding for fine-grained referring segmentation. arXiv preprint arXiv:2602.09701 (2026)

    arXiv 2026

  17. [17]

    In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

    Jian, P., Wu, J., Sun, W., Wang, C., Ren, S., Zhang, J.: Look again, think slowly: Enhancing visual reflection in vision-language models. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 9251–9270 (2025).https://doi.org/10.18653/v1/2025.emnlp-main.470

    doi:10.18653/v1/2025.emnlp-main.470 2025

  18. [18]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kamath, A., Singh, M., LeCun, Y., Synnaeve, G., Misra, I., Carion, N.: Mdetr- modulated detection for end-to-end multi-modal understanding. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1780–1790 (2021)

    2021

  19. [19]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4015–4026 (2023)

    2023

  20. [20]

    Naval research logistics quarterly2(1-2), 83–97 (1955)

    Kuhn, H.W.: The hungarian method for the assignment problem. Naval research logistics quarterly2(1-2), 83–97 (1955)

    1955

  21. [21]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Lai, X., Tian, Z., Chen, Y., Li, Y., Yuan, Y., Liu, S., Jia, J.: LISA: Reasoning seg- mentation via large language model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9579–9589 (2024)

    2024

  22. [22]

    In: European Conference on Computer Vision

    Li, F., Zhang, H., Sun, P., Zou, X., Liu, S., Li, C., Yang, J., Zhang, L., Gao, J.: Segment and recognize anything at any granularity. In: European Conference on Computer Vision. pp. 467–484. Springer (2024)

    2024

  23. [23]

    In: European Conference on Computer Vision

    Li, J., Wu, J., Zhao, W., Bai, S., Bai, X.: PartGLEE: A foundation model for recognizing and parsing any objects. In: European Conference on Computer Vision. pp. 475–494. Springer (2024)

    2024

  24. [24]

    In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition

    Li, L.H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., Hwang, J.N., et al.: Grounded language-image pre-training. In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10965–10975 (2022)

    2022

  25. [25]

    In: Chiruzzo, L., Ritter, A., Wang, L

    Li, Z., Luo, R., Zhang, J., Qiu, M., Huang, X., Wei, Z.: VoCoT: Unleashing vi- sually grounded multi-step reasoning in large multi-modal models. In: Chiruzzo, L., Ritter, A., Wang, L. (eds.) Proceedings of the 2025 Conference of the Na- tions of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: L...

    2025

  26. [26]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Liu, C., Ding, H., Jiang, X.: Gres: Generalized referring expression segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 23592–23601 (2023)

    2023

  27. [27]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition

    Liu, M., Zhu, Y., Cai, H., Han, S., Ling, Z., Porikli, F., Su, H.: Partslip: Low-shot part segmentation for 3d point clouds via pretrained image-language models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 21736–21746 (2023)

    2023

  28. [28]

    In: European conference on computer vision

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al.: Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection. In: European conference on computer vision. pp. 38–55. Springer (2024)

    2024

  29. [29]

    Advances in Neu- ral Information Processing Systems38, 126078–126108 (2025),https://papers

    Liu, Y., Ma, Z., Pu, J., Qi, Z., Wu, Y., Shan, Y., Wen, C.C.: Unipixel: Unified ob- ject referring and segmentation for pixel-level visual reasoning. Advances in Neu- ral Information Processing Systems38, 126078–126108 (2025),https://papers. nips.cc/paper_files/paper/2025/file/b783c44ba9adbc30344473dc633b4869- Paper-Conference.pdf

    2025

  30. [30]

    arXiv preprint arXiv:2503.06520 (2025)

    Liu, Y., Peng, B., Zhong, Z., Yue, Z., Lu, F., Yu, B., Jia, J.: Seg-Zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520 (2025)

    Pith/arXiv arXiv 2025

  31. [31]

    In: International Conference on Learning Representations (2026)

    Liu,Y.,Qu,T.,Zhong,Z.,Peng,B.,Liu,S.,Yu,B.,Jia,J.:VisionReasoner:Unified reasoning-integrated visual perception via reinforcement learning. In: International Conference on Learning Representations (2026)

    2026

  32. [32]

    In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision

    Liu, Z., Sun, Z., Zang, Y., Dong, X., Cao, Y., Duan, H., Lin, D., Wang, J.: Visual- RFT: Visual reinforcement fine-tuning. In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision. pp. 2034–2044 (2025)

    2034

  33. [33]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition

    Lüddecke, T., Ecker, A.: Image segmentation using text and image prompts. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 7086–7096 (2022)

    2022

  34. [34]

    Ma, X., Ding, Z., Luo, Z., Chen, C., Guo, Z., Wong, D.F., Feng, X., Sun, M.: DeepPerception: Advancing R1-like cognitive visual perception in MLLMs for knowledge-intensive visual grounding (2025),https://arxiv.org/abs/2503. 12797

    2025

  35. [35]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (2025)

    Man, Y., Huang, D.A., Liu, G., Sheng, S., Liu, S., Gui, L.Y., Kautz, J., Wang, Y.X., Yu, Z.: Argus: Vision-centric reasoning with grounded chain-of-thought. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (2025)

    2025

  36. [36]

    arXiv preprint arXiv:2503.07365 (2025)

    Meng, F., Du, L., Liu, Z., Zhou, Z., Lu, Q., Fu, D., Han, T., Shi, B., Wang, W., He, J., et al.: MM-Eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365 (2025)

    Pith/arXiv arXiv 2025

  37. [37]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR)

    Mitra, C., Huang, B., Darrell, T., Herzig, R.: Compositional chain-of-thought prompting for large multimodal models. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR). pp. 14420–14431 (June 2024)

    2024

  38. [38]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Rasheed,H.,Maaz,M.,Shaji,S.,Shaker,A.,Khan,S.,Cholakkal,H.,Anwer,R.M., Xing, E., Yang, M.H., Khan, F.S.: GLaMM: Pixel grounding large multimodal model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13009–13018 (2024)

    2024

  39. [39]

    In: International Conference on Learning Representations (2025) Reasoning-Guided Part-Level Visual Grounding 19

    Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., et al.: SAM 2: Segment anything in images and videos. In: International Conference on Learning Representations (2025) Reasoning-Guided Part-Level Visual Grounding 19

    2025

  40. [40]

    arXiv preprint arXiv:2401.14159 (2024)

    Ren, T., Liu, S., Zeng, A., Lin, J., Li, K., Cao, H., Chen, J., Huang, X., Chen, Y., Yan, F., et al.: Grounded SAM: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 (2024)

    Pith/arXiv arXiv 2024

  41. [41]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Ren, Z., Huang, Z., Wei, Y., Zhao, Y., Fu, D., Feng, J., Jin, X.: Pixellm: Pixel rea- soning with large multimodal model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 26374–26383 (2024)

    2024

  42. [42]

    arXiv preprint arXiv:2402.03300 (2024)

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024)

    Pith/arXiv arXiv 2024

  43. [43]

    In: Workshop on Reasoning and Planning for Large Language Models (2025),https://openreview.net/forum?id=T2IHuIib74

    Sharma, K., Vats, V.: Think to ground: Improving spatial reasoning in LLMs for better visual grounding. In: Workshop on Reasoning and Planning for Large Language Models (2025),https://openreview.net/forum?id=T2IHuIib74

    2025

  44. [44]

    arXiv preprint arXiv:2505.19094 (2025)

    Shen, C., Wei, W., Qu, X., Cheng, Y.: Satori-R1: Incentivizing multimodal reason- ing with spatial grounding and verifiable rewards. arXiv preprint arXiv:2505.19094 (2025)

    arXiv 2025

  45. [45]

    arXiv preprint arXiv:2504.07615 (2025)

    Shen, H., Liu, P., Li, J., Fang, C., Ma, Y., Liao, J., Shen, Q., Zhang, Z., Zhao, K., Zhang, Q., et al.: VLM-R1: A stable and generalizable R1-style large vision- language model. arXiv preprint arXiv:2504.07615 (2025)

    Pith/arXiv arXiv 2025

  46. [46]

    arXiv preprint arXiv: 2409.19256 (2024)

    Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., Wu, C.: Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256 (2024)

    Pith/arXiv arXiv 2024

  47. [47]

    In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision

    Sun, P., Chen, S., Zhu, C., Xiao, F., Luo, P., Xie, S., Yan, Z.: Going denser with open-vocabulary part segmentation. In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision. pp. 15453–15465 (2023)

    2023

  48. [48]

    In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Wan,Z.,Xie,Y.,Zhang,C.,Lin,Z.,Wang,Z.,Stepputtis,S.,Ramanan,D.,Sycara, K.P.: InstructPart: Task-oriented part segmentation with instruction reasoning. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 24202–24227 (2025)

    2025

  49. [49]

    arXiv preprint arXiv:2409.12191 (2024)

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al.: Qwen2-VL: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024)

    Pith/arXiv arXiv 2024

  50. [50]

    Advances in Neural Information Processing Systems36, 70094–70114 (2023)

    Wei, M., Yue, X., Zhang, W., Kong, S., Liu, X., Pang, J.: OV-PARTS: Towards open-vocabulary part segmentation. Advances in Neural Information Processing Systems36, 70094–70114 (2023)

    2023

  51. [51]

    arXiv preprint arXiv:2312.17240 (2023)

    Yang, S., Qu, T., Lai, X., Tian, Z., Peng, B., Liu, S., Jia, J.: Lisa++: An improved baseline for reasoning segmentation with large language model. arXiv preprint arXiv:2312.17240 (2023)

    Pith/arXiv arXiv 2023

  52. [52]

    Yang, Y., He, X., Pan, H., Jiang, X., Deng, Y., Yang, X., Lu, H., Yin, D., Rao, F., Zhu, M., et al.: R1-Onevision: Advancing generalized multimodal reasoning throughcross-modalformalization.In:ProceedingsoftheIEEE/CVFInternational Conference on Computer Vision. pp. 2376–2385 (2025)

    2025

  53. [53]

    In: International Conference on Learning Representations (2024)

    You, H., Zhang, H., Gan, Z., Du, X., Zhang, B., Wang, Z., Cao, L., Chang, S.F., Yang, Y.: Ferret: Refer and ground anything anywhere at any granularity. In: International Conference on Learning Representations (2024)

    2024

  54. [54]

    arXiv preprint arXiv:2506.22624 (2025)

    You, Z., Wu, Z.: Seg-R1: Segmentation can be surprisingly simple with reinforce- ment learning. arXiv preprint arXiv:2506.22624 (2025)

    Pith/arXiv arXiv 2025

  55. [55]

    arXiv preprint arXiv:2501.04001 (2025) 20 K

    Yuan,H.,Li,X.,Zhang,T.,Sun,Y.,Huang,Z.,Xu,S.,Ji,S.,Tong,Y.,Qi,L.,Feng, J., et al.: Sa2VA: Marrying SAM2 with LLaVA for dense grounded understanding of images and videos. arXiv preprint arXiv:2501.04001 (2025) 20 K. Mehrab et al

    Pith/arXiv arXiv 2025

  56. [56]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yuan, Y., Li, W., Liu, J., Tang, D., Luo, X., Qin, C., Zhang, L., Zhu, J.: Os- prey: Pixel understanding with visual instruction tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 28202– 28211 (2024)

    2024

  57. [57]

    arXiv preprint arXiv:2601.13633 (2026)

    Zhan, G., Li, C., Liu, Z., Lu, Y., Wu, Y., Han, S., Zhu, L.: Scaling test-time inference for visual grounding. arXiv preprint arXiv:2601.13633 (2026)

    Pith/arXiv arXiv 2026

  58. [58]

    In: Che, W., Nabende, J., Shutova, E., Pilehvar, M.T

    Zhang, R., Zhang, B., Li, Y., Zhang, H., Sun, Z., Gan, Z., Yang, Y., Pang, R., Yang, Y.: Improve vision language model chain-of-thought reasoning. In: Che, W., Nabende, J., Shutova, E., Pilehvar, M.T. (eds.) Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1631–1662. Association for Compu...

    doi:10.18653/v1/2025.acl- 2025

  59. [59]

    Transactions on Machine Learning Re- search (2023)

    Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., Smola, A.: Multimodal chain- of-thought reasoning in language models. Transactions on Machine Learning Re- search (2023)

    2023

  60. [60]

    arXiv preprint arXiv:2602.03547 (2026)

    Zhou, D., He, M., Fang, Z., Yao, X., Liu, Y., Knoll, A., Cao, H.: AffordanceGrasp- R1: Leveraging reasoning-based affordance segmentation with reinforcement learn- ing for robotic grasping. arXiv preprint arXiv:2602.03547 (2026)

    arXiv 2026

  61. [61]

    {Question}

    Zhu, L., Ouyang, B., Zhang, Y., Cheng, T., Hu, R., Shen, H., Ran, L., Chen, X., Yu, L., Liu, W., Wang, X.: LENS: Learning to segment anything with unified rein- forced reasoning. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 40, pp. 13952–13960 (2026).https://doi.org/10.1609/aaai.v40i16.38405 Reasoning-Guided Part-Level Visual Gr...

    doi:10.1609/aaai.v40i16.38405 2026