Pith. sign in

REVIEW 3 major objections 6 minor 41 references

Pixel masks beat bounding boxes for visual reasoning in AI models

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-08 23:38 UTC pith:Y6RWRMZN

load-bearing objection Solid method, consistent gains, but the central claim (mask vs. bbox) is never directly ablated the 3 major comments →

arxiv 2607.05798 v1 pith:Y6RWRMZN submitted 2026-07-07 cs.CV cs.AI

Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning

Yake Wei , Yuan Wang , Fengyun Rao , Jing Lyu , Di Hu This is my paper
classification cs.CV cs.AI
keywords multimodal large language modelsvisual reasoningsemantic segmentationpixel groundingthinking with imagesreinforcement learningreferring expression segmentation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that when multimodal large language models (MLLMs) need to zoom in on a region of an image to answer a question, the standard practice of cropping a rectangular bounding box is a mistake. Bounding boxes inevitably include irrelevant background and fail to separate overlapping objects, introducing visual noise that degrades reasoning. The authors propose SegAnswer, which replaces bounding-box crops with pixel-level segmentation masks. The model learns to generate a precise mask isolating only the target object, and the segmented patches retain their original position indices so the model preserves spatial relationships. The central claim is that this pixel-level grounding produces cleaner visual input, which directly improves downstream reasoning quality. The method is trained in three stages: first teaching the model to segment using a special token and a mask decoder, then fine-tuning it to use segmentation as an intermediate reasoning step, and finally optimizing the whole pipeline with reinforcement learning using an outcome-based reward. SegAnswer is built on Qwen2.5-VL-7B and evaluated across high-resolution perception, general perception, and hallucination benchmarks, showing consistent gains over both the base model and prior visual reasoning methods that use bounding-box zoom-in. The paper also reports strong segmentation performance on RefCOCO-family benchmarks, arguing the model's grounding is reliable enough to trust as an intermediate step.

Core claim

The core discovery is that pixel-level segmentation masks, when used as the zoom-in mechanism during MLLM visual reasoning, yield consistent improvements over bounding-box-based crops across multiple benchmark categories. On high-resolution perception, V* rises from 77.5 to 86.4 and HR-Bench 8K from 63.4 to 71.3 relative to the base model, surpassing two prior visual reasoning methods. On general perception, MMBench improves from 83.0 to 87.5. The mechanism is twofold: segmentation removes background clutter that bounding boxes would otherwise include, and the retained position indices of segmented patches preserve spatial structure in a way that aligns with how MLLMs already encode visual位置

What carries the argument

The central object is the <|seg|> token, a special token the MLLM emits to trigger segmentation. The hidden state at that token is projected via an MLP into the input space of a SAM 2.1 mask decoder, which produces a pixel-level mask. The segmented image patches retain their original position indices from the source image, so they slot into the model's positional embedding structure as sparse but spatially coherent tokens. The three-stage pipeline—pixel grounding, interleaved SFT, and RL with outcome rewards—trains the model to decide when to segment, execute the segmentation, and use the result for answering.

Load-bearing premise

The paper assumes the performance gains come from using segmentation masks instead of bounding boxes, but it never runs the same pipeline with bounding-box zoom-in as the only changed variable, so the improvements could stem from the training stages or data rather than the mask design itself.

What would settle it

Run SegAnswer's full three-stage pipeline identically but replace the segmentation-based zoom-in with bounding-box crops at inference and training time; if results remain comparable, the pixel-grounding mechanism is not the causal driver of the gains.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If pixel-level grounding is the right zoom-in unit, then any MLLM visual reasoning pipeline that currently uses bounding-box crops could be upgraded by swapping the crop mechanism for a segmentation-based one, potentially gaining improvements without changing the base language model.
  • The position-index preservation trick—keeping original patch coordinates on segmented patches—suggests that spatial coherence matters more than visual completeness for downstream reasoning, which could inform how other visual operations (e.g., style transfer, highlighting) are designed.
  • The outcome-driven RL stage shows that intermediate visual operations can be optimized without ground-truth supervision on the segmentation itself, only on final answer quality, which lowers the barrier to training multi-step visual reasoning.
  • Strong segmentation performance on RefCOCO-family benchmarks suggests the model's grounding is not task-specific to reasoning but generalizes as a referring segmentation system, which could make it useful as a standalone grounding tool.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper does not include a controlled ablation where the same three-stage pipeline is run with bounding-box zoom-in instead of segmentation, so the gains could in principle come from the training recipe, data mix, or RL optimization rather than from the mask-vs-box design choice itself. A clean ablation isolating the zoom-in unit would be the decisive test.
  • The position-index preservation argument implies that a dense re-encoding of the segmented crop (as a new contiguous image) would lose spatial information, but the paper does not directly compare against a version where segmented patches are re-indexed from zero, so the contribution of position preservation is asserted but not isolated.
  • If segmentation quality is the bottleneck, then the method's reasoning gains should correlate with segmentation accuracy on a per-instance basis—a per-question analysis linking mask IoU to answer correctness would test whether the mechanism is truly grounding-driven.
  • The approach may face scaling challenges: generating and encoding sparse segmented patches for multiple regions in a single reasoning trajectory could increase token sequence length and computational cost compared to a single bounding-box crop, which the paper does not analyze.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes SegAnswer, which replaces bounding-box-based zoom-in with pixel-level segmentation masks during MLLM visual reasoning. The method uses a three-stage training pipeline: (1) pixel grounding with SAM 2.1 as mask decoder, (2) multimodal interleaved SFT using VisualCOT, and (3) RL-based reasoning optimization using DAPO on ViRL39K. The base model is Qwen2.5-VL-7B. Evaluations span high-resolution perception (V*, HR-Bench), general perception (MMBench, VisuLogic, MMVP), hallucination (POPE, HallusionBench), and referring segmentation (RefCOCO family). Results show consistent improvements over the base model and over bbox-based visual reasoning methods (Pixel Reasoner, DeepEyes) on most benchmarks, and strong segmentation performance surpassing segmentation-specialized methods.

Significance. The paper addresses a well-motivated limitation of bbox-based zoom-in in MLLM visual reasoning and proposes a concrete alternative with a clearly described training pipeline. The segmentation results on RefCOCO/RefCOCO+/RefCOCOg (Table 3) are strong, and the consistent gains across multiple external benchmarks are noteworthy. The position-index-preservation design (§3.2) is a thoughtful detail. However, the central causal claim—that pixel-level segmentation is superior to bounding boxes for zoom-in—is not directly tested via a controlled ablation, which limits the strength of the contribution claim.

major comments (3)
  1. §4.4, Table 1: The paper's central claim is that pixel-level segmentation masks yield better reasoning than bounding-box crops (§1, §4.4). However, no controlled ablation isolates this design choice. Table 1 compares SegAnswer against the base Qwen2.5-VL-7B (no zoom-in at all) and against Pixel Reasoner/DeepEyes (which differ in base model, training data, RL algorithm, and overall pipeline). The delta over the base model (+8.9 on V*, +4.5 on MMBench) conflates 'having a zoom-in mechanism' with 'having segmentation-based zoom-in specifically.' Without running the identical three-stage pipeline with bounding-box zoom-in replacing segmentation, one cannot distinguish whether gains come from the segmentation design or from the training recipe. This is the single most load-bearing gap because the paper frames its contribution as a design choice (mask vs. bbox), not as a training recipe. Abbox
  2. §3.2 and §1: The paper claims that keeping the position index of segmented patches 'aligns more seamlessly with how MLLMs structure visual tokens via positional embeddings' and that 'sparse segmented image patches can effectively reflect spatial relations.' This is a potentially important design point, but it is never empirically tested. No ablation compares position-index-preservation vs. re-indexing from (0,0), and no analysis demonstrates that spatial relations are actually preserved in practice. Since this is presented as a core design rationale, at least a small ablation or quantitative analysis should be included to support the claim. Without it, the claim remains speculative.
  3. The paper contains no ablation table whatsoever—no stage-wise contribution analysis, no data-mix ablation, no comparison of segmentation vs. bbox within the same pipeline. Given that the three-stage training recipe (pixel grounding SFT, interleaved SFT, RL) is itself a substantial component of the method, the absence of any ablation makes it impossible to attribute gains to specific design choices. At minimum, an ablation isolating the zoom-in mechanism (mask vs. bbox) and a stage-wise contribution analysis should be included.
minor comments (6)
  1. Table 2: Chain-of-Focus achieves 88.0 on V*, higher than SegAnswer's 86.4, but this is not discussed in the text. The paper should acknowledge this result and discuss the comparison.
  2. §3.1: The prompt template shows 'Segment the food wrapped up closest to plate' but does not specify how the target phrase is generated during training—whether it comes from ground-truth annotations or model-generated descriptions. Clarifying this would help reproducibility.
  3. §4.2: The number of rollouts is listed as 4, which seems low for RL training. Please confirm this value and, if correct, briefly justify why this is sufficient. Also, the total number of training steps for Stage 3 is not reported, though Figure 4 shows up to 600 steps.
  4. Figure 1: The figure effectively illustrates the motivation but the patch position indices are small and difficult to read. Consider enlarging or simplifying the visualization for clarity.
  5. §4.6: The qualitative examples in Figure 3 are helpful but all show successful cases. Including failure cases where segmentation is incorrect or where the model fails to identify the correct region would strengthen the analysis.
  6. The paper does not report inference time or computational overhead of the segmentation step. Since the method adds a mask decoder and additional forward passes, a brief discussion of computational cost would be valuable.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The central concern—lack of a controlled ablation isolating segmentation-based zoom-in from bbox-based zoom-in within the same pipeline—is well-taken and we agree it must be addressed. We will add the requested ablations (mask vs. bbox, position-index preservation, stage-wise contributions) in the revision.

read point-by-point responses

  1. Referee: §4.4, Table 1: No controlled ablation isolates segmentation vs. bbox zoom-in. Table 1 conflates 'having a zoom-in mechanism' with 'having segmentation-based zoom-in specifically.'

    Authors: We agree this is the most important gap. The referee is correct that Table 1 compares against methods that differ in base model, training data, and RL algorithm, so the delta over the base model conflates the zoom-in mechanism with the segmentation design specifically. We will run the controlled ablation the referee requests: the identical three-stage pipeline (pixel grounding SFT → interleaved SFT → RL on ViRL39K) with bounding-box zoom-in replacing segmentation, keeping all other components fixed. This will be added as an ablation table in the revised manuscript. We expect this to directly isolate the contribution of the mask-vs-bbox design choice. revision: yes

  2. Referee: §3.2 and §1: Position-index-preservation claim is never empirically tested. No ablation compares position-index-preservation vs. re-indexing from (0,0).

    Authors: The referee is correct that we present position-index preservation as a design rationale without empirical validation. We will add an ablation comparing position-index preservation against re-indexing from (0,0) on representative benchmarks (e.g., V* and MMBench, where spatial reasoning matters). This will either confirm the design choice or prompt us to soften the claim to a design heuristic rather than an empirically supported advantage. Either way, the revised manuscript will not leave this claim untested. revision: yes

  3. Referee: No ablation table whatsoever—no stage-wise contribution analysis, no data-mix ablation, no comparison of segmentation vs. bbox within the same pipeline.

    Authors: We accept this criticism. The absence of ablation tables is a genuine weakness in the current submission. In the revision, we will add: (1) the mask-vs-bbox controlled ablation described above, (2) a stage-wise contribution analysis showing the effect of each training stage (base model → +Stage 1 pixel grounding → +Stage 2 interleaved SFT → +Stage 3 RL), and (3) the position-index ablation. We agree that the three-stage training recipe is itself a substantial component and that stage-wise attribution is necessary. We will not add a data-mix ablation unless the referee considers it essential, as our data sources follow established datasets (VisualCOT, ViRL39K) without novel mixing strategies. revision: yes

Circularity Check

0 steps flagged

No significant circularity: derivation is self-contained against external benchmarks

full rationale

The paper proposes SegAnswer, a method replacing bounding-box zoom-in with pixel-level segmentation masks in MLLM visual reasoning. The central claim is evaluated entirely against external benchmarks (V*, HR-Bench, MMBench, POPE, RefCOCO, etc.) authored by independent groups, using externally sourced training data (VisualCOT, ViRL39K, RefCOCO family). No prediction or result reduces to a fitted parameter by construction. The three-stage training pipeline (pixel grounding SFT, interleaved SFT, RL with DAPO) is a standard recipe with independently chosen components (SAM 2.1, Qwen2.5-VL-7B). The only self-citation is LaSagnA (Wei et al. 2024), which shares author Yake Wei with the present paper, but it appears solely as one of six comparison baselines in Table 3 for segmentation performance — it is not load-bearing for any derivation, theorem, or ansatz. No uniqueness theorem is invoked, no ansatz is smuggled via self-citation, and no result is renamed from a known empirical pattern. The absence of a controlled bbox-vs-mask ablation within the same pipeline is a correctness/experimental-design concern (the gains may stem from the training recipe rather than the segmentation design choice), but this is not circularity — the results are genuinely measured against external data, not defined into existence. The derivation chain from method design to empirical evaluation is self-contained.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 1 invented entities

The paper's central premise — that segmentation masks are better than bounding boxes for zoom-in — is a domain assumption stated in §1 and §3. The position-index preservation claim in §3.2 is also an unverified domain assumption. DAPO's transfer to multimodal RL is assumed from its text-only validation. The MLP projector alignment between MLLM hidden states and SAM 2.1 is an architectural assumption not compared against alternatives.

free parameters (7)
  • wa (accuracy reward weight) = 0.8
    Set manually in §4.2; controls the RL reward balance between accuracy and format compliance.
  • wf (format reward weight) = 0.2
    Set manually in §4.2; controls the RL reward balance.
  • Stage 1 projector learning rate = 1e-3
    Chosen for projector pre-training; not tuned via systematic search.
  • Stage 1 LoRA learning rate = 2e-5
    Chosen for joint fine-tuning; standard range but not justified.
  • Stage 2 learning rate = 2e-6
    Chosen for full-parameter SFT; not justified.
  • Stage 3 learning rate = 2e-5
    Chosen for RL training; not justified.
  • Number of rollouts (Stage 3) = 4
    Set for RL training; not justified or ablated.
axioms (4)
  • domain assumption Pixel-level segmentation masks provide cleaner regions of interest than bounding boxes for MLLM visual reasoning.
    Stated in §1 and §3; the central premise of the paper. Not independently verified via controlled ablation.
  • domain assumption Preserving original position indices of segmented patches maintains spatial relations useful for reasoning.
    Asserted in §3.2 and §3.3; not ablated against alternative position encoding schemes.
  • domain assumption DAPO is an effective RL algorithm for multimodal visual reasoning with interleaved visual tokens.
    Invoked in §3.3; cited from Yu et al. 2025. Assumed to transfer from text-only to multimodal setting.
  • domain assumption SAM 2.1 mask decoder can be effectively aligned to MLLM hidden states via a simple MLP projector.
    Stated in §3.1; the architectural choice is assumed sufficient without comparison to alternatives.
invented entities (1)
  • <|seg|> special token independent evidence
    purpose: Interface token that triggers mask decoding from MLLM hidden states during reasoning.
    The token's function is validated empirically through segmentation benchmark results (Table 3) and qualitative reasoning trajectories (Figure 3). It is a design choice rather than a postulated physical entity.

pith-pipeline@v1.1.0-glm · 15030 in / 3718 out tokens · 336187 ms · 2026-07-08T23:38:27.678254+00:00 · methodology

0 comments
read the original abstract

Recent advancements in Multimodal Large Language Models (MLLMs) have evolved from static perception to interleaved visual-language reasoning, often referred to as ``thinking with images''. A basic operation in this reasoning process is to zoom in on regions of interest (often represented with bounding boxes) to acquire finer visual details. In this paper, we propose \textbf{Seg}mentation before \textbf{Answer}ing (SegAnswer), which shifts the unit of zoom-in from the popular bounding box to pixel-level segmentation mask. By employing fine-grained masks to isolate the target area from cluttered environments, segmented visual input yields a more precise region of interest, effectively filtering out redundant background and interfering objects. Furthermore, the discrete patches of segmented visual input align more seamlessly with how MLLMs structure visual tokens via positional embeddings. In experiments, we evaluate SegAnswer across diverse benchmarks, including high-resolution perception, general perception, and hallucination. It achieves consistent improvements and also exhibits considerable performance on segmentation tasks, validating its capability for reliable pixel grounding.

Figures

Figures reproduced from arXiv: 2607.05798 by Di Hu, Fengyun Rao, Jing Lyu, Yake Wei, Yuan Wang.

Figure 1
Figure 1. Figure 1: Comparison between the BBox-based zoom-in operation and our segmentation-based pixel grounding. (a) Rectangular bounding boxes inevitably introduce redundant background regions (e.g., the background around the sloping tennis racket) and fail to precisely disentangle the region of interest from overlapping objects (e.g., the fork and napkin), leading to visual noise and semantic ambiguity. (b) Pixel-level s… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our SegAnswer method. The training pipeline progresses through three stages: Stage 1: Pixel Grounding aligns textual semantics with pixel-level features, training the MLLM to generate segmentation masks via a specialized <|seg|> token and a mask decoder. Stage 2: Multimodal Interleaved SFT enables the model to employ segmentation as an intermediate conversation step, using the generated mask to… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative visualization of reasoning trajectories with SegAnswer on the V* benchmark. We showcase four examples (a-d) where the model answers fine-grained visual perception questions. In each case, the model autonomously determines the need for visual refinement, predicts the <|seg|> token to isolate the target object (e.g., the woman in the sample a, and utilizes the segmented visual context to derive t… view at source ↗
Figure 4
Figure 4. Figure 4: Training reward curves during Stage 3: Reasoning with Pixel Grounding. The solid lines represent the smoothed moving average, while the shaded areas indicate the raw variation at each step. swer with other segmentation-specific methods, including LISA (Lai et al., 2024), Groundhog (Zhang et al., 2024), LaSagnA (Wei et al., 2024), VideoLISA (Bai et al., 2024), VISA (Yan et al., 2024) and Vitron (Fei et al.,… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages · 11 internal anchors

  1. [1]

    Transactions on Machine Learning Research , year=

    LLaVA-OneVision: Easy Visual Task Transfer , author=. Transactions on Machine Learning Research , year=

    work page

  2. [2]

    Qwen2.5-VL Technical Report

    Qwen2. 5-vl technical report , author=. arXiv preprint arXiv:2502.13923 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [3]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    V?: Guided visual search as a core mechanism in multimodal llms , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    work page

  4. [4]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    work page

  5. [5]

    Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs

    Chain-of-Focus: Adaptive Visual Search and Zooming for Multimodal Reasoning via RL , author=. arXiv preprint arXiv:2505.15436 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning

    Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning , author=. arXiv preprint arXiv:2505.15966 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning

    DeepEyes: Incentivizing" Thinking with Images" via Reinforcement Learning , author=. arXiv preprint arXiv:2505.14362 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Lisa: Reasoning segmentation via large language model , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    work page

  9. [9]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Groundhog: Grounding large language models to holistic segmentation , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    work page

  10. [10]

    European Conference on Computer Vision , pages=

    Visa: Reasoning video object segmentation via large language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    work page 2024

  11. [11]

    Advances in neural information processing systems , volume=

    Vitron: A unified pixel-level vision llm for understanding, generating, segmenting, editing , author=. Advances in neural information processing systems , volume=

    work page

  12. [12]

    The Thirteenth International Conference on Learning Representations , year=

    SAM 2: Segment Anything in Images and Videos , author=. The Thirteenth International Conference on Learning Representations , year=

    work page

  13. [13]

    Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers

    Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers , author=. arXiv preprint arXiv:2506.23918 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Transactions on Machine Learning Research , year=

    Multimodal Chain-of-Thought Reasoning in Language Models , author=. Transactions on Machine Learning Research , year=

    work page

  15. [15]

    UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning

    UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning , author=. arXiv preprint arXiv:2505.14231 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    work page

  17. [17]

    European conference on computer vision , pages=

    Mmbench: Is your multi-modal model an all-around player? , author=. European conference on computer vision , pages=. 2024 , organization=

    work page 2024

  18. [18]

    VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models

    Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models , author=. arXiv preprint arXiv:2504.15279 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Eyes wide shut? exploring the visual shortcomings of multimodal llms , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    work page

  20. [20]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Evaluating Object Hallucination in Large Vision-Language Models , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2023

  21. [21]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    work page

  22. [22]

    Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages=

    Referitgame: Referring to objects in photographs of natural scenes , author=. Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages=

    work page 2014

  23. [23]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Generation and comprehension of unambiguous object descriptions , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    work page

  24. [24]

    Advances in neural information processing systems , volume=

    Flamingo: a visual language model for few-shot learning , author=. Advances in neural information processing systems , volume=

    work page

  25. [25]

    OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

    Openflamingo: An open-source framework for training large autoregressive vision-language models , author=. arXiv preprint arXiv:2308.01390 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

    work page

  27. [27]

    International conference on machine learning , pages=

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models , author=. International conference on machine learning , pages=. 2023 , organization=

    work page 2023

  28. [28]

    Advances in Neural Information Processing Systems , volume=

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models , author=. Advances in Neural Information Processing Systems , volume=

    work page

  29. [29]

    arXiv preprint arXiv:2505.20164 , year=

    Visual Abstract Thinking Empowers Multimodal Reasoning , author=. arXiv preprint arXiv:2505.20164 , year=

    work page arXiv

  30. [30]

    ICLR 2025 Workshop on Foundation Models in the Wild , year=

    ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding , author=. ICLR 2025 Workshop on Foundation Models in the Wild , year=

    work page 2025

  31. [31]

    , author=

    Lora: Low-rank adaptation of large language models. , author=. ICLR , volume=

    work page

  32. [32]

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale

    Dapo: An open-source llm reinforcement learning system at scale , author=. arXiv preprint arXiv:2503.14476 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  33. [33]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Scene parsing through ade20k dataset , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    work page

  34. [34]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Coco-stuff: Thing and stuff classes in context , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    work page

  35. [35]

    Proceedings of the IEEE international conference on computer vision , pages=

    The mapillary vistas dataset for semantic understanding of street scenes , author=. Proceedings of the IEEE international conference on computer vision , pages=

    work page

  36. [36]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Paco: Parts and attributes of common objects , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    work page

  37. [37]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Detect what you can: Detecting and representing objects using holistic models and body parts , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    work page

  38. [38]

    VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning

    Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning , author=. arXiv preprint arXiv:2504.08837 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  39. [39]

    CoRR , year=

    Visual cot: Unleashing chain-of-thought reasoning in multi-modal language models , author=. CoRR , year=

    work page

  40. [40]

    LaSagnA: Language-based Segmentation Assistant for Complex Queries

    Lasagna: Language-based segmentation assistant for complex queries , author=. arXiv preprint arXiv:2404.08506 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  41. [41]

    Advances in Neural Information Processing Systems , volume=

    One token to seg them all: Language instructed reasoning segmentation in videos , author=. Advances in Neural Information Processing Systems , volume=

    work page