Pith. sign in

REVIEW 4 major objections 5 minor 39 references

Attribute-based Object Grounding and Robot Grasp Detection with Spatial Reasoning

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A compact bidirectional fusion model grounds objects from spatial language and grasps them, even when identical objects are present.

desk verdict A competent, well-ablated language-grasping system whose real-robot advantage is statistically unestablished at 24 trials. read the letter →

arxiv 2509.08126 v1 pith:OLYOYXS4 submitted 2025-09-09 cs.RO

Houjian Yu , Zheming Zhou , Min Sun , Omid Ghasemalizadeh , Yuyin Sun , Cheng-Hao Kuo , Arnie Sen , Changhyun Choi This is my paper
classification cs.RO
keywords language-guidedgraspingreferringexpressionsegmentationspatialreasoningcross-modalfusionweaklysupervisedlearninggraspaffordancepredictionobjectgroundingrobotmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a compact vision-language model can both find the object a person means and compute a grasp for it, even when several identical objects sit on a table and the person refers to one by attributes and spatial relations. It proposes OGRG, built around a bidirectional aligner that repeatedly exchanges information between visual/depth features and language features, plus an FCN decoder. On the OCID-VLG benchmark, OGRG reports 95.60 mIoU grounding and 90.81 J@1 grasping, ahead of the compared baselines. In the weakly supervised setting, using only single-pixel grasp labels, it reports 94.02 average oIoU in simulation and a 70.8% real-robot grasp success rate. If correct, this shows that spatial-language robot grasping need not wait for huge multimodal models or dense grasp annotations.

What carries the argument

The load-bearing mechanism is the bidirectional aligner (Bi-Aligner): at each of four Swin Transformer stages, two cross-attention blocks let visual-depth features read language features and language features read visual-depth features, then update both branches through learnable gates and residual additions. Depth features enter only at the first stage via element-wise addition. For weak supervision, a Mask-Conditioned Grasping Network takes the predicted grounding mask together with RGB and depth, and outputs grasp-affordance maps over six discretized rotation angles, trained with a motion loss on single-pixel labels.

What would settle it

Run the trained model on the same tabletop setup but with human-typed, non-template instructions (for example, 'the tall one behind the red thing') over 100 scenes with duplicate objects; if grounding accuracy or grasp success falls to near baseline levels, the claimed open-form and spatial-reasoning advantage is refuted. A second check: move the camera to a novel viewpoint without retraining and measure grasp success.

Watch

Extended reading notes

Core claim

The central claim is that bi-directional, hierarchical fusion of vision and language—with depth injected at the first stage—lets a single 240M-parameter model resolve attribute and spatial referring expressions in scenes with duplicate object instances and turn the grounded mask into grasp poses. In the fully supervised RGS setting, the same model predicts a segmentation mask plus grasp quality, angle, and gripper-width maps, achieving the reported grounding and grasping improvements over CROG, ETRG, HiFi-CS, and LA VT. In the weakly supervised RGA setting, OGRG supplies the grounding mask to a Mask-Conditioned Grasping Network, which learns grasp affordances from sparse single-pixel labels

Load-bearing premise

The central claim rests on assuming that template-generated language instructions and simulator scenes with a fixed top-down camera and parallel-jaw gripper are good enough proxies for real open-form human language and unconstrained tabletop robot use.

Editorial extensions

If this is right

  • A 240M-parameter model can run at 17.59 FPS on a single RTX 2080 Ti, making closed-loop or multi-object sequential grasping plausible without large multimodal models.
  • Single-pixel grasp annotations plus a motion loss can replace dense grasp labels for learning affordances, substantially lowering data-collection cost.
  • Depth fusion measurably improves both grounding and grasp prediction, and the ablation claims it is especially helpful for relative spatial reasoning.
  • Because the grounding and grasping heads are task-specific, the same aligned vision-language-depth features can support different grasping decoders, which the paper calls embodiment-agnostic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is free-form human instructions that do not follow the paper's templates; the simulator language was template-generated, so the 'open-form' claim is only partially validated as stated.
  • The bidirectional aligner could transfer to other referring tasks beyond grasping, such as language-conditioned rearrangement or video object grounding, since the fusion mechanism is not grasp-specific.
  • The single-pixel weak-supervision loss may extend to other robotic affordances, such as pushing, pouring, or tool use, wherever a single success/failure point can be labeled.
  • The fixed top-down camera and parallel-jaw gripper assumptions likely bound the real-world transfer; a test with varied viewpoints and grippers would show how much of the spatial-reasoning advantage survives.
  • If depth is cheaply available, injecting it at only one stage may be a general recipe for grounding models with strong geometric cues, a hypothesis the paper does not directly explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents OGRG, a compact Swin-B/BERT framework for language-guided object grounding and grasp detection. It introduces a bidirectional cross-modal aligner (Bi-Aligner) and depth fusion at the first encoder stage, and evaluates two settings: RGS (fully supervised, predicting segmentation masks and grasp rectangles on OCID-VLG) and RGA (weakly supervised, predicting grounding masks plus a separate Mask-conditioned Grasping Network for grasp affordances). The authors report state-of-the-art results on OCID-VLG for RGS, and on a newly collected simulated RGA dataset plus a small real-robot experiment. The central claim is that OGRG outperforms strong baselines in both grounding and grasping, including a 4.86-point simulation grasp-success improvement over ETRG and a real-robot grasp-success improvement of 70.8% vs 62.5%.

Significance. If the RGS results are reliable, the paper demonstrates a meaningful step toward a compact (≈240M parameter) alternative to MLLMs for language-conditioned tabletop grasping, with fast inference (17.59 FPS on an RTX 2080 Ti) and no dependence on pre-aligned vision–language models. The RGA contribution is also valuable as a weak-supervision formulation that reduces dense annotation requirements, and the authors collect and share a substantial simulated dataset. The strengths are the breadth of the experimental evaluation, the inclusion of a real-robot test, and the careful ablations of depth and aligner variants. However, the real-robot statistical evidence is too weak to support the headline claim, and the ablation isolating the Bi-Aligner is inconsistent across tasks.

major comments (4)
  1. [§IV.B, Table IV] The real-robot comparison is based on 24 attempts per method. OGRG achieves 17/24 (70.8%) grasp success versus ETRG 15/24 (62.5%); a two-proportion test (e.g., Fisher's exact test) gives a p-value around 0.5, far above any conventional threshold. The grounding accuracy (21/24 vs 18/24) is likewise not significant. Without confidence intervals, repeated trials, or pre-registered protocols, the abstract's claim that OGRG 'surpasses baseline grasp-success rates in ... real-robot trials' is not supported. This is load-bearing because the real-robot result is a headline empirical claim. Please report CIs, increase trial counts, or temper the claim.
  2. [§IV.B, Tables II and I] The ablation isolating the Bi-Aligner over the unidirectional LA VT aligner is inconsistent. In RGS (Table I), OGRG-nodepth (Bi-Aligner, no depth) improves over LA VT by 2.35 mIoU and 0.94 J@1. In RGA (Table II), OGRG-nodepth obtains 92.73 avg oIoU versus LA VT's 92.76—slightly worse. Since both variants use no depth, the claimed advantage of the bi-directional fusion module is not robust across the two tasks. The paper should explain the discrepancy or soften the claim that the Bi-Aligner itself improves both grounding and grasping.
  3. [§IV.B] The RGA baseline comparisons are not sufficiently reproducible. ETRG is said to be 'modified to predict grounding masks with minor adjustments to its architecture,' and LA VT is used without depth; however, the exact modifications, training settings, or adaptation details are not given. Without this information, the reader cannot verify that the comparisons are fair, which is critical because the RGA object-grounding results (Table II) are central to the weak-supervision contribution. Please provide implementation details or code.
  4. [§III-F, §IV.B, and §V] The RGA experiments use template-generated language instructions ('Language instructions were generated from multiple templates') and a fixed top-down camera; the Conclusion candidly acknowledges scope limits (fixed camera, parallel-jaw gripper, uncluttered tabletops). The abstract's unqualified statement that OGRG 'interprets open-form language expressions and performs spatial reasoning' is therefore only fully supported for the RGS setting on OCID-VLG, not for the RGA real-robot setting. Please qualify the open-form claim to the RGS benchmark or demonstrate free-form language in RGA.
minor comments (5)
  1. [Table I] HiFi-CS is listed with mIoU but no J@1 or J@Any values. Please add the grasp metrics or explicitly state why they are unavailable.
  2. [Table III] The column headers are garbled ('AVG4-obj 7-obj 7-obj'). Please reorganize the table so that object counts and conditions (Seen/Unseen BG, Absolute/Relative) are clear.
  3. [§III-D] The choice V_i = f_i_v for RGS and V_i = f_i_back_v for RGA is described as 'from an empirical result.' Please report the supporting ablation or validation curve for this architecture choice.
  4. [§IV.B] The sentence '246 grasping data points were generated after applying data augmentation [6]' is unclear. What is augmented, and how are training/test splits maintained after augmentation?
  5. [Abstract and §V] The abstract's 'surpasses baseline grasp-success rates in both simulation and real-robot trials' should be tempered until statistical support is provided. Similarly, the conclusion's 'embodiment-agnostic and can be transferred to humanoid tabletop manipulation tasks' is speculative and should be reframed as future work.

Circularity Check

0 steps flagged · score 1.0 of 10

No definitional or fitted-input circularity; central claims rest on held-out data and ablations. Minor same-group citations are not load-bearing.

full rationale

I traced the derivation chain for both claimed predictions. For RGS, the model is trained on OCID-VLG and evaluated on the same dataset's held-out test split against external baselines (CROG, ETRG, HiFi-CS, LA VT); the Bi-Aligner and depth contribution are isolated by the OGRG-nodepth ablation (Table I). The grounding and grasp metrics are computed from ground-truth masks and grasp rectangles, not from any quantity the model was fit to reproduce. For RGA, the simulator test scenes (Tables II-III) and the real-robot trials (Table IV) are separately collected, with OGRG-nodepth and OGRG-db as controlled ablations; the reported numbers are measured success rates, not algebraic consequences of the loss or the architecture. The only potential concerns are empirical rather than circular: the head-input choice in Sec. III.D ('From an empirical result, we use V_i = {f^i_v} in RGS, and V_i = {f^i_back v} in RGA') is a post-hoc design decision, not a fitted parameter renamed as a prediction; ETRG [17] and Attribute-Grasp [6] are prior works with overlapping authorship, but they are used as baselines/technical components, not as the justification for OGRG's superiority. The real-robot comparison (17/24 vs 15/24) is statistically fragile, but fragility is an evidence-strength issue, not circularity. No equation in the paper defines the output in terms of the target metric, and no self-citation is invoked to force the method's choice. Hence no circular step rises to the level of the enumerated patterns.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claims rest on standard deep learning components plus several hand choices: template-generated training language, depth injection only at stage one, per-task head input selection, fixed gripper width, and small real-robot test sets. These are not free constants in a physical theory, but they are load-bearing empirical choices that limit generalization and make the reported margins fragile.

free parameters (6)
  • Input resolution 416x416
    Hand-picked for all RGS baselines; affects receptive field, runtime, and thus the reported metrics.
  • Maximum sentence length 20 (RGS), 25 (RGA)
    Hand-set token truncation; could discard relevant language in longer open-form expressions.
  • Rotation discretization N=6 at 30 degree intervals
    RGA affordance maps depend on this hand-chosen discretization; finer or coarser bins change the achievable grasp poses.
  • Fixed gripper width l* in RGA
    All RGA grasp attempts use a predefined width instead of predicting width, simplifying data collection but limiting grasp generality.
  • Training hyperparameters (epochs 26/50, batch sizes 4/12/32, learning rate 5e-5)
    Chosen without a reported search; differences in tuning could affect baseline comparisons.
  • Per-task head input choice V_i = f_i_v (RGS), V_i = f_i_back_v (RGA)
    Selected based on 'an empirical result' in Section III.D; this post-hoc choice may inflate reported performance relative to a fixed architecture.
assumptions (5)
  • standard math Transformer cross-attention and FCN decoding as described are valid for aligning Swin and BERT features and producing dense predictions.
    Invoked throughout Section III; relies on standard attention operations and pretrained backbones without formal proof in the paper.
  • domain assumption OCID-VLG ground truth and the new simulator RGA dataset are representative of real tabletop language-guided grasping.
    RGS claims rely on OCID-VLG; RGA claims rely on self-collected template data in Section III.F and IV.B.
  • domain assumption Depth image is aligned with RGB and provides useful geometric signal when added to visual features only at the first fusion stage.
    Section III.C fuses depth only at stage one; the paper assumes this is sufficient, though depth could be needed at every scale.
  • domain assumption Argmax on predicted affordance maps yields an executable grasp with the fixed gripper width.
    Section III.A defines the RGA pose as the argmax over A(x,y,theta); the entire grasp success measure depends on this choice.
  • ad hoc to paper Depth fusion only at stage one and per-task head inputs V_i are justified by the paper's ablations.
    Section III.D states 'From an empirical result' without a derivation; these choices are tuned to maximize reported scores.
invented entities (2)
  • Bi-Aligner
    purpose: Bidirectional cross-attention fusion of visual-depth and language features at four stages.
    New module introduced in Section III.C; the only evidence is this paper's own experiments, with no external benchmark or released code.
  • Mask-conditioned Grasping Network (MGN)
    purpose: Predicts grasp affordance maps conditioned on the OGRG grounding mask for weakly supervised RGA.
    Introduced in Section III.E; evaluated only inside this pipeline and not released as a standalone artifact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attribute-based Object Grounding and Robot Grasp Detection with Spatial Reasoning." pith.science (2026). https://pith.science/paper/OLYOYXS4

@misc{pith2026250908126,
  author       = {Pith},
  title        = {Pith review of: Attribute-based Object Grounding and Robot Grasp Detection with Spatial Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OLYOYXS4}},
  note         = {Machine review of arXiv:2509.08126}
}
read the original abstract

Enabling robots to grasp objects specified through natural language is essential for effective human-robot interaction, yet it remains a significant challenge. Existing approaches often struggle with open-form language expressions and typically assume unambiguous target objects without duplicates. Moreover, they frequently rely on costly, dense pixel-wise annotations for both object grounding and grasp configuration. We present Attribute-based Object Grounding and Robotic Grasping (OGRG), a novel framework that interprets open-form language expressions and performs spatial reasoning to ground target objects and predict planar grasp poses, even in scenes containing duplicated object instances. We investigate OGRG in two settings: (1) Referring Grasp Synthesis (RGS) under pixel-wise full supervision, and (2) Referring Grasp Affordance (RGA) using weakly supervised learning with only single-pixel grasp annotations. Key contributions include a bi-directional vision-language fusion module and the integration of depth information to enhance geometric reasoning, improving both grounding and grasping performance. Experiment results show that OGRG outperforms strong baselines in tabletop scenes with diverse spatial language instructions. In RGS, it operates at 17.59 FPS on a single NVIDIA RTX 2080 Ti GPU, enabling potential use in closed-loop or multi-object sequential grasping, while delivering superior grounding and grasp prediction accuracy compared to all the baselines considered. Under the weakly supervised RGA setting, OGRG also surpasses baseline grasp-success rates in both simulation and real-robot trials, underscoring the effectiveness of its spatial reasoning design. Project page: https://z.umn.edu/ogrg

Figures

Figures reproduced from arXiv: 2509.08126 by the authors.

Figure 1
Figure 1. Object Grounding and Robot Grasping (OGRG) model with open-form expressions for spatial reasoning. The model is designed to solve the attribute-based grounding and grasp detection task. The RGS subtask aims at predicting grasp rectangles with pixel-wise full supervision. The RGA subtask focuses on predicting grasp affordances with weak grasping supervision. tic attribute descriptions (e.g., “red apple”), while typic… view at source ↗
Figure 2
Figure 2. OGRG Architecture. OGRG processes open-form language expressions, visual images, and depth maps as inputs to generate task predictions. The bidirectional aligner (Bi-Aligner) fuses the multimodal features extracted from Swin Transformer [29] at different stages and the BERT language model [33]. The updated multimodal features f i back v and f i back l are fed back into their corresponding visual and linguistic branc… view at source ↗
Figure 3
Figure 3. Mask-conditioned Grasping Network (MGN). Condi￾tioned on the object grounding mask M predicted from OGRG, the MGN network uses a fully convolutional encoder-decoder ar￾chitecture for pixel-level grasp affordance prediction with different rotation angles. process is formulated as:  Y4 = V4, Yi = Conv([Up(Yi+1); Vi ]), i = 3, 2, 1, (5) where Conv(·) denotes a 3 × 3 convolution layer followed by batch normalization an… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Target objects used in both simulation and the real world [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Grounding masks and grasp affordances with spatial reasoning in simulation. The green bounding boxes highlight the correct language-referred target object. The first column shows the input scene. The second column shows the affordance predictions from ETRG [17]. The th…
Figure 7
Figure 7. Figure 7: Visualization of grounding masks and grasp affordance for challenging real robot spatial reasoning. The green bounding boxes highlight the correct language-referred target object. The visualization in the first column shows the original scene arrange￾ment and the groun…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 25 canonical work pages

  1. [1]

    Adversarial object rearrangement in constrained environments with heterogeneous graph neural networks,

    X. Lou, H. Yu, R. Worobel, Y . Yang, and C. Choi, “Adversarial object rearrangement in constrained environments with heterogeneous graph neural networks,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 1008– 1015

    work page 2023

  2. [2]

    Self-supervised interactive object segmentation through a singulation-and-grasping approach,

    H. Yu and C. Choi, “Self-supervised interactive object segmentation through a singulation-and-grasping approach,” inEuropean Confer- ence on Computer Vision. Springer, 2022, pp. 621–637

    work page 2022

  3. [3]

    Iosg: Image-driven object searching and grasping,

    H. Yu, X. Lou, Y . Yang, and C. Choi, “Iosg: Image-driven object searching and grasping,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 3145– 3152

    work page 2023

  4. [4]

    Antipodal robotic grasping using generative residual convolutional neural network,

    S. Kumra, S. Joshi, and F. Sahin, “Antipodal robotic grasping using generative residual convolutional neural network,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 9626–9633

    work page 2020

  5. [5]

    Language-driven grasp detection,

    A. D. Vuong, M. N. Vu, B. Huang, N. Nguyen, H. Le, T. V o, and A. Nguyen, “Language-driven grasp detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 17 902–17 912

    work page 2024

  6. [6]

    Attribute-based robotic grasping with data-efficient adaptation,

    Y . Yang, H. Yu, X. Lou, Y . Liu, and C. Choi, “Attribute-based robotic grasping with data-efficient adaptation,”IEEE Transactions on Robotics, 2024

    work page 2024

  7. [7]

    A joint modeling of vision-language-action for target- oriented grasping in clutter,

    K. Xu, S. Zhao, Z. Zhou, Z. Li, H. Pi, Y . Zhu, Y . Wang, and R. Xiong, “A joint modeling of vision-language-action for target- oriented grasping in clutter,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 11 597–11 604

    work page 2023

  8. [8]

    Visually grounding language instruction for history-dependent manipulation,

    H. Ahn, O. Kwon, K. Kim, J. Jeong, H. Jun, H. Lee, D. Lee, and S. Oh, “Visually grounding language instruction for history-dependent manipulation,” in2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022, pp. 675–682

    work page 2022

Show all 39 references
  1. [9]

    Reasoning grasping via multi- modal large language model,

    S. Jin, J. Xu, Y . Lei, and L. Zhang, “Reasoning grasping via multi- modal large language model,”arXiv preprint arXiv:2402.06798, 2024

    2024 arXiv

  2. [10]

    Open- vla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi,et al., “Open- vla: An open-source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  3. [11]

    Glover: Generaliz- able open-vocabulary affordance reasoning for task-oriented grasping,

    T. Ma, Z. Wang, J. Zhou, M. Wang, and J. Liang, “Glover: Generaliz- able open-vocabulary affordance reasoning for task-oriented grasping,” arXiv preprint arXiv:2411.12286, 2024

    2024 arXiv

  4. [12]

    Robotic control via embodied chain-of-thought reasoning,

    M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine, “Robotic control via embodied chain-of-thought reasoning,”arXiv preprint arXiv:2407.08693, 2024

    2024 arXiv

  5. [13]

    Towards open-world grasping with large vision-language models,

    G. Tziafas and H. Kasaei, “Towards open-world grasping with large vision-language models,”8th Conference on Robot Learning (CoRL 2024), 2024

    2024

  6. [14]

    Thinkgrasp: A vision-language system for strategic part grasping in clutter,

    Y . Qian, X. Zhu, O. Biza, S. Jiang, L. Zhao, H. Huang, Y . Qi, and R. Platt, “Thinkgrasp: A vision-language system for strategic part grasping in clutter,”arXiv preprint arXiv:2407.11298, 2024

    2024

  7. [15]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  8. [16]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,” inInternational conference on machine learning. PMLR, 2022, pp. 12 888–12 900

    2022

  9. [17]

    A parameter- efficient tuning framework for language-guided object grounding and robot grasping,

    H. Yu, M. Li, A. Rezazadeh, Y . Yang, and C. Choi, “A parameter- efficient tuning framework for language-guided object grounding and robot grasping,”arXiv preprint arXiv:2409.19457, 2024

    2024 arXiv

  10. [18]

    Language-guided robot grasping: Clip-based referring grasp synthesis in clutter,

    G. Tziafas, X. Yucheng, A. Goel, M. Kasaei, Z. Li, and H. Kasaei, “Language-guided robot grasping: Clip-based referring grasp synthesis in clutter,” in7th Annual Conference on Robot Learning, 2023

    2023

  11. [19]

    Grounding language with visual affordances over unstructured data,

    O. Mees, J. Borja-Diaz, and W. Burgard, “Grounding language with visual affordances over unstructured data,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 11 576–11 582

    2023

  12. [20]

    Segmentation from natural language expressions,

    R. Hu, M. Rohrbach, and T. Darrell, “Segmentation from natural language expressions,” inComputer Vision–ECCV 2016: 14th Euro- pean Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14. Springer, 2016, pp. 108–124

    2016

  13. [21]

    Referring image segmentation via recurrent refinement networks,

    R. Li, K. Li, Y .-C. Kuo, M. Shu, X. Qi, X. Shen, and J. Jia, “Referring image segmentation via recurrent refinement networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 5745–5753

    2018

  14. [22]

    Recurrent multimodal interaction for referring image segmentation,

    C. Liu, Z. Lin, X. Shen, J. Yang, X. Lu, and A. Yuille, “Recurrent multimodal interaction for referring image segmentation,” inProceed- ings of the IEEE international conference on computer vision, 2017, pp. 1271–1280

    2017

  15. [23]

    Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation,

    Z. Xu, Z. Chen, Y . Zhang, Y . Song, X. Wan, and G. Li, “Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17 503–17 512

    2023

  16. [24]

    Barleria: An efficient tuning framework for referring image segmentation,

    Y . Wang, J. Li, X. Zhang, B. Shi, C. Li, W. Dai, H. Xiong, and Q. Tian, “Barleria: An efficient tuning framework for referring image segmentation,” inThe Twelfth International Conference on Learning Representations

  17. [25]

    Cris: Clip-driven referring image segmentation,

    Z. Wang, Y . Lu, Q. Li, X. Tao, Y . Guo, M. Gong, and T. Liu, “Cris: Clip-driven referring image segmentation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11 686–11 695

    2022

  18. [26]

    Lavt: Language-aware vision transformer for referring image segmentation,

    Z. Yang, J. Wang, Y . Tang, K. Chen, H. Zhao, and P. H. Torr, “Lavt: Language-aware vision transformer for referring image segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 155–18 165

    2022

  19. [27]

    Contrastive grouping with transformer for referring image segmentation,

    J. Tang, G. Zheng, C. Shi, and S. Yang, “Contrastive grouping with transformer for referring image segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23 570–23 580

    2023

  20. [28]

    Beyond one-to-one: Rethinking the referring image segmentation,

    Y . Hu, Q. Wang, W. Shao, E. Xie, Z. Li, J. Han, and P. Luo, “Beyond one-to-one: Rethinking the referring image segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4067–4077

    2023

  21. [29]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  22. [30]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, 2024

    2024

  23. [31]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi`ere, N. Goyal, E. Hambro, F. Azhar,et al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  24. [32]

    Cliport: What and where pathways for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Cliport: What and where pathways for robotic manipulation,” inConference on robot learning. PMLR, 2022, pp. 894–906

    2022

  25. [33]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,”arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  26. [34]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inCVPR, 2016, pp. 770–778

    2016

  27. [35]

    V-rep: A versatile and scalable robot simulation framework,

    E. Rohmer, S. P. N. Singh, and M. Freese, “V-rep: A versatile and scalable robot simulation framework,” in2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 1321–1326

    2013

  28. [36]

    The ycb object and model set: Towards common benchmarks for manipulation research,

    B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar, “The ycb object and model set: Towards common benchmarks for manipulation research,” in2015 international conference on ad- vanced robotics (ICAR). IEEE, 2015, pp. 510–517

    2015

  29. [37]

    Vl-grasp: a 6-dof interactive grasp policy for language-oriented objects in cluttered indoor scenes,

    Y . Lu, Y . Fan, B. Deng, F. Liu, Y . Li, and S. Wang, “Vl-grasp: a 6-dof interactive grasp policy for language-oriented objects in cluttered indoor scenes,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 976–983

    2023

  30. [38]

    Film: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “Film: Visual reasoning with a general conditioning layer,” inProceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018

    2018

  31. [39]

    Vima: General robot manipulation with multimodal prompts,

    Y . Jiang, A. Gupta, Z. Zhang, G. Wang, Y . Dou, Y . Chen, L. Fei- Fei, A. Anandkumar, Y . Zhu, and L. Fan, “Vima: General robot manipulation with multimodal prompts,” inFortieth International Conference on Machine Learning, 2023

    2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.