Pith. sign in

REVIEW 5 major objections 6 minor 59 references

GENNAV: Polygon Mask Generation for Generalized Referring Navigable Regions

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An automated driving system can now answer navigation instructions with zero, one, or several region masks, outperforming prior pixel-based and multimodal baselines on the new GRiN-Drive benchmark and in zero-shot road tests.

desk verdict GENNAV's real task extension deserves a serious referee, but the custom metric and post hoc threshold choices mean the reported margin is not fully established. read the letter →

arxiv 2508.21102 v1 pith:WSFADWZR submitted 2025-08-28 cs.CV cs.RO

Kei Katsumata , Yui Iioka , Naoki Hosomi , Teruhisa Misu , Kentaro Yamada , Komei Sugiura This is my paper · ORCID
classification cs.CVcs.RO
keywords referringnavigableregionsexistencepredictionpolygonsegmentationvision-and-languagenavigationautonomousdrivingstuff-typeGRiN-Drivebenchmarkzero-shottransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GENNAV claims that the hardest part of following a natural-language navigation instruction from a moving vehicle is deciding whether the described target exists at all and then drawing a region for each instance, not simply locating one object. The paper proposes an architecture that predicts a three-way existence label — no target, single target, or multiple targets — and simultaneously emits polygon vertex sequences for any number of target regions, so the network can output zero masks when the requested landmark is absent. On the newly built GRiN-Drive benchmark, it reports the best mean stuff IoU (46.35 percent) against pixel-based and multimodal-LLM baselines, with accuracy of 75.41 percent and inference at 31.31 ms per sample. The same model, applied without fine-tuning to four automobiles in five urban areas, also beats the baselines, which matters if such systems are to be deployed where instructions can be invalid or refer to several places at once.

What carries the argument

The argument is carried by three modules inside one network. Existence Aware Polygon Segmentation Module (ExPo) is the load-bearing one: a classification head predicts whether the instruction names zero, one, or several target regions, and a regression head predicts polygon vertices $\hat{c}_i$ for every region, trained by cross-entropy plus an L1 vertex loss that is masked out for no-target samples. Landmark Distribution Patchification Module (LDPM) cuts the high-resolution image into patches whose layout follows where landmarks concentrate in training data and encodes each patch, which gives distant small landmarks enough pixels. Visual-Linguistic Spatial Integration Module (VLSiM) multiplies the language embedding with visual and pseudo-depth features and with a road-region feature, so mask predictions are pulled toward the road surface and away from sky and occluded objects. The polygon decoder, rather than a pixel classifier, is what makes arbitrary many masks cheap: GENNAV runs at 31.31 ms per sample.

What would settle it

Take a fresh test set of the same Talk2Car-RegSeg and Refer-KITTI-V2 images annotated by humans with their own natural navigation instructions, without GPT-4o filtering or template substitution, and with per-sample agreement recorded; if GENNAV's msIoU margin over TNRSM shrinks toward zero, or if human labels disagree as much as the models differ, the benchmark itself, not the model, would be carrying the reported gap.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that explicit existence prediction can be attached to polygon-based segmentation without sacrificing efficiency, and that this combination is what lets a navigation-grounding model handle the full range of real instructions: targets that are absent, targets that occur once, and targets that occur several times in one image. GENNAV's Existence Aware Polygon Segmentation Module takes a fused multimodal representation plus the road feature and outputs both a class distribution over no/single/multi target and, when targets exist, clockwise-ordered polygon vertices for each region. On GRiN-Drive, built by adding no-target and multi-target samples to Talk2Car-RegSeg and Refer-KITTI-V2 data, GENNAV reaches msIoU 46.35 against the strongest baseline TNRSM at 37.90, with the gains statistically significant at p < 0.05, and it transfers zero-shot to real-world driving video with msIoU 34.32. The paper also introduces msIoU to stop existence-only 'no target' predictions from inflating scores.

Load-bearing premise

The load-bearing premise is that GRiN-Drive's ground truth is a fair picture of how people actually give instructions and mark navigable regions, even though no-target samples were created by swapping instructions and filtering with GPT-4o and multi-target samples were made by filling MLLM-generated templates with noun phrases.

Editorial extensions

If this is right

  • A vehicle can safely answer an instruction like 'park by the red car' with no mask when no red car is visible, instead of inventing a destination.
  • Multiple valid destinations in one scene can be produced in the same forward pass, which pixel-wise single-region baselines cannot do.
  • The 31.31 ms per-sample speed puts polygon-based grounding in range for real-time use in an onboard system.
  • Stuff-type regions with soft boundaries, such as road space next to a landmark, are tractable for this architecture, not just countable objects.
  • Zero-shot results across four cars and five urban areas suggest the model does not need per-city retraining to transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If GRiN-Drive's artificially swapped and template-generated instructions are systematically simpler than human speech, the benchmark margin may overstate real-world gains; a test set of naturally spoken instructions would reveal the gap.
  • The same existence head could be grafted onto other polygon-based referring-expression segmentation models, since ExPo is decoupled from the specific visual encoder choices.
  • Because LDPM's patch layout is learned from the training landmark distribution, it may degrade on environments with a different spatial layout, suggesting an adaptive patch assignment as a testable extension.
  • Adding temporal context, which the paper lists as a limitation, could reduce frame-to-frame mask inconsistency and is a natural next experiment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This manuscript proposes GENNAV for the Generalized Referring Navigable Regions (GRNR) task, in which a model receives a front-camera image and a natural-language navigation instruction and must predict whether a target region exists and, if so, output zero or more polygon masks for stuff-type regions. The method combines an existence-aware polygon segmentation head (ExPo) with a landmark-distribution patchification module (LDPM) and a visual-linguistic spatial integration module (VLSiM) that fuses depth and road priors. The authors introduce a new benchmark, GRiN-Drive, covering single-, no-, and multi-target cases, and report experiments on this benchmark plus zero-shot real-world tests with four vehicles in five urban areas. The headline result is an msIoU of 46.35 on GRiN-Drive, outperforming the best baseline (TNRSM, 37.90), with faster inference than most pixel-based baselines.

Significance. If the empirical claims hold, GENNAV is a useful step: it handles absent and multiple stuff-type targets, produces compact polygon outputs at about 31 ms per sample, and the real-world deployment study is a strength. The paper also contains a detailed error taxonomy (Appendix F) and explicit discussion of limitations. However, the central superiority claim rests on a newly introduced metric (msIoU), a self-constructed benchmark, and a threshold range that is truncated post hoc; the evaluation therefore needs substantial strengthening before the contribution can be assessed against prior work.

major comments (5)
  1. [§4.1, Eq. (3)–(4), Appendix D] The primary metric msIoU is new to this paper and its definition is incomplete: the threshold K is never given, and the averaging range 'k=1,...,1/K' is not operationalized. More importantly, Appendix D restricts P@K to K=0.1 and 0.2 and excludes K≥0.3 with the explicit justification that baseline methods already outperform human performance there. That is a post hoc restriction of the evaluation range to values where the proposed method wins, so the claim of 'superior performance on standard evaluation metrics' is not established. Because Eq. (4) gives a full score to every true-negative sample, an always-no-target predictor already obtains 256/758 ≈ 33.8% msIoU on the reported test set; the paper should report the mask-quality component separately (e.g., IoU averaged over positive samples only) and provide a sensitivity analysis over K and over the full P@K curve.
  2. [Appendix B] The ground truth of GRiN-Drive is the basis for all conclusions, but its reliability is unquantified. No-target samples are created by swapping instructions and filtering with GPT-4o followed by manual inspection; multi-target instructions are produced by MLLM template filling with noun phrases from Refer-KITTI-V2; and no inter-annotator agreement is reported for the 244 annotators (average 29.07 samples per annotator). If the generated instructions are systematically simpler or more template-like than natural user instructions, the reported gap between GENNAV and baselines may partly reflect dataset artifact. Please report inter-annotator agreement on mask placement and existence labels, describe the manual filtering criteria, and provide a human-written instruction subset or an analysis of instruction diversity.
  3. [Table 1, §4.1] The baselines are not evaluated under a common protocol. Pixel-based methods are fine-tuned on the benchmark and then also tested at 640×640 even though their backbones were fine-tuned at 224×224; MLLMs are evaluated zero-shot with bounding-box outputs (converted to masks), which is a different output representation from GENNAV's polygons; only one MLLM is given a polygon prompt. The 8.45-point msIoU improvement over TNRSM (224×224) therefore mixes differences in task formulation, output type, resolution, and training data. Please either train all comparable methods under the same output modality and data split, or clearly separate the 'trained on GRiN-Drive' results from the 'zero-shot' results and discuss the consequences for the comparison.
  4. [Appendix E.1, Table 6] The confusion matrix in Table 6 (TP=359, FN=143, FP=81, TN=175) yields accuracy (359+175)/758 = 70.45%, which contradicts the reported Acc. of 75.41% in Table 1. This discrepancy must be resolved; as written, the quantitative claim is not reproducible from the paper's own appendix.
  5. [§5, Appendix G] In the real-world experiments, the navigation instructions themselves are generated by GPT-4o conditioned on Grounding DINO detections, and no-target instructions are generated by swapping. Because the test distribution is produced by the same type of model used to construct the synthetic benchmark, the zero-shot claim is weakened: the model may be tuned to the distribution of LLM-generated instructions. Please report the raw prompts, state how many unique instructions were used, and include a human-authored instruction set (even a small one) to validate the zero-shot conclusion.
minor comments (6)
  1. [Eq. (1)] The Hadamard product between hinst and spatial visual features requires a channel and spatial alignment that is not stated; please specify the shapes of the visual, depth, and road feature tensors.
  2. [§3.2] 'Depth Anything V2 [47]' should cite reference [45] rather than [47] (which is DINOv2); Figure 2 also lists 'Depth Anything' without a version.
  3. [§3.1] 'An navigation instruction' should be 'A navigation instruction'.
  4. [Eq. (3)] The notation 'mean(1/N ∑ ...)' is redundant, and the relationship between the threshold K and the averaging index k is unclear; define K explicitly.
  5. [Table 1] The baseline name 'GSV A-Vicuna-7B' has a spacing artifact; use GSVA consistently.
  6. [§4.1] 'Statistical power greater than 0.999' is reported without the assumed effect size or the statistical test used; please specify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are empirical and evaluated on held-out data, not derived from self-citations or fitted inputs.

full rationale

GENNAV's contribution is an empirical system evaluated under a supervised train/validation/test protocol: Eq. (2) trains the existence head and polygon head on labeled examples, and the test-set numbers in Table 1 come from the model selected on validation, with five trials. The same-group citations (TNRSM [2], TRiP [28]) and the polygon RES line [6,7,20,21] are used as architectural inspiration or baselines, not as an unverified premise that forces GENNAV's claimed advantage; no 'uniqueness theorem' or fitted parameter is imported. The GRiN-Drive benchmark and msIoU metric are introduced by the authors, but the metric is not part of the loss and the benchmark labels are not produced by GENNAV, so evaluating on them is standard benchmark construction rather than a definitional identity between output and input. The paper candidly reports limitations and error categories in Section F. Concerns about the nonstandard msIoU, the post hoc choice of P@K thresholds in Appendix D, and LLM-generated instructions are evaluation-validity or generalization risks, not circularity: they do not make any reported prediction equal to the model's training input by construction. Thus no circular step can be quoted, and the honest finding is no significant circularity (score 0).

Assumptions & free parameters 3 free parameters · 4 assumptions · 3 invented entities

The central claim rests on the new benchmark, the new metric, and several domain assumptions about road segmentation, depth estimation, and the transferability of a fixed landmark-distribution prior. The trainable model parameters are fitted to the training set, as is normal for supervised learning; the entries above are the hand-chosen elements that are load-bearing for the evaluation and for the module ablations.

free parameters (3)
  • lambda_pt = 3
    Weight for the polygon vertex L1 loss in Eq. (2). Set without sensitivity analysis; changes the balance between existence classification and mask fidelity.
  • LDPM patch layout = not learned; derived from Talk2Car landmark distribution
    The Landmark Distribution Patchification Module fixes a spatial partition of 560x315 images based on landmark density in the Talk2Car training set. Ablation shows it is worth +3.3 msIoU, but the layout is a hand-selected prior that may transfer poorly.
  • msIoU threshold K = implicit (0.1 from P@0.1)
    Equation (3) averages sIoU over k=1..1/K; K sets the steepness of the mask-quality reward. The paper does not state K explicitly or test sensitivity, so metric values depend on an unexamined choice.
assumptions (4)
  • domain assumption Road mask from PIDNet correctly identifies navigable, non-sky, non-occluded regions.
    VLSiM multiplies visual features with f_road(x_img) to suppress inappropriate masks; if the road segmentation is poor (snow, rain, unusual layouts), the model's mask generation is degraded.
  • domain assumption Pseudo-depth from Depth Anything V2 provides spatial cues accurate enough to ground relations like 'near', 'left of', 'behind'.
    VLSiM overlays depth and fuses it with language features; the paper reports a 2.22 point drop without fdepth, but no independent validation of depth accuracy on the test domains is given.
  • domain assumption The landmark distribution in the Talk2Car training set is representative of other datasets and the five real-world cities.
    LDPM patch layout is fixed from Talk2Car; zero-shot real-world experiments assume this distribution transfers, which is not empirically validated.
  • domain assumption Human annotator agreement on target-region polygons is high enough to treat GRiN-Drive labels as ground truth.
    No inter-annotator agreement is reported for the 244 annotators; the human test on 11 subjects shows P@0.1 only 56%, indicating low label consistency.
invented entities (3)
  • GRiN-Drive benchmark
    purpose: Provides training and test data for the GRNR task with no/single/multi-target samples.
    Constructed by the authors from Talk2Car-RegSeg and Refer-KITTI-V2 with GPT-4o-assisted filtering and instruction generation; no external validation or release details are included in this preprint.
  • msIoU metric
    purpose: Replaces gIoU to reduce bias from trivial no-target predictions.
    A novel evaluation metric defined in Eq. (3); its threshold K and weighting are selected by the authors, and no external comparison of metric behavior is provided.
  • ExPo / LDPM / VLSiM modules
    purpose: Architectural components of GENNAV for existence prediction, landmark-distribution patchification, and visual-linguistic spatial integration.
    Validated only through the authors' ablations and benchmarks; no independent implementation demonstrates their necessity outside this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GENNAV: Polygon Mask Generation for Generalized Referring Navigable Regions." pith.science (2026). https://pith.science/paper/WSFADWZR

@misc{pith2026250821102,
  author       = {Pith},
  title        = {Pith review of: GENNAV: Polygon Mask Generation for Generalized Referring Navigable Regions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WSFADWZR}},
  note         = {Machine review of arXiv:2508.21102}
}
read the original abstract

We focus on the task of identifying the location of target regions from a natural language instruction and a front camera image captured by a mobility. This task is challenging because it requires both existence prediction and segmentation, particularly for stuff-type target regions with ambiguous boundaries. Existing methods often underperform in handling stuff-type target regions, in addition to absent or multiple targets. To overcome these limitations, we propose GENNAV, which predicts target existence and generates segmentation masks for multiple stuff-type target regions. To evaluate GENNAV, we constructed a novel benchmark called GRiN-Drive, which includes three distinct types of samples: no-target, single-target, and multi-target. GENNAV achieved superior performance over baseline methods on standard evaluation metrics. Furthermore, we conducted real-world experiments with four automobiles operated in five geographically distinct urban areas to validate its zero-shot transfer performance. In these experiments, GENNAV outperformed baseline methods and demonstrated its robustness across diverse real-world environments. The project page is available at https://gennav.vercel.app/.

Figures

Figures reproduced from arXiv: 2508.21102 by the authors.

Figure 1
Figure 1. Overview of GENNAV. The model predicts target regions from a natural language instruction and a front camera image captured by a moving mobility. mechanism for predicting no-target or multi-target responses, as we show in Section 4.2. By con￾trast, models that support multiple or no-target outputs [4, 5] are primarily designed for thing-type scenarios and do not account for the stuff-type target regions. To address … view at source ↗
Figure 2
Figure 2. Overall architecture of GENNAV. DA represents Depth Anything [ [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results of GENNAV and baseline methods on the GRiN-Drive benchmark. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Qualitative results of GENNAV in the real world ex [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Typical examples of the GRNR task. Left: single target. Center: multi-target. Right: no [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Annotation interface for multi-target samples in the GRiN-Drive benchmark. Annotators [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results of the proposed method and baseline methods on the GRiN-Drive [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Qualitative results of a failed case. Columns (a), (b), (c), and (d) show the predictions [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results of the proposed method in the GRiN-Drive benchmark. The green and [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Qualitative analysis of failure cases of the proposed method categorized as Reduced [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Categorization of failure modes. Er￾ror analysis results categorized into Cases (i) Ex￾istence Prediction and (ii) Polygon Generation. To investigate the limitations of GENNAV, we analyzed cases where the method did not per￾form as expected. In this study, failures ar…
Figure 12
Figure 12. Figure 12: Additional qualitative results of the proposed method in the real-world experiments. The [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 48 canonical work pages

  1. [1]

    Rufus, K

    N. Rufus, K. Jain, K. Nair, V . Gandhi, and M. Krishna. Grounding Linguistic Commands to Navigable Regions. In IROS, pages 8593–8600, 2021

    work page 2021

  2. [2]

    Hosomi, S

    N. Hosomi, S. Hatanaka, Y . Iioka, W. Yang, K. Kuyo, T. Misu, K. Yamada, and K. Sugiura. Trimodal Navigable Region Segmentation Model: Grounding Navigation Instructions in Urban Areas. IEEE RA-L, 9(5):4162–4169, 2024

    work page 2024

  3. [3]

    Kirillov, K

    A. Kirillov, K. He, R. Girshick, C. Rother, and P. Dollár. Panoptic Segmentation. In CVPR, pages 9404–9413, 2019

    work page 2019

  4. [4]

    C. Liu, H. Ding, and X. Jiang. GRES: Generalized Referring Expression Segmentation. In CVPR, pages 23592–23601, 2023

    work page 2023

  5. [5]

    Z. Xia, D. Han, Y . Han, X. Pan, S. Song, and G. Huang. GSV A: Generalized Segmentation via Multimodal Large Language Models. In CVPR, pages 3858–3869, 2024

    work page 2024

  6. [6]

    Nishimura, K

    T. Nishimura, K. Kuyo, M. Kambara, and K. Sugiura. Object Segmentation from Open- V ocabulary Manipulation Instructions Based on Optimal Transport Polygon Matching with Multimodal Foundation Models. In IROS, pages 9549–9556, 2024

    work page 2024

  7. [7]

    J. Liu, H. Ding, Z. Cai, Y . Zhang, R. Satzoda, V . Mahadevan, and R. Manmatha. PolyFormer: Referring Image Segmentation as Sequential Polygon Generation. In CVPR, pages 18653– 18663, 2023

    work page 2023

  8. [8]

    C. Cui, Y . Ma, X. Cao, W. Ye, Y . Zhou, K. Liang, J. Chen, J. Lu, Z. Yang, K.-D. Liao, et al. A Survey on Multimodal Large Language Models for Autonomous Driving. In WACV, pages 958–979, 2024

    work page 2024

Show all 59 references
  1. [9]

    M. Liu, E. Yurtsever, J. Fossaert, X. Zhou, W. Zimmer, Y . Cui, B. Zagar, and A. Knoll. A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook. IEEE T-IV, pages 1–29, 2024

    2024

  2. [10]

    Huang, B

    K. Huang, B. Shi, X. Li, X. Li, S. Huang, and Y . Li. Multi-modal Sensor Fusion for Auto Driving Perception: A Survey. arXiv preprint arXiv:2202.02703, 2022

    2022 arXiv

  3. [11]

    X. Zhou, M. Liu, E. Yurtsever, B. Zagar, W. Zimmer, H. Cao, and A. Knoll. Vision Language Models in Autonomous Driving: A Survey and Outlook. IEEE T-IV, pages 1–20, 2024

    2024

  4. [12]

    Y . Du, C. Lei, Z. Zhao, and F. Su. iKUN: Speak to Trackers without Retraining. In CVPR, pages 19135–19144, 2024

    2024

  5. [13]

    Nguyen, K

    P. Nguyen, K. Quach, K. Kitani, and K. Luu. Type-to-Track: Retrieve Any Object via Prompt- based Tracking. In NeurIPS, volume 36, pages 3205–3219, 2023

    2023

  6. [14]

    Deruyttere, S

    T. Deruyttere, S. Vandenhende, D. Grujicic, L. Gool, and M. Moens. Talk2Car: Taking Control of Your Self-Driving Car. In EMNLP, pages 2088–2098, 2019. 9

    2019

  7. [15]

    D. Wu, W. Han, T. Wang, X. Dong, X. Zhang, and J. Shen. Referring Multi-Object Tracking. In CVPR, pages 14633–14642, 2023

    2023

  8. [16]

    Zhang, D

    Y . Zhang, D. Wu, W. Han, and X. Dong. Bootstrapping Referring Multi-Object Tracking. arXiv preprint arXiv:2406.05039, 2024

    2024

  9. [17]

    Kamath, M

    A. Kamath, M. Singh, Y . LeCun, G. Synnaeve, I. Misra, and N. Carion. MDETR-Modulated Detection for End-to-end Multi-modal Understanding. In ICCV, pages 1780–1790, 2021

    2021

  10. [18]

    F. Zeng, B. Dong, Y . Zhang, T. Wang, X. Zhang, and Y . Wei. MOTR: End-to-End Multiple- Object Tracking with Transformer. In ECCV, pages 659–675, 2022

    2022

  11. [19]

    Zhang, C

    Y . Zhang, C. Wang, X. Wang, W. Zeng, and W. Liu. FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking. IJCV, 129:3069–3087, 2021

    2021

  12. [20]

    C. Zhu, Y . Zhou, Y . Shen, G. Luo, X. Pan, M. Lin, C. Chen, L. Cao, X. Sun, and R. Ji. SeqTR: A Simple yet Universal Network for Visual Grounding. In ECCV, pages 598–615, 2022

    2022

  13. [21]

    Cheng, K

    Z. Cheng, K. Li, P. Jin, S. Li, X. Ji, L. Yuan, C. Liu, and J. Chen. Parallel Vertex Diffusion for Unified Visual Grounding. In AAAI, pages 1326–1334, 2024

    2024

  14. [22]

    Z. Yang, J. Wang, Y . Tang, K. Chen, H. Zhao, and P. H. Torr. LA VT: Language-Aware Vision Transformer for Referring Image Segmentation. In CVPR, pages 18155–18165, 2022

    2022

  15. [23]

    X. Lai, Z. Tian, Y . Chen, Y . Li, Y . Yuan, S. Liu, and J. Jia. LISA: Reasoning Segmentation via Large Language Model. In CVPR, pages 9579–9589, 2024

    2024

  16. [24]

    Cheng, S

    M. Cheng, S. Zheng, W. Lin, V . Vineet, P. Sturgess, N. Crook, N. Mitra, and P. Torr. Image- Spirit: Verbal Guided Image Parsing. ACM Trans. Graph., 34(1), 2015

    2015

  17. [25]

    Y . Hu, Q. Wang, W. Shao, E. Xie, Z. Li, J. Han, and P. Luo. Beyond One-to-One: Rethinking the Referring Image Segmentation. In ICCV, pages 4067–4077, 2023

    2023

  18. [26]

    Z. Luo, Y . Wu, T. Cheng, Y . Liu, Y . Xiao, H. Wang, X. Zhang, and Y . Yang. CoHD: A Counting-Aware Hierarchical Decoding Framework for Generalized Referring Expression Segmentation. arXiv preprint arXiv:2405.15658, 2024

    2024 arXiv

  19. [27]

    W. Li, Z. Zhao, H. Bai, and F. Su. Bring Adaptive Binding Prototypes to Generalized Referring Expression Segmentation. arXiv preprint arXiv:2405.15169, 2024

    2024 arXiv

  20. [28]

    Hosomi, Y

    N. Hosomi, Y . Iioka, S. Hatanaka, T. Misu, K. Yamada, N. Tsukamoto, S. Kobayashi, and K. Sugiura. Multimodal Target Localization With Landmark-Aware Positioning for Urban Mobility. IEEE RA-L, 10(1):716–723, 2025

    2025

  21. [29]

    Anderson, Q

    P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. Sünderhauf, I. Reid, S. Gould, and A. Hengel. Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation In- structions in Real Environments. In CVPR, pages 3674–3683, 2018

    2018

  22. [30]

    F. Zhu, X. Liang, Y . Zhu, Q. Yu, X. Chang, and X. Liang. SOON: Scenario Oriented Object Navigation with Graph-Based Exploration. In CVPR, pages 12689–12699, 2021

    2021

  23. [31]

    Y . Qi, Q. Wu, P. Anderson, X. Wang, Y . Wang, C. Shen, and A. Hengel. REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments. In CVPR, pages 9982– 9991, 2020

    2020

  24. [32]

    Zhang, K

    J. Zhang, K. Wang, R. Xu, G. Zhou, Y . Hong, X. Fang, Q. Wu, Z. Zhang, and H. Wang. NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation. In RSS, 2024

    2024

  25. [33]

    Krantz, E

    J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee. Beyond the Nav-Graph: Vision- and-Language Navigation in Continuous Environments. In ECCV, pages 104–120, 2020. 10

    2020

  26. [34]

    Deruyttere, D

    T. Deruyttere, D. Grujicic, M. B. Blaschko, and M.-F. Moens. Talk2car: Predicting Physical Trajectories for Natural Language Commands. IEEE Access, 10:123809–123834, 2022

    2022

  27. [35]

    Omama, P

    M. Omama, P. Inani, P. Paul, and et al. ALT-Pilot: Autonomous navigation with Language augmented Topometric maps. arXiv preprint arXiv:2310.02324, 2023

    2023 arXiv

  28. [36]

    D. Shah, B. Osi ´nski, B. Ichter, and S. Levine. LM-Nav: Robotic Navigation with Large Pre- Trained Models of Language, Vision, and Action. In CoRL, pages 492–504, 2022

    2022

  29. [37]

    K. Jain, V . Chhangani, A. Tiwari, K. M. Krishna, and V . Gandhi. Ground then Navigate: Language-guided Navigation in Dynamic Scenes. In ICRA, pages 4113–4120, 2023

    2023

  30. [38]

    Xiang, X

    J. Xiang, X. Wang, and Y . Wang. Learning to Stop: A Simple yet Effective Approach to Urban Vision-Language Navigation. In EMNLP, pages 699–707, 2020

    2020

  31. [39]

    Schumann and S

    R. Schumann and S. Riezler. Analyzing Generalization of Vision and Language Navigation to Unseen Outdoor Areas. In ACL, pages 7519–7532, 2022

    2022

  32. [40]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR, pages 12888–12900, 2021

    2021

  33. [41]

    F. Yu, H. Chen, X. Wang, W. Xian, Y . Chen, F. Liu, V . Madhavan, and T. Darrell. BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning. In CVPR, pages 2636–2645, 2020

    2020

  34. [42]

    Cordts, M

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The Cityscapes Dataset for Semantic Urban Scene Understanding. In CVPR, pages 3213–3223, 2016

    2016

  35. [43]

    Geiger, P

    A. Geiger, P. Lenz, and R. Urtasun. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In CVPR, pages 3354–3361, 2012

    2012

  36. [44]

    Caesar, V

    H. Caesar, V . Bankiti, A. Lang, S. V ora, V . Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom. nuScenes: A Multimodal Dataset for Autonomous Driving. In CVPR, pages 11621–11631, 2020

    2020

  37. [45]

    L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao. Depth Anything V2. arXiv:2406.09414, 2024

    2024 arXiv

  38. [46]

    Solatorio

    A. Solatorio. GISTEmbed: Guided In-sample Selection of Training Negatives for Text Em- bedding Fine-tuning. arXiv preprint arXiv:2402.16829, 2024

    2024 arXiv

  39. [47]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, R. Howes, Y . Huang, H. Xu, V . Sharma, S.-W. Li, W. Galuba, et al. DINOv2: Learning Robust Visual Features without Supervision, 2023

    2023

  40. [48]

    J. Xu, Z. Xiong, and S. Bhattacharyya. PIDNet: A Real-Time Semantic Segmentation Network Inspired by PID Controllers. In CVPR, pages 19529–19539, 2023

    2023

  41. [49]

    M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, et al. Gemini 1.5: Un- locking Multimodal Understanding Across Millions of Tokens of Context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  42. [50]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, et al. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  43. [51]

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y . Fan, K. Dang, M. Du, X. Ren, R. Men, et al. Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution. arXiv preprint arXiv:2409.12191, 2024. 11

    2024 arXiv

  44. [52]

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, Z. Zeng, et al. Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks. arXiv preprint arXiv:2401.14159, 2024

    2024 arXiv

  45. [53]

    X. Zhou, R. Girdhar, A. Joulin, P. Krähenbühl, and I. Misra. Detecting Twenty-thousand Classes using Image-level Supervision. In ECCV, pages 350–368, 2022

    2022

  46. [54]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu, et al. Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection. arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  47. [55]

    H. Yuan, X. Li, C. Zhou, Y . Li, K. Chen, and C. Loy. Open-V ocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively. In ECCV, pages 419–437, 2024

    2024

  48. [56]

    J. Mao, J. Huang, A. Toshev, O. Camburu, A. Yuille, and K. Murphy. Generation and Com- prehension of Unambiguous Object Descriptions. In CVPR, pages 11–20, 2016

    2016

  49. [57]

    Fiedler, M

    N. Fiedler, M. Bestmann, and N. Hendrich. ImageTagger: An Open Source Online Platform for Collaborative Image Labeling. In RoboCup, pages 162–169, 2019

    2019

  50. [58]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. InICCV, pages 10012–10022, 2021

    2021

  51. [59]

    Pull up the left side of the right bicycle,

    A. Kumar, T. Kashiyama, H. Maeda, and Y . Sekimoto. Citywide Reconstruction of Cross- Sectional Traffic Flow from Moving Camera Videos. In Big Data, pages 1670–1678, 2021. 12 Appendix A Task Details Figure 5: Typical examples of the GRNR task. Left: single target. Center: mult...

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.