{"id":"4a3df179-79f7-4ddc-be4d-6323311c62f4","arxiv_id":"2508.21102","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"GENNAV predicts whether a navigable region exists and generates polygon masks for multiple stuff-type target regions, outperforming baselines on a new GRiN-Drive benchmark and in zero-shot real-vehicle tests.","lead":"GENNAV is a model that reads a natural language driving instruction and a front camera image, decides whether the requested destination exists, and if so draws a polygon around each reachable target region. The paper also introduces a new benchmark, GRiN-Drive, with no-target, single-target, and multi-target cases, and tests the model on real cars in five cities.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported superiority rests on a custom msIoU metric and post hoc P@K thresholds; it is untested under standard evaluation protocols.","rationale":"The paper proposes a meaningful generalization of RNR, a new benchmark, and a method with a plausible architecture. The ablation study and real-world experiments provide supporting evidence. However, the central numerical claim rests on the custom msIoU metric and on P@K thresholds selected post hoc (Appendix D). The paper's own admission that baselines outperform humans at K>=0.3, and the consequent decision to exclude those thresholds, is a circular justification: it removes precisely the regime where the proposed method might not dominate. The metric's TN=1 weighting means existence accuracy dominates the score, and GENNAV's advantage over TNRSM in accuracy is larger than its advantage in mask overlap. Without the full P@K curve and a standard aggregation protocol, the reported 'superior performance' is not established under a conventional, pre-registered evaluation. This is addressable in a revision by reporting the complete operating curve and releasing code and data, so the reader's CONDITIONAL verdict stands; the concern is load-bearing and should be explicitly tested. I also note that the regression head's fixed npt output is not fully reconciled with variable numbers of target polygons; a code release would clarify whether the architecture truly predicts an arbitrary number of masks, but I did not make this the primary attack.","tokens_in":20748,"tokens_out":13168,"duration_ms":124389,"concrete_test":"Regenerate Table 1 reporting P@K for K in {0.1, 0.2, 0.3, 0.5, 0.7, 0.9} and msIoU computed with both K=0.1 and K=0.5, plus a standard aggregation such as gIoU or cIoU for multi-polygon samples using an explicit polygon-matching rule. If GENNAV's margin over TNRSM shrinks below significance at K>=0.3 or under msIoU@0.5, the reported superiority is an artifact of the post hoc threshold choice rather than a robust performance gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"GENNAV's claimed superiority is measured on msIoU, a metric introduced in this paper, and on P@K reported only at K=0.1 and 0.2. Appendix D explicitly excludes thresholds above 0.3 because 'baseline methods already outperform human performance' there; this is a post hoc restriction that favors the proposed method. msIoU (Eq. 3-4) assigns full score (1.0) to every true-negative sample and gives partial credit k·IoU to low-IoU true positives. In the test set, 256/758 samples (33.8%) are negative (Table 6); a trivial always-no-target classifier already attains roughly 33.8% msIoU. Thus the 8.45-point msIoU advantage over TNRSM could reflect better existence classification rather than better mask generation. Indeed, P@0.1 (49.60 vs 44.05) and accuracy (75.41 vs 67.23) show the gap is partly driven by existence prediction. Because the primary metric is non-standard and thresholds are restricted on a post hoc basis, the paper's claim of 'superior performance on standard evaluation metrics' is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes GENNAV for the Generalized Referring Navigable Regions (GRNR) task, in which a model receives a front-camera image and a natural-language navigation instruction and must predict whether a target region exists and, if so, output zero or more polygon masks for stuff-type regions. The method combines an existence-aware polygon segmentation head (ExPo) with a landmark-distribution patchification module (LDPM) and a visual-linguistic spatial integration module (VLSiM) that fuses depth and road priors. The authors introduce a new benchmark, GRiN-Drive, covering single-, no-, and multi-target cases, and report experiments on this benchmark plus zero-shot real-world tests with four vehicles in five urban areas. The headline result is an msIoU of 46.35 on GRiN-Drive, outperforming the best baseline (TNRSM, 37.90), with faster inference than most pixel-based baselines.","tokens_in":21080,"tokens_out":8828,"duration_ms":79968,"significance":"If the empirical claims hold, GENNAV is a useful step: it handles absent and multiple stuff-type targets, produces compact polygon outputs at about 31 ms per sample, and the real-world deployment study is a strength. The paper also contains a detailed error taxonomy (Appendix F) and explicit discussion of limitations. However, the central superiority claim rests on a newly introduced metric (msIoU), a self-constructed benchmark, and a threshold range that is truncated post hoc; the evaluation therefore needs substantial strengthening before the contribution can be assessed against prior work.","major_comments":[{"comment":"The primary metric msIoU is new to this paper and its definition is incomplete: the threshold K is never given, and the averaging range 'k=1,...,1/K' is not operationalized. More importantly, Appendix D restricts P@K to K=0.1 and 0.2 and excludes K≥0.3 with the explicit justification that baseline methods already outperform human performance there. That is a post hoc restriction of the evaluation range to values where the proposed method wins, so the claim of 'superior performance on standard evaluation metrics' is not established. Because Eq. (4) gives a full score to every true-negative sample, an always-no-target predictor already obtains 256/758 ≈ 33.8% msIoU on the reported test set; the paper should report the mask-quality component separately (e.g., IoU averaged over positive samples only) and provide a sensitivity analysis over K and over the full P@K curve.","section":"§4.1, Eq. (3)–(4), Appendix D"},{"comment":"The ground truth of GRiN-Drive is the basis for all conclusions, but its reliability is unquantified. No-target samples are created by swapping instructions and filtering with GPT-4o followed by manual inspection; multi-target instructions are produced by MLLM template filling with noun phrases from Refer-KITTI-V2; and no inter-annotator agreement is reported for the 244 annotators (average 29.07 samples per annotator). If the generated instructions are systematically simpler or more template-like than natural user instructions, the reported gap between GENNAV and baselines may partly reflect dataset artifact. Please report inter-annotator agreement on mask placement and existence labels, describe the manual filtering criteria, and provide a human-written instruction subset or an analysis of instruction diversity.","section":"Appendix B"},{"comment":"The baselines are not evaluated under a common protocol. Pixel-based methods are fine-tuned on the benchmark and then also tested at 640×640 even though their backbones were fine-tuned at 224×224; MLLMs are evaluated zero-shot with bounding-box outputs (converted to masks), which is a different output representation from GENNAV's polygons; only one MLLM is given a polygon prompt. The 8.45-point msIoU improvement over TNRSM (224×224) therefore mixes differences in task formulation, output type, resolution, and training data. Please either train all comparable methods under the same output modality and data split, or clearly separate the 'trained on GRiN-Drive' results from the 'zero-shot' results and discuss the consequences for the comparison.","section":"Table 1, §4.1"},{"comment":"The confusion matrix in Table 6 (TP=359, FN=143, FP=81, TN=175) yields accuracy (359+175)/758 = 70.45%, which contradicts the reported Acc. of 75.41% in Table 1. This discrepancy must be resolved; as written, the quantitative claim is not reproducible from the paper's own appendix.","section":"Appendix E.1, Table 6"},{"comment":"In the real-world experiments, the navigation instructions themselves are generated by GPT-4o conditioned on Grounding DINO detections, and no-target instructions are generated by swapping. Because the test distribution is produced by the same type of model used to construct the synthetic benchmark, the zero-shot claim is weakened: the model may be tuned to the distribution of LLM-generated instructions. Please report the raw prompts, state how many unique instructions were used, and include a human-authored instruction set (even a small one) to validate the zero-shot conclusion.","section":"§5, Appendix G"}],"minor_comments":[{"comment":"The Hadamard product between hinst and spatial visual features requires a channel and spatial alignment that is not stated; please specify the shapes of the visual, depth, and road feature tensors.","section":"Eq. (1)"},{"comment":"'Depth Anything V2 [47]' should cite reference [45] rather than [47] (which is DINOv2); Figure 2 also lists 'Depth Anything' without a version.","section":"§3.2"},{"comment":"'An navigation instruction' should be 'A navigation instruction'.","section":"§3.1"},{"comment":"The notation 'mean(1/N ∑ ...)' is redundant, and the relationship between the threshold K and the averaging index k is unclear; define K explicitly.","section":"Eq. (3)"},{"comment":"The baseline name 'GSV A-Vicuna-7B' has a spacing artifact; use GSVA consistently.","section":"Table 1"},{"comment":"'Statistical power greater than 0.999' is reported without the assumed effect size or the statistical test used; please specify.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the combination of a self-constructed benchmark, a self-proposed metric, and a post hoc threshold restriction; even if each is defensible, the revised paper should present standard metrics and full threshold curves. The confusion-matrix/accuracy discrepancy in Appendix E.1 should be checked and corrected before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read it. GENNAV genuinely generalizes referring navigable regions to no-target and multi-target stuff-type cases, which is a practical gap in the existing single-target-only RNR literature. And it does something most papers in this area skip: a zero-shot real-world test with four cars in five urban areas, plus an honest error analysis and a confusion matrix. The central question is not whether the task is worth doing—it is—but whether the performance claim survives a fairer evaluation setup.\n\nThe task formulation is the strongest contribution. Single-target-only navigation commands are an artificial restriction, and the paper makes a credible case that absent targets and multiple targets are the realistic cases. The LDPM landmark-density patchification is a reasonable idea, and putting an existence head on a polygon decoder is a sensible engineering move. The real-world evaluation is another genuine plus; not many papers validate in moving vehicles with phone cameras on snow-covered roads.\n\nNow the soft spots, in proportion. The stress-test note is partly right: msIoU is a new metric that gives full credit to true negatives, so an always-no-target classifier gets roughly a third of the score. That said, GENNAV also leads on P@0.1 and accuracy, so the margin is not purely a metric artifact. The bigger red flag is the post hoc threshold restriction in Appendix D: thresholds above 0.3 are excluded because the baselines outperform human performance. That justification should have been made in advance, or the paper should show the ranking is stable across K. The GRiN-Drive benchmark is also self-constructed with LLM-generated instructions and no inter-annotator agreement, which means the test set could encode systematic simplicity. The real-world experiment helps, but those instructions were also GPT-generated, so it is partially the same concern.\n\nNone of this is fatal. The paper would be materially stronger with code and data release, an unreported IoU metric without the custom normalization, and a robustness check on P@K. As it stands, it is an honest engineering contribution with a meaningful task extension and a plausible but not airtight superiority claim.\n\nFor people working on language-conditioned driving or referring segmentation, this is a useful anchor. I would send it to peer review, and I would expect a conditional accept after revision addressing the metric transparency and benchmark reliability. It deserves referee time, not a desk reject.","headline":"GENNAV's real task extension deserves a serious referee, but the custom metric and post hoc threshold choices mean the reported margin is not fully established.","tokens_in":21552,"tokens_out":2127,"would_cite":true,"duration_ms":23888,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An automated driving system can now answer navigation instructions with zero, one, or several region masks, outperforming prior pixel-based and multimodal baselines on the new GRiN-Drive benchmark and in zero-shot road tests.","keywords":["referring navigable regions","existence prediction","polygon segmentation","vision-and-language navigation","autonomous driving","stuff-type segmentation","GRiN-Drive benchmark","zero-shot transfer"],"falsifier":"Take a fresh test set of the same Talk2Car-RegSeg and Refer-KITTI-V2 images annotated by humans with their own natural navigation instructions, without GPT-4o filtering or template substitution, and with per-sample agreement recorded; if GENNAV's msIoU margin over TNRSM shrinks toward zero, or if human labels disagree as much as the models differ, the benchmark itself, not the model, would be carrying the reported gap.","tokens_in":20529,"feed_emoji":"🚗","tokens_out":5531,"duration_ms":49376,"temperature":0.7,"pith_summary":"GENNAV claims that the hardest part of following a natural-language navigation instruction from a moving vehicle is deciding whether the described target exists at all and then drawing a region for each instance, not simply locating one object. The paper proposes an architecture that predicts a three-way existence label — no target, single target, or multiple targets — and simultaneously emits polygon vertex sequences for any number of target regions, so the network can output zero masks when the requested landmark is absent. On the newly built GRiN-Drive benchmark, it reports the best mean stuff IoU (46.35 percent) against pixel-based and multimodal-LLM baselines, with accuracy of 75.41 percent and inference at 31.31 ms per sample. The same model, applied without fine-tuning to four automobiles in five urban areas, also beats the baselines, which matters if such systems are to be deployed where instructions can be invalid or refer to several places at once.","feed_headline":"GENNAV finds absent, single, or multiple navigable targets in one pass","feed_subtitle":"Outperforms pixel-based and MLLM baselines on the new GRiN-Drive benchmark at 31 ms per sample.","key_machinery":"The argument is carried by three modules inside one network. Existence Aware Polygon Segmentation Module (ExPo) is the load-bearing one: a classification head predicts whether the instruction names zero, one, or several target regions, and a regression head predicts polygon vertices $\\hat{c}_i$ for every region, trained by cross-entropy plus an L1 vertex loss that is masked out for no-target samples. Landmark Distribution Patchification Module (LDPM) cuts the high-resolution image into patches whose layout follows where landmarks concentrate in training data and encodes each patch, which gives distant small landmarks enough pixels. Visual-Linguistic Spatial Integration Module (VLSiM) multiplies the language embedding with visual and pseudo-depth features and with a road-region feature, so mask predictions are pulled toward the road surface and away from sky and occluded objects. The polygon decoder, rather than a pixel classifier, is what makes arbitrary many masks cheap: GENNAV runs at 31.31 ms per sample.","core_discovery":"On its own terms, the paper's central discovery is that explicit existence prediction can be attached to polygon-based segmentation without sacrificing efficiency, and that this combination is what lets a navigation-grounding model handle the full range of real instructions: targets that are absent, targets that occur once, and targets that occur several times in one image. GENNAV's Existence Aware Polygon Segmentation Module takes a fused multimodal representation plus the road feature and outputs both a class distribution over no/single/multi target and, when targets exist, clockwise-ordered polygon vertices for each region. On GRiN-Drive, built by adding no-target and multi-target samples to Talk2Car-RegSeg and Refer-KITTI-V2 data, GENNAV reaches msIoU 46.35 against the strongest baseline TNRSM at 37.90, with the gains statistically significant at p < 0.05, and it transfers zero-shot to real-world driving video with msIoU 34.32. The paper also introduces msIoU to stop existence-only 'no target' predictions from inflating scores.","pith_inferences":["If GRiN-Drive's artificially swapped and template-generated instructions are systematically simpler than human speech, the benchmark margin may overstate real-world gains; a test set of naturally spoken instructions would reveal the gap.","The same existence head could be grafted onto other polygon-based referring-expression segmentation models, since ExPo is decoupled from the specific visual encoder choices.","Because LDPM's patch layout is learned from the training landmark distribution, it may degrade on environments with a different spatial layout, suggesting an adaptive patch assignment as a testable extension.","Adding temporal context, which the paper lists as a limitation, could reduce frame-to-frame mask inconsistency and is a natural next experiment."],"forward_implications":["A vehicle can safely answer an instruction like 'park by the red car' with no mask when no red car is visible, instead of inventing a destination.","Multiple valid destinations in one scene can be produced in the same forward pass, which pixel-wise single-region baselines cannot do.","The 31.31 ms per-sample speed puts polygon-based grounding in range for real-time use in an onboard system.","Stuff-type regions with soft boundaries, such as road space next to a landmark, are tractable for this architecture, not just countable objects.","Zero-shot results across four cars and five urban areas suggest the model does not need per-city retraining to transfer."],"supporting_citations":[{"why":"Supplies the Talk2Car-RegSeg base of images, instructions, and ground-truth masks, and serves as a pixel-based baseline.","marker":"[1]"},{"why":"TNRSM is the strongest RNR baseline that GENNAV must beat, and it motivates the stuff-region pixel approach being superseded.","marker":"[2]"},{"why":"GRES defines the generalized referring-expression-segmentation setting that covers absent and multiple targets and supplies the existence-aware framing.","marker":"[4]"},{"why":"GSVA serves as a pixel-based multimodal-LLM baseline for generalized segmentation.","marker":"[5]"},{"why":"PolyFormer provides the sequential polygon-generation formulation that the ExPo regression head extends with existence prediction.","marker":"[7]"},{"why":"Talk2Car provides the landmark distribution and instruction-image pairs used for LDPM patch layout and benchmark construction.","marker":"[14]"},{"why":"Refer-KITTI-V2 supplies frames with multiple visible landmarks, the source for the multi-target samples in GRiN-Drive.","marker":"[16]"},{"why":"Depth Anything V2 generates the pseudo-depth image overlaid with the input in VLSiM.","marker":"[45]"},{"why":"DINOv2 supplies the frozen visual features used by both the image branch and the depth-overlay branch.","marker":"[47]"},{"why":"PIDNet provides the road-region mask used to keep generated polygons on drivable surfaces.","marker":"[48]"}],"fun_headline_variants":["One-pass GENNAV segments absent, single, or multiple navigable regions","Existence-aware polygon masks beat baselines on no/single/multi target navigation","GENNAV: polygon masks with existence prediction handle any target count","Zero-shot real-world driving: GENNAV wins on multi-target referring segmentation","New benchmark GRiN-Drive, new model GENNAV: multiple stuff-targets segmented"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that GRiN-Drive's ground truth is a fair picture of how people actually give instructions and mark navigable regions, even though no-target samples were created by swapping instructions and filtering with GPT-4o and multi-target samples were made by filling MLLM-generated templates with noun phrases.","fun_headline_variants_meta":{"raw":{"variants":["One-pass GENNAV segments absent, single, or multiple navigable regions","Existence-aware polygon masks beat baselines on no/single/multi target navigation","GENNAV: polygon masks with existence prediction handle any target count","Zero-shot real-world driving: GENNAV wins on multi-target referring segmentation","New benchmark GRiN-Drive, new model GENNAV: multiple stuff-targets segmented"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00069,"raw_usage":{"total_tokens":3126,"prompt_tokens":948,"completion_tokens":2178,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":2074}},"tokens_in":564,"tokens_out":2178,"duration_ms":14594,"temperature":1.0,"reasoning_tokens":2074,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:44:14.685615+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fresh test set of the same Talk2Car-RegSeg and Refer-KITTI-V2 images annotated by humans with their own natural navigation instructions, without GPT-4o filtering or template substitution, and with per-sample agreement recorded; if GENNAV's msIoU margin over TNRSM shrinks toward zero, or if human labels disagree as much as the models differ, the benchmark itself, not the model, would be carrying the reported gap.","supporting_citations":[{"cited_title":"Rufus, K","cited_arxiv_id":null,"evidence_quote":"Supplies the Talk2Car-RegSeg base of images, instructions, and ground-truth masks, and serves as a pixel-based baseline."},{"cited_title":"Hosomi, S","cited_arxiv_id":null,"evidence_quote":"TNRSM is the strongest RNR baseline that GENNAV must beat, and it motivates the stuff-region pixel approach being superseded."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GRES defines the generalized referring-expression-segmentation setting that covers absent and multiple targets and supplies the existence-aware framing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GSVA serves as a pixel-based multimodal-LLM baseline for generalized segmentation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PolyFormer provides the sequential polygon-generation formulation that the ExPo regression head extends with existence prediction."},{"cited_title":"Deruyttere, S","cited_arxiv_id":null,"evidence_quote":"Talk2Car provides the landmark distribution and instruction-image pairs used for LDPM patch layout and benchmark construction."},{"cited_title":"Oquab, T","cited_arxiv_id":null,"evidence_quote":"DINOv2 supplies the frozen visual features used by both the image branch and the depth-overlay branch."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PIDNet provides the road-region mask used to keep generated polygons on drivable surfaces."}],"review_version":2}