Pith. sign in

REVIEW 4 major objections 7 minor 42 references

Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that 3D visual grounding in autonomous driving improves when the language prompt itself decides which sensor's physical evidence to read, and it supplies the first camera-LiDAR-radar benchmark and a routing model to show…

desk verdict The tri-sensor benchmark and the routing idea are solid; the Mono3DRefer 'transfer' claim overreaches because TSFormer gets LiDAR while baselines stay monocular. read the letter →

arxiv 2608.04568 v1 pith:NPNXHI47 submitted 2026-08-05 cs.CV

Runwei Guan , Di Tian , Ningwei Ouyang , Ruixiao Zhang , Shaofeng Liang , Haocheng Zhao , Lianqing Zheng , Xiaokai Bai
show 4 more authors
Guotao Wang Daizong Liu Henghui Ding Hui Xiong
This is my paper
classification cs.CV
keywords 3Dvisualgroundingautonomousdrivingmulti-sensorfusionmultimodallearning4DradarLiDARreferringexpressioncomprehensionlanguage-drivenperception
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that language-guided 3D visual grounding in autonomous driving should be driven by the physical property a prompt asks about, and that each property is best read from the sensor that actually measures it: texture from camera, geometry from LiDAR, velocity and motion from 4D radar. To make this concrete, it introduces Talk2Sensors, a dataset of 8,682 natural-language instructions and 20,558 referred objects over synchronized camera, LiDAR, and radar views, with prompts deliberately written to name measurable physical cues rather than subjective impressions. It then proposes TSFormer, a Transformer pipeline whose Language-Routed Property Sampler biases deformable feature sampling by query-level text similarity, and whose Sparse-Preserving Modality Arbiter gates each modality with a text-conditioned weight so dense camera features cannot swamp sparse radar signals. On its own benchmark, TSFormer reports 51.000 EAA mAP, 8.05 points above the strongest fusion baseline, and it transfers to the monocular Mono3DRefer benchmark at 53.05% Acc@0.5.

What carries the argument

The load-bearing machinery is the pair of language-conditioned modules inside TSFormer. The Language-Routed Property Sampler (LRPS) projects the global text embedding into each sensor's feature space, computes a query-level text-sensor similarity $G_m=\sigma(\tau_m D_m)$, and broadcasts this modulation factor into the deformable sampling attention weights before softmax, so sampling is biased toward the regions a prompt cares about without any calibration-dependent geometric projection. The Sparse-Preserving Modality Arbiter (SPMA) then predicts a text-conditioned gate $\boldsymbol{\alpha}$ over available modalities, optionally with Top-$K$ sparsification, fuses the sampled features by weighted summation, and refines them with a text-guided multi-head cross-attention layer. Together the two modules implement the paper's central mechanism: the prompt, not a fixed fusion rule, decides which sensor's evidence is read at fine scale.

What would settle it

Take a random sample of Talk2Sensors scenes, give independent annotators only the sensor visualizations and the 3D boxes (not the original prompts or the paper's property taxonomy), and measure their agreement on the referred object and on which sensor the prompt's attributes require; if agreement on the sensor-category assignment is low, the property-aware evaluation and the routing model's advantage rest on an annotation convention rather than on objectively grounded cues.

Watch

Extended reading notes

Core claim

The central claim is that a tri-sensor grounding system localizes referred objects more accurately precisely because the sensors measure different physical quantities, provided the model routes each query toward the sensor that carries the asked-for attribute instead of fusing everything indiscriminately. The paper supports this with three empirical results. First, a text-free version of TSFormer collapses to 0.007 mAP, showing the task is genuinely referential rather than plain detection. Second, naive fusion of all three sensors stays near its LiDAR-only level, while TSFormer improves monotonically as sensors are added (46.18 to 48.28 to 51.00 EAA mAP), evidencing that language-guided routing extracts complementary value from each modality. Third, the largest gains concentrate on radar-involved and tri-sensor prompts, exactly the cases where a single modality cannot answer and correct property routing is decisive.

Load-bearing premise

The benchmark's value rests on the assumption that the annotation pipeline truly makes every prompt objectively grounded in what the sensors measure, since no inter-annotator agreement or independent verification is reported.

Editorial extensions

If this is right

  • A grounding model with language-routed sensor selection beats fixed fusion by 8.05 mAP on Talk2Sensors, so query-adaptive routing is the reason multi-sensor input helps.
  • Adding sensors only helps when fusion is language-conditioned: TSFormer improves monotonically as camera, LiDAR, and radar are added, whereas the naive fusion baseline does not.
  • The same architecture transfers without task-specific tuning to the monocular Mono3DRefer benchmark, reaching 53.05% Acc@0.5 and beating the specialized monocular model at the strict IoU threshold.
  • TSFormer degrades more gracefully than the fusion baseline under camera or radar dropout, suggesting the soft modality gate acts as a fallback when a sensor is lost at runtime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inferred: the routing hypothesis is testable independently of this dataset. A prompt set that decouples property words (color, velocity, distance) from object identity would show whether text-conditioned gating, rather than the benchmark's annotation style, explains the reported gains.
  • Inferred: if property-aware annotation is what carries the result, then re-annotating existing monocular or LiDAR-only benchmarks with the same physical-cue discipline should produce larger gains for routing models than for fixed-fusion models.
  • Inferred: the paper's own dropout results suggest an ordering of sensor priorities the authors do not state: LiDAR is the geometric backbone, with camera and radar as routed assistants, so a practical deployment would keep LiDAR always on and treat the other two as query-driven supplements.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces Talk2Sensors, a camera-LiDAR-4D-radar 3D visual grounding benchmark built on View-of-Delft with 8,682 prompts and 20,558 referred objects, and proposes TSFormer, a query-based framework combining a Language-Routed Property Sampler and a Sparse-Preserving Modality Arbiter for text-conditioned sensor routing. Experiments on Talk2Sensors report state-of-the-art mAP (51.000 EAA and 66.499 DCA in the full tri-sensor setting, +8.05 over FUTR3D) together with ablations, robustness, and efficiency analyses, plus a cross-dataset result on Mono3DRefer (53.05 Overall Acc@0.5). The main issue is that the Mono3DRefer evaluation gives TSFormer KITTI LiDAR while baselines remain monocular, so the cross-dataset claim as stated is unsupported.

Significance. The Talk2Sensors benchmark is a potentially useful contribution: it is the first tri-sensor dataset of this kind, with a documented closed-loop annotation pipeline and a text-free control that collapses to near-zero mAP, supporting that the task is genuinely referential. The TSFormer results and ablations are internally consistent: gains concentrate on radar-involved prompts, sensor additions improve mAP monotonically, and inference-time modality dropout degrades gracefully. If the Mono3DRefer comparison is corrected and the dataset split is published, the benchmark and model could be valuable for multi-sensor 3D visual grounding. The current evidence does not, however, support the abstract's claim of transfer to a monocular benchmark, and the absence of a specified split and auditable baselines tempers the headline SOTA claim.

major comments (4)
  1. [V.A, Table IV] The Mono3DRefer comparison is not apples-to-apples. The text in V.A states that "we evaluate all baselines under the standard monocular setting, while further supplementing the corresponding KITTI LiDAR" for TSFormer; Mono3DRefer, however, is defined as a monocular-RGB benchmark. Giving TSFormer synchronized LiDAR changes the task, so the reported 53.05% Overall Acc@0.5 cannot support the abstract's claim that TSFormer "transfers to the monocular Mono3DRefer benchmark," and the margin over Mono3DVG-TR (44.25) and TPCNet (50.67) is not evidence for the property-aware design. Please run a camera-only TSFormer variant on Mono3DRefer and either use it for the headline comparison or explicitly relabel the LiDAR-augmented result as a different protocol.
  2. [III, V.A] No train/validation/test split is specified for Talk2Sensors. The paper reports aggregate statistics and then evaluates models in V.C, but it never states which scenes, tracks, or object instances are used for training versus testing. Without a fixed split, the mAP comparisons in Table III and the 8.05-point gain over FUTR3D are not reproducible. Please publish the exact split (e.g., VoD track IDs or scene indices) and confirm that no test scenes are used for training; report key results over at least three random seeds with mean and standard deviation.
  3. [III-A] The dataset's central validity claim is that every prompt is objectively grounded in sensor measurements, but no inter-annotator agreement is reported. The triple-expert review is described qualitatively; there are no numbers on agreement among the three experts, rejection rates, or corrections after re-annotation. Since the prompts are constructed by the same group that designs the routing model, the absence of a quantitative verification protocol leaves the "objective prompt" claim and the property-aware evaluation vulnerable to co-design bias. Please report agreement statistics, first-pass rejection rates, and examples of rejected or corrected samples.
  4. [V.B] The baseline reimplementations cannot be audited. The paper states that "all baselines are re-implemented under the same backbone, query setting, and training schedule for fairness," but no code, configuration files, or checkpoints are provided, and the project link is a placeholder ("available at here"). Without this material, claims of state-of-the-art over reimplemented baselines are not verifiable. Please release code and detailed configs, or include a supplementary section with complete hyperparameters and an exact description of the data split used for each baseline.
minor comments (7)
  1. [Table IV] In the TSFormer row, Multiple Acc@0.25 and Acc@0.5 are both reported as 47.62; this duplication is likely a typo and should be checked.
  2. [Table III, Fig. 3] The symbol R5 is used for the radar modality without a definition; clarify whether R5 denotes the 4D radar sensor and use a consistent notation throughout.
  3. [IV.D] A duplicated paragraph describing the sine-cosine yaw encoding appears twice in Section IV.D; remove one occurrence.
  4. [IV.B, Eqs. (15)-(17)] The text promises residual connections and layer normalization in the arbiter, but Equations (15)-(17) do not make the residual paths explicit; please clarify whether the residual is around the MHA, the FFN, or both.
  5. [V.B] Implementation details omit the loss-balancing constants lambda_L1, lambda_IoU, and lambda_aux, as well as the learning-rate schedule; list these values for reproducibility.
  6. [Table IV] It is unclear how TPCNet, described in Section II as a LiDAR+radar fusion method, is adapted to the monocular Mono3DRefer benchmark; please specify its input protocol.
  7. [Figures 9-12] The qualitative figures contain typos in displayed prompts (e.g., "pedestirans", "motorbike") and some described visual cues such as yellow arrows are hard to see; please proofread and increase figure legibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TSFormer's empirical gains are supported by ablated, self-contained experiments; the Mono3DRefer LiDAR-augmentation and annotation co-design are validity concerns, not circular derivations.

full rationale

The paper's load-bearing result is empirical: TSFormer is trained with a grounding loss (Eqs. 23-27) and evaluated on Talk2Sensors and Mono3DRefer. No fitted constant is renamed as a prediction, and no output equation is defined in terms of the quantity it claims to predict. The main ablation chain (Tables V-IX) isolates LRPS and SPMA against standard samplers and fusion blocks under fixed settings, and the text-free variant collapses to 0.007 mAP, showing the metric is referential. The self-citations (Talk2Radar [9], TPCNet [13], MMDrive [14]) are context and baselines, not load-bearing assumptions. Two weaknesses are noted but are not circular: the Talk2Sensors annotation protocol (Section III-A) has prompts written by the same group to be sensor-aligned, which introduces a possible co-design bias; and the Mono3DRefer comparison (Section V-A, Table IV) gives TSFormer KITTI LiDAR while baselines stay monocular, confounding the 'monocular transfer' claim. These are dataset-validity and experimental-fairness concerns, not definitional reductions, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central results depend on the View-of-Delft sensor synchronization, the annotation and review pipeline, the CLIP text encoder's ability to encode physical quantities, and several hand-chosen design constants such as K=2, inference filters, and query count. The method is an empirical architecture, and no new natural-kind entity is introduced beyond learned model components.

free parameters (4)
  • Learnable text-sensor temperature tau_m = learned, initialized to 1
    Eq. 3; modulates LRPS sampling weights. Ablation with fixed temperature drops EAA mAP from 51.000 to 50.128.
  • Top-K sparsity K in SPMA = K=2
    Table IX; K=2 gives 51.000 mAP versus 49.216 for K=1 and 50.628 for soft gating only.
  • Inference filtering thresholds = confidence 0.1, circle-NMS IoU 0.2, top 300 boxes
    Section V-B; these post-processing choices directly affect the reported mAP.
  • Decoder and training setup = 900 queries, 6 decoder layers, dim 256, FFN 1024, 8 heads, 80 epochs, batch size 8
    Section V-B; the chosen architecture and training schedule are used in all comparisons and ablations.
assumptions (4)
  • domain assumption View-of-Delft provides synchronized and calibrated camera, LiDAR, and 4D radar frames.
    Section III-A.1; all dataset samples come from this external source.
  • domain assumption The triplet expert review ensures grammatical, factual, and sensor-consistent prompts.
    Section III-A.4; no quantitative inter-annotator agreement is provided, so label quality is assumed.
  • domain assumption CLIP text embeddings encode physical attributes such as distance, velocity, and direction well enough for routing.
    Section V-B; the architecture routes based on CLIP text features, so this is load-bearing.
  • ad hoc to paper Adding KITTI LiDAR to Mono3DRefer is a fair way to evaluate cross-dataset generalization against monocular baselines.
    Section V-A; this assumption is questionable because TSFormer receives an extra modality that the monocular baselines do not receive.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching." pith.science (2026). https://pith.science/paper/NPNXHI47

@misc{pith2026260804568,
  author       = {Pith},
  title        = {Pith review of: Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NPNXHI47}},
  note         = {Machine review of arXiv:2608.04568}
}
read the original abstract

As a key capability for embodied intelligence, 3D visual grounding (3DVG) has been predominantly studied in indoor scenes with RGB-D or point-cloud inputs, while existing outdoor extensions largely rely on monocular images alone. Both settings fall short of real-world outdoor perception, where heterogeneous sensors capture complementary yet distinct physical properties, such as visual texture, 3D geometry, and object kinematics, that are indispensable for flexible and robust query-adaptive grounding but remain under-exploited. To bridge this gap, we introduce Talk2Sensors, the first multi-sensor 3D visual grounding dataset built upon camera, LiDAR, and 4D radar. It contains 8,682 language instructions and 20,558 referred objects, with diverse prompts explicitly aligned with sensor-specific physical cues. Furthermore, we propose TSFormer, a unified Transformer-based framework for language-guided 3D visual grounding in autonomous driving. TSFormer adopts a coarse-to-fine property-aware fusion strategy: the Language-Routed Property Sampler first performs coarse text-conditioned feature retrieval by modulating sensor sampling weights with query-level linguistic cues, while the subsequent Sparse-Preserving Modality Arbiter module conducts fine-grained modality arbitration and text-guided refinement to determine the precise referred spatial location. This design enables dynamic routing of appearance, geometry, and motion cues according to the semantic requirements of each prompt, preventing dense modalities from overwhelming sparse but critical sensor signals. Extensive experiments demonstrate that TSFormer achieves state-of-the-art performance across multiple benchmarks: it improves over the strongest baseline by 8.05 mAP on Talk2Sensors, and transfers to the monocular Mono3DRefer benchmark with 53.05\% Acc@0.5.

Figures

Figures reproduced from arXiv: 2608.04568 by the authors.

Figure 1
Figure 1. The overview of Talk2Sensors pipeline, regarding textual prompt-guided [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The annotation process of our proposed Talk2Sensors dataset. Following a pipeline of quantification-annotation-verification, implicit sensor attributes are [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Statistical overview of the Talk2Sensors dataset. (a) Proportion of queries relying on different sensor configurations. (b) Distribution of prompt lengths, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Spatial distribution (BEV) of referred objects in Talk2Sensors. The [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The pipeline of TSFormer for multi-sensor 3D visual grounding. The framework comprises a Language-Routed Property Sampler for semantic-driven [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The structure of Language-Routed Property Sampler (LRPS). By inject [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: The structure of the Sparse-Preserving Modality Arbiter. A text [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: The comparison regarding efficiency, robustness and sensor combination [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Qualitative grounding results of TSFormer under progressively enriched sensor configurations (Camera [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Heatmap visualization by various sensor configurations. For certain [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Sampling response of the proposed Language-Routed Property Sampler [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison with the specialized monocular Mono3DVG-TR [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 34 canonical work pages

  1. [1]

    A survey on text-guided 3-d visual grounding: Elements, recent advances, and future directions,

    D. Liu, Y. Liu, W. Huang, and W. Hu, “A survey on text-guided 3-d visual grounding: Elements, recent advances, and future directions,”IEEE Transactions on Neural Networks and Learning Systems, 2025

    work page 2025

  2. [2]

    Mtrag: Multi-target referring and grounding via hybrid semantic-spatial integration,

    Y. Ren, J. Du, X. Liu, Q. Su, Y. Deng, and H. Li, “Mtrag: Multi-target referring and grounding via hybrid semantic-spatial integration,”IEEE Transactions on Image Processing, 2026

    work page 2026

  3. [3]

    Visual grounding in 2d and 3d: A unified perspective and survey,

    K. Guo, Y. Huang, T. Jia, X. Song, S. Sun, H. Wei, X.-F. Han, S. Huang, N. Strisciuglio, and S. Li, “Visual grounding in 2d and 3d: A unified perspective and survey,”Information Fusion, p. 103625, 2025

    work page 2025

  4. [4]

    Embodied intelligence: A synergy of morphology, action, perception and learning,

    H. Liu, D. Guo, and A. Cangelosi, “Embodied intelligence: A synergy of morphology, action, perception and learning,”ACM Computing Surveys, vol. 57, no. 7, pp. 1–36, 2025

    2025

  5. [5]

    Scanrefer: 3d object localiza- tion in rgb-d scans using natural language,

    D. Z. Chen, A. X. Chang, and M. Nießner, “Scanrefer: 3d object localiza- tion in rgb-d scans using natural language,” inEuropean conference on computer vision. Springer, 2020, pp. 202–221

    work page 2020

  6. [6]

    Refer-it-in- rgbd: A bottom-up approach for 3d visual grounding in rgbd images,

    H. Liu, A. Lin, X. Han, L. Yang, Y. Yu, and S. Cui, “Refer-it-in- rgbd: A bottom-up approach for 3d visual grounding in rgbd images,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 6032–6041

    work page 2021

  7. [7]

    Referit3d: Neural listeners for fine-grained 3d object identification in real- world scenes,

    P. Achlioptas, A. Abdelreheem, F. Xia, M. Elhoseiny, and L. Guibas, “Referit3d: Neural listeners for fine-grained 3d object identification in real- world scenes,” inEuropean conference on computer vision. Springer, 2020, pp. 422–440

    work page 2020

  8. [8]

    Talk2car: Taking control of your self-driving car,

    T. Deruyttere, S. Vandenhende, D. Grujicic, L. Van Gool, and M. F. Moens, “Talk2car: Taking control of your self-driving car,” inProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language process- ing (EMNLP-IJCNLP), 2019, pp. 2088–2098

    work page 2019

Show all 42 references
  1. [9]

    Talk2radar: Bridging natural language with 4d mmwave radar for 3d referring expression comprehension,

    R. Guan, R. Zhang, N. Ouyang, J. Liu, K. L. Man, X. Cai, M. Xu, J. Smith, E. G. Lim, Y. Yueet al., “Talk2radar: Bridging natural language with 4d mmwave radar for 3d referring expression comprehension,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IE...

    2025

  2. [10]

    Mono3dvg: 3d visual grounding in monocular images,

    Y. Zhan, Y. Yuan, and Z. Xiong, “Mono3dvg: 3d visual grounding in monocular images,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 7, 2024, pp. 6988–6996

    2024

  3. [11]

    Enhanced vision- language models for diverse sensor understanding: Cost-efficient opti- mization and benchmarking,

    S. Chung, Y. Yu, S. Kim, Y. Chee, and Y. M. Ro, “Enhanced vision- language models for diverse sensor understanding: Cost-efficient opti- mization and benchmarking,”IEEE Transactions on Image Processing, 2026

    2026

  4. [12]

    Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding,

    R. Li, S. Li, L. Kong, X. Yang, and J. Liang, “Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding,” in2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025, pp. 3707–3717

    2025

  5. [13]

    Enhance 3d visual grounding through lidar and radar point clouds fusion for autonomous driving,

    R. Guan, J. Liu, N. Ouyang, S. Liang, D. Liu, X. Sun, L. Zheng, M. Xu, Y. Yue, G. Maoet al., “Enhance 3d visual grounding through lidar and radar point clouds fusion for autonomous driving,”IEEE Transactions on Intelligent Transportation Systems, 2026

    2026

  6. [14]

    Mmdrive: Interactive scene understanding beyond vision with multi-representational fusion,

    M. Hou, W.-H. Huang, S. Liang, D. Liu, T.-H. Wen, G. Wang, R. Guan, and W. Ding, “Mmdrive: Interactive scene understanding beyond vision with multi-representational fusion,”Information Fusion, p. 104314, 2026

    2026

  7. [15]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,

    Z. Liu, H. Tang, A. Amini, X. Yang, H. Mao, D. L. Rus, and S. Han, “Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,” in2023 IEEE international conference on robotics and automation (ICRA). IEEE, 2023, pp. 2774–2781

    2023

  8. [16]

    Futr3d: A unified sensor fusion framework for 3d detection,

    X. Chen, T. Zhang, Y. Wang, Y. Wang, and H. Zhao, “Futr3d: A unified sensor fusion framework for 3d detection,” inproceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 172–181

    2023

  9. [17]

    Gated multimodal units for information fusion,

    J. Arevalo, T. Solorio, M. Montes-y G ´omez, and F. A. Gonz ´alez, “Gated multimodal units for information fusion,” inProceedings of the International Conference on Learning Representations Workshop, 2017

    2017

  10. [18]

    Squeeze-and-excitation networks,

    J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132–7141

    2018

  11. [19]

    Talk2Radar: Bridging natural language with 4d mmwave radar for 3d referring expression comprehension,

    R. Guan, R. Zhang, N. Ouyang, J. Liu, K. L. Man, X. Cai, M. Xu, J. Smith, E. G. Lim, Y. Yueet al., “Talk2Radar: Bridging natural language with 4d mmwave radar for 3d referring expression comprehension,”IEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

  12. [20]

    Detr3d: 3d object detection from multi-view images via 3d-to-2d queries,

    Y. Wang, V. C. Guizilini, T. Zhang, Y. Wang, H. Zhao, and J. Solomon, “Detr3d: 3d object detection from multi-view images via 3d-to-2d queries,” inConference on robot learning. PMLR, 2022, pp. 180–191

    2022

  13. [21]

    Rolic: A robust lidar-camera fusion frame- work for 3d object detection,

    L. Wang, S. Sun, and J. Zhao, “Rolic: A robust lidar-camera fusion frame- work for 3d object detection,”IEEE Transactions on Image Processing, vol. 35, pp. 6846–6859, 2026

    2026

  14. [22]

    Omnihd-scenes: A next-generation multimodal dataset for autonomous driving,

    L. Zheng, L. Yang, Q. Lin, W. Ai, M. Liu, S. Lu, J. Liu, H. Ren, J. Mo, X. Baiet al., “Omnihd-scenes: A next-generation multimodal dataset for autonomous driving,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026

    2026

  15. [23]

    Availability-aware sensor fusion via unified canonical space,

    D.-H. Paek and S.-H. Kong, “Availability-aware sensor fusion via unified canonical space,”Advances in Neural Information Processing Systems, vol. 38, pp. 89 259–89 277, 2026

    2026

  16. [24]

    Samfusion: Sensor-adaptive multimodal fusion for 3d object detection in adverse weather,

    E. Palladin, R. Dietze, P. Narayanan, M. Bijelic, and F. Heide, “Samfusion: Sensor-adaptive multimodal fusion for 3d object detection in adverse weather,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 484–503

    2024

  17. [25]

    Boosting faithful multi-modal llms via complementary visual grounding,

    Z. Fu, Z. Mao, L. Zhang, and Y. Zhang, “Boosting faithful multi-modal llms via complementary visual grounding,”IEEE Transactions on Image Processing, vol. 34, pp. 8641–8655, 2025. 14

    2025

  18. [26]

    Talk to parallel lidars: A human-lidar interaction method based on 3d visual grounding,

    Y. Liu, B. Sun, Y. Wang, J. Yang, X. Wang, and F.-Y. Wang, “Talk to parallel lidars: A human-lidar interaction method based on 3d visual grounding,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 305–321

    2024

  19. [27]

    Lidarefer: Context-aware outdoor 3d visual grounding for autonomous driving,

    Y.-S. Baek and H.-S. Oh, “Lidarefer: Context-aware outdoor 3d visual grounding for autonomous driving,”arXiv preprint arXiv:2411.04351, 2024

    2024 arXiv

  20. [28]

    Multi-sensor fusion technology for 3d object detection in autonomous driving: A review,

    X. Wang, K. Li, and A. Chehri, “Multi-sensor fusion technology for 3d object detection in autonomous driving: A review,”IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 2, pp. 1148–1165, 2023

    2023

  21. [29]

    Language-guided 3d object detection in point cloud for autonomous driving,

    W. Cheng, J. Yin, W. Li, R. Yang, and J. Shen, “Language-guided 3d object detection in point cloud for autonomous driving,”arXiv preprint arXiv:2305.15765, 2023

    2023 arXiv

  22. [30]

    Gpt-4 enhanced multimodal grounding for autonomous driving: Leveraging cross-modal attention with large language models,

    H. Liao, H. Shen, Z. Li, C. Wang, G. Li, Y. Bie, and C. Xu, “Gpt-4 enhanced multimodal grounding for autonomous driving: Leveraging cross-modal attention with large language models,”Communications in Transportation Research, vol. 4, p. 100116, 2024

    2024

  23. [31]

    Nugrounding: A multi-view 3d visual grounding framework in autonomous driving,

    F. Li, H. Jin, B. Gao, L. Fan, L. Jiang, and L. Zeng, “Nugrounding: A multi-view 3d visual grounding framework in autonomous driving,” arXiv preprint arXiv:2503.22436, 2025

    2025 arXiv

  24. [32]

    Vlm- grounder: A vlm agent for zero-shot 3d visual grounding,

    R. Xu, Z. Huang, T. Wang, Y. Chen, J. Pang, and D. Lin, “Vlm- grounder: A vlm agent for zero-shot 3d visual grounding,”arXiv preprint arXiv:2410.13860, 2024

    2024 arXiv

  25. [33]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” inEu- ropean conference on computer vision. Springer, 2020, pp. 213–229

    2020

  26. [34]

    Pointpillars: Fast encoders for object detection from point clouds,

    A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 12 697–12 705

    2019

  27. [35]

    Second: Sparsely embedded convolutional detection,

    Y. Yan, Y. Mao, and B. Li, “Second: Sparsely embedded convolutional detection,”Sensors, vol. 18, no. 10, p. 3337, 2018

    2018

  28. [36]

    Generalized intersection over union: A metric and a loss for bounding box regression,

    H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 658–666

    2019

  29. [37]

    The hungarian method for the assignment problem,

    H. W. Kuhn, “The hungarian method for the assignment problem,”Naval research logistics quarterly, vol. 2, no. 1-2, pp. 83–97, 1955

    1955

  30. [38]

    Multi-class road user detection with 3+ 1d radar in the view-of-delft dataset,

    A. Palffy, E. Pool, S. Baratam, J. F. Kooij, and D. M. Gavrila, “Multi-class road user detection with 3+ 1d radar in the view-of-delft dataset,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4961–4968, 2022

    2022

  31. [39]

    A transformer-based framework for visual grounding on 3d point clouds,

    A. Solgi and M. Ezoji, “A transformer-based framework for visual grounding on 3d point clouds,” inthe 20th IEEE CSI International Symposium on Artificial Intelligence and Signal Processing (AISP), 2024, pp. 1–5

    2024

  32. [40]

    Eda: Explicit text-decoupling and dense alignment for 3d visual grounding,

    Y. Wu, X. Cheng, R. Zhang, Z. Cheng, and J. Zhang, “Eda: Explicit text-decoupling and dense alignment for 3d visual grounding,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 19 231–19 242

    2023

  33. [41]

    Ges3vig: Incorporating pointing gestures into language-based 3d visual grounding for embodied reference understanding,

    A. M. Mane, D. Weerakoon, V. Subbaraju, S. Sen, S. E. Sarma, and A. Misra, “Ges3vig: Incorporating pointing gestures into language-based 3d visual grounding for embodied reference understanding,” inProceed- ings of the Computer Vision and Pattern Recognition Conference, 2025, ...

    2025

  34. [42]

    Text-guided sparse voxel pruning for efficient 3d visual grounding,

    W. Guo, X. Xu, Z. Wang, J. Feng, J. Zhou, and J. Lu, “Text-guided sparse voxel pruning for efficient 3d visual grounding,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 3666– 3675

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.