Pith. sign in

REVIEW 3 major objections 4 minor 71 references

OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read OpenAD: a real-world benchmark for 3D detection of rare road objects.

desk verdict OpenAD is a genuinely useful 3D open-world detection benchmark, but the annotation-reliability evidence is thinner than the claims require. read the letter →

arxiv 2411.17761 v2 pith:KJRXTXMH submitted 2024-11-26 cs.CV

Zhongyu Xia , Jishuo Li , Zhiwei Lin , Xinhao Wang , Yongtao Wang , Ming-Hsuan Yang This is my paper · ORCID
classification cs.CV
keywords open-worldobjectdetectionautonomousdrivingbenchmark3Dcornercasesmultimodallargelanguagemodeldomaingeneralizationsemantic-awareevaluation2D-to-3Dlifting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces OpenAD, a real-world benchmark for open-world 3D object detection in autonomous driving. It selects 2,000 scenes from five existing driving datasets and re-annotates rare 'corner case' objects with natural-language labels spanning 206 categories, giving 19,761 total 2D/3D bounding boxes. The paper also proposes a semantic-aware evaluation protocol that matches predictions to ground truth by both geometry and semantic similarity, so synonymous descriptions count as correct and long-tailed categories do not skew the metric. On top of the benchmark, the authors propose a vision-centric 3D open-world detector baseline that lifts 2D open-world proposals to 3D boxes with a lightweight converter, plus an ensemble method that fuses open-world and specialized models. If the benchmark is sound, it gives the field a common yardstick for measuring whether detectors can find and describe uncommon objects, not just common categories.

What carries the argument

The load-bearing mechanism is the unified annotation and evaluation pipeline built on a multimodal large language model (MLLM). An MLLM abnormal filter flags scenes with corner-case objects, a second MLLM step assigns natural-language descriptions and seen/unseen flags, and geometric boxes are produced by point-cloud clustering plus manual correction. On the evaluation side, the matching rule treats a prediction as a true positive only when it passes both a positional threshold (IoU in 2D, center distance in 3D) and a semantic similarity threshold computed with CLIP features, so synonymous descriptions count as matches and long-tailed categories do not distort the metric. The same pipeline supports the proposed 2D-to-3D BBox Converter baseline, which turns frozen 2D open-world proposals into 3D boxes with a trainable depth network, pseudo point clouds, and a PointNet branch.

What would settle it

Take a random sample of, say, 200 corner-case objects spanning rare categories, have independent human annotators label them with the same protocol, and compare against OpenAD's MLLM-generated tags. If per-category agreement falls well below the claimed 90% for rare classes, or if swapping in the corrected labels changes the relative ranking of top models, the benchmark's core assumption of trustworthy ground truth is violated.

Watch

Extended reading notes

Core claim

The central claim is that OpenAD is the first real-world open-world benchmark for 3D object detection in autonomous driving, and that it changes what evaluation can reveal. Existing driving benchmarks only test a handful of predefined categories, so a model that excels on them may still miss unusual objects that matter for driving. OpenAD provides 6,597 corner-case objects plus 13,164 common objects across 2,000 temporally consistent scenes, each annotated with 2D and 3D boxes and a semantic tag, plus a per-object seen/unseen flag for each source dataset. The paper further argues that its semantic-aware metrics, which use CLIP cosine similarity and multiple positional thresholds, expose strengths and weaknesses that in-domain benchmarks hide, such as overfitting that trades away open-world recall.

Load-bearing premise

The benchmark's usefulness rests on the accuracy of the MLLM-produced semantic labels and seen/unseen flags; the paper estimates about 90% accuracy from manual review of only 30 challenging samples, and full category labels are withheld, so errors concentrated in rare categories could bias rankings.

Editorial extensions

If this is right

  • Models that look strong on nuScenes-style in-domain benchmarks can be much weaker on OpenAD's unseen categories and out-of-domain scenes, so OpenAD provides a way to detect overfitting that existing benchmarks miss.
  • Open-world 2D detectors can be turned into 3D open-world detectors with a small trainable converter, without retraining the 2D model.
  • Fusing a specialized 3D detector with an open-world detector raises both seen and unseen recall, suggesting the two capabilities are complementary rather than exclusive.
  • The seen/unseen split lets the benchmark attribute performance separately to domain generalization and open-ended recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because OpenAD withholds most category labels, future users cannot easily audit per-category annotation quality; releasing a small public seed of rare-category labels would make benchmark quality independently checkable.
  • The benchmark could be extended beyond detection: the same natural-language labels and seen/unseen splits could support open-world segmentation, occupancy prediction, or tracking evaluation.
  • The semantic matching via CLIP may favor models whose embeddings align with CLIP; a model with better geometry but different vocabulary could be under-scored, so an alternative semantic matcher trained on driving text would be a useful stress test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces OpenAD, a real-world open-world autonomous driving benchmark for 3D object detection. OpenAD contains 2,000 scenes drawn from five existing datasets (Argoverse 2, KITTI, nuScenes, ONCE, Waymo), with 6,597 newly annotated corner-case objects plus 13,164 objects taken from the original dataset annotations, for 19,761 objects in total. Objects are labeled with 2D and 3D bounding boxes, natural-language semantic tags organized into 206 categories, and per-dataset seen/unseen flags. The paper also proposes a semantic-aware evaluation protocol using CLIP-based semantic similarity with positional thresholds, a vision-centric 3D open-ended detection baseline that lifts 2D open-world proposals to 3D boxes, and an ensemble method that fuses open-world and specialized detectors. Extensive experiments compare 2D and 3D open-world and specialized models on the benchmark, and an online challenge is hosted on EvalAI.

Significance. If the annotation quality holds, OpenAD fills a real gap: it is the first real-world benchmark that evaluates 3D open-world object detection in autonomous driving with both corner-case categories and domain-transfer evaluation across multiple datasets. The unified data format, the natural-language semantic labels, and the seen/unseen split are useful innovations, and the paper ships a toolkit, evaluation code, and a public challenge. The baseline analysis is also valuable: it quantifies the complementarity of specialized and open-world models and shows that in-domain gains can come at the cost of open-world capability. However, the benchmark's central value depends on the reliability of its ground-truth semantic labels and seen/unseen flags, and that reliability is not yet convincingly established.

major comments (3)
  1. [Section 4 and Appendix A] The paper claims in Section 4 that the final MLLM and prompt achieve approximately 90% accuracy on the entire OpenAD dataset, but Appendix A states that accuracy was manually calculated from five repetitions on 30 highly challenging samples. It is also unclear whether the 90% refers to raw MLLM output or to the final labels after the manual correction stage. Because OpenAD's primary contribution is as ground truth for rare corner-case objects, this extrapolation is too thin to establish benchmark reliability. Please report a stratified accuracy estimate on the final released annotations, per-category accuracy for rare categories, and inter-annotator agreement (or an equivalent dual-annotation study), and clarify the role of manual correction in the reported figure.
  2. [Section 3.2] The true-positive criterion in the evaluation protocol uses CLIP cosine similarity thresholds (0.5, 0.7, 0.9) for semantic matching, but no calibration or human-agreement analysis is provided to show that these thresholds correctly separate synonymous from incorrect labels for the 206 corner-case categories. Since several evaluated methods (GLIP, OWL-ViT v2, GroundingDINO, YOLO-World) are CLIP-based, the metric may inadvertently favor methods whose embeddings align with CLIP. Please validate the semantic matching with human judgments or an independent text-similarity measure and report how model rankings change across semantic thresholds.
  3. [Appendix C and Tables 2/3] The AR-seen and AR-unseen splits in Tables 2 and 3 are central evidence for the paper's domain-generalization and open-endedness conclusions, but Appendix C withholds the full category labels and the underlying seen/unseen flags. These flags are produced by the same MLLM pipeline whose accuracy is only estimated on 30 samples, so errors concentrated in rare corner-case categories could directly bias the seen/unseen comparisons. Please release a public validation subset with full labels and flags, or otherwise provide a detailed audit of the seen/unseen flag-generation step, so that the benchmark's core evaluation can be independently checked.
minor comments (4)
  1. [Tables 2 and 3] There are several typographical errors in the table headers and method names, including 'V ovNetv2-99' (should be 'VoVNetv2-99'), 'ARnuscseen', and 'ARothersseen'; these should be corrected in the camera-ready version.
  2. [Appendix A, Figure 6] Figure 6 is difficult to read because the font sizes for the prompt variants and per-model accuracy rows are very small; please enlarge the figure or restructure it so that the ablation results are legible.
  3. [Section 3.1] The statement that each of the 2,000 scenes 'contains at least one corner case object that the original dataset has not annotated' should be reconciled with Figure 2's scene counts by source; please clarify whether the listed counts are before or after the manual scene filtering stage.
  4. [Section 3.2] The metric section does not specify how predictions with multiple or free-form labels are handled when computing CLIP semantic similarity, nor how the 'top 300 predictions' limit is applied to methods that output thousands of low-confidence boxes; a sentence clarifying this would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: OpenAD is a new benchmark whose annotation pipeline, evaluation protocol, and baselines are not derived from the claims they are intended to support.

full rationale

OpenAD is presented as a new dataset and evaluation benchmark rather than as a derived theoretical result, so there is no derivation chain that reduces a prediction to its own inputs. The annotation pipeline (Section 4, Figure 3) uses an MLLM plus manual correction to produce ground-truth labels; this is an empirical construction, not a circular argument. The evaluation metric in Section 3.2 uses CLIP cosine similarity as a semantic score; this is a design choice and does not make the benchmark's conclusions equivalent to its inputs, even though some evaluated methods also use CLIP-style text encoders. The self-citations in the paper (e.g., BEVFusion [37] and HENet [59]) are used as comparison baselines or related work, not as load-bearing justification for the benchmark's validity. The paper does extrapolate an 'approximately 90%' annotation accuracy from manual evaluation of 30 challenging samples (Section 4 and Appendix A), and Appendix C withholds the full label set; these are transparency and statistical-support limitations, not circularity. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors, and no known result is merely relabeled as organization. The central contribution is a real-world benchmark with independent value relative to its own evaluation protocol.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The benchmark depends on the accuracy of MLLM-assisted annotations and on the CLIP-based semantic scoring. The evaluation thresholds are chosen by hand and the exact CLIP variant is unspecified. No new physical entities are introduced; the dataset is a compilation of existing data with new labels.

free parameters (6)
  • Semantic similarity thresholds = 0.5, 0.7, 0.9
    Chosen by the authors for the TP definition in AP/AR; these thresholds affect all reported scores and are not derived from the data.
  • Positional thresholds for 3D AP = 0.5m, 1m, 2m, 4m
    Adopted from nuScenes evaluation; combined with semantic thresholds to compute average AP.
  • Top-300 prediction limit = 300
    All AP/AR are computed on the top 300 predictions per scene, a design choice.
  • IoU threshold range for 2D = 0.5 to 0.95 step 0.05
    Standard COCO-style thresholds; adopted without modification.
  • MLLM prompt and visual prompt design = chosen via 30-sample ablation
    The prompt ('This image was taken in a traffic environment...') and visual cue (closed bounding square) were selected based on manual accuracy over 30 challenging samples, so the annotation pipeline is tuned to this small set.
  • CLIP model for semantic scoring = not specified in paper
    The semantic similarity score uses 'the CLIP model' but the exact variant (e.g., ViT-B/32 vs ViT-L/14) is not given, making the metric underspecified.
assumptions (4)
  • domain assumption The MLLM-based annotation pipeline produces accurate semantic labels for corner case objects.
    The whole benchmark relies on this; paper reports 90% accuracy on the full dataset, but the evidence is manual evaluation on 30 samples (Appendix A).
  • domain assumption CLIP cosine similarity is an adequate measure of semantic equivalence between predicted and ground-truth labels.
    Used as the semantic score in all AP/AR computations; synonyms or paraphrases not in CLIP's embedding may be missed (Section 3.2).
  • domain assumption The original annotations in Argoverse 2, KITTI, nuScenes, ONCE, and Waymo are accurate for common objects.
    OpenAD reuses 13,164 common object annotations from the source datasets as ground truth, assuming they are correct (Section 3.1).
  • domain assumption The selected validation/test splits of the source datasets are suitable to measure domain generalization.
    Scenes are selected from validation/test splits of the five datasets; models trained on nuScenes training set are evaluated on all scenes, which assumes no overlap between training and evaluation scenes (Section 3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection." pith.science (2026). https://pith.science/paper/KJRXTXMH

@misc{pith2026241117761,
  author       = {Pith},
  title        = {Pith review of: OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KJRXTXMH}},
  note         = {Machine review of arXiv:2411.17761}
}
read the original abstract

Open-world perception aims to develop a model adaptable to novel domains and various sensor configurations and can understand uncommon objects and corner cases. However, current research lacks sufficiently comprehensive open-world 3D perception benchmarks and robust generalizable methodologies. This paper introduces OpenAD, the first real open-world autonomous driving benchmark for 3D object detection. OpenAD is built upon a corner case discovery and annotation pipeline that integrates with a multimodal large language model (MLLM). The proposed pipeline annotates corner case objects in a unified format for five autonomous driving perception datasets with 2000 scenarios. In addition, we devise evaluation methodologies and evaluate various open-world and specialized 2D and 3D models. Moreover, we propose a vision-centric 3D open-world object detection baseline and further introduce an ensemble method by fusing general and specialized models to address the issue of lower precision in existing open-world methods for the OpenAD benchmark. We host an online challenge on EvalAI. Data, toolkit codes, and evaluation codes are available at https://github.com/VDIGPKU/OpenAD.

Figures

Figures reproduced from arXiv: 2411.17761 by the authors.

Figure 1
Figure 1. Examples of corner case objects in OpenAD. These object categories have not been encountered by models trained on common 3D perception datasets during their training phase. 7, 44, 19, 12, 21, 48, 4, 25, 33] have been proposed. However, some datasets only provide semantic segmentation annotations without specific instances or annotate objects as abnormal but lack semantic tags. Moreover, datasets collected from real-… view at source ↗
Figure 2
Figure 2. Data composition of OpenAD. OpenAD covers multiple cities in various countries, including scenes during the day and night, on different weather and road scenarios. Additionally, we annotate each object with an indication of whether its category is observed in the training set of each dataset, allowing for separate evaluations of the model’s specialized performance and open-ended performance. 4 [PITH_FULL_IMAGE:figu… view at source ↗
Figure 3
Figure 3. Annotation pipeline. OpenAD is built upon a corner case discovery and annotation pipeline that integrates with a multimodal large language model (MLLM). 5 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The 3D open-world object detection framework we proposed. After obtaining 2D proposals from any frozen open-world 2D object detection model, we train a 2D-to-3D BBox Converter to predict 3D bounding boxes. The converter has a dual-branch architecture, which extracts ps…
Figure 5
Figure 5. Figure 5: Example results of open-world models, specialized models, and our proposed ensemble method. detection utilizes the capabilities of 2D open-world models. Specifically, by harnessing the open￾world capabilities of Yolo-world v2, our method obtains 6.7 AP and 17.1 AR impr…
Figure 6
Figure 6. Figure 6: Ablation on annotation pipeline. We conduct experiments by employing diverse visual and textual prompts, along with various MLLMs, and select the optimal approach. Accuracy is manually calculated based on five repetitions of testing on 30 highly challenging samples. B.…
Figure 7
Figure 7. Figure 7: Statistics on the number of objects in certain categories in OpenAD. 2 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 53 canonical work pages

  1. [1]

    Towards optimal strategies for training self-driving perception models in simulation

    David Acuna, Jonah Philion, and Sanja Fidler. Towards optimal strategies for training self-driving perception models in simulation. In NeurIPS, 2021

    work page 2021

  2. [2]

    Introducing the next generation of claude

    Anthropic. Introducing the next generation of claude. www.anthropic.com/news/claude-3-family, 2024

    work page 2024

  3. [3]

    Transfu- sion: Robust lidar-camera fusion for 3d object detection with transformers

    Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu, and Chiew-Lan Tai. Transfu- sion: Robust lidar-camera fusion for 3d object detection with transformers. In CVPR, 2022

    work page 2022

  4. [4]

    Nieto, Roland Y

    Hermann Blum, Paul-Edouard Sarlin, Juan I. Nieto, Roland Y . Siegwart, and César Cadena. The fishyscapes benchmark: Measuring blind spots in semantic segmentation. IJCV, 2019

    work page 2019

  5. [5]

    Marius Zöllner

    Daniel Bogdoll, Iramm Hamdard, Lukas Namgyu Rößler, Felix Geisler, Muhammed Bayram, Felix Wang, Jan Imhof, Miguel de Campos, Anushervon Tabarov, Yitian Yang, Hanno Gottschalk, and J. Marius Zöllner. Anovox: A benchmark for multimodal anomaly detection in autonomous driving. ECCV W-CODA workshop, 2024

    work page 2024

  6. [6]

    Fast range image-based segmentation of sparse 3d laser scans for online operation

    Igor Bogoslavskyi and Cyrill Stachniss. Fast range image-based segmentation of sparse 3d laser scans for online operation. In IROS, 2016

    work page 2016

  7. [7]

    Carla simulated data for rare road object detection

    Tom Bu, Xinhe Zhang, Christoph Mertz, and John M Dolan. Carla simulated data for rare road object detection. In IEEE International Intelligent Transportation Systems Conference , 2021

    work page 2021

  8. [8]

    Nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. Nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020

    work page 2020

Show all 71 references
  1. [9]

    Coda: Collaborative novel box discovery and cross-modal alignment for open-vocabulary 3d object detection

    Yang Cao, Yihan Zeng, Hang Xu, and Dan Xu. Coda: Collaborative novel box discovery and cross-modal alignment for open-vocabulary 3d object detection. In NeurIPS, 2023

    2023

  2. [10]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020

    2020

  3. [11]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021

    2021

  4. [12]

    Segmentmeifyoucan: A benchmark for anomaly segmentation

    Robin Chan, Krzysztof Lis, Svenja Uhlemeyer, Hermann Blum, Sina Honari, Roland Siegwart, Pascal Fua, Mathieu Salzmann, and Matthias Rottmann. Segmentmeifyoucan: A benchmark for anomaly segmentation. In NeurIPS Datasets and Benchmarks Track, 2021

    2021

  5. [13]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR, 2024

    2024

  6. [14]

    Yolo-world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In CVPR, 2024

    2024

  7. [15]

    Generative region-language pretraining for open-ended object detection

    Lin Chuang, Jiang Yi, Qu Lizhen, Yuan Zehuan, and Cai Jianfei. Generative region-language pretraining for open-ended object detection. In CVPR, 2024

    2024

  8. [16]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017

    2017

  9. [17]

    CARLA: An open urban driving simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Annual Conference on Robot Learning , 2017

    2017

  10. [18]

    Find n’ propagate: Open-vocabulary 3d object detection in urban environments

    Djamahl Etchegaray, Zi Huang, Tatsuya Harada, and Yadan Luo. Find n’ propagate: Open-vocabulary 3d object detection in urban environments. In CVPR, 2024

    2024

  11. [19]

    Muad: Multiple uncertainties for autonomous driving, a benchmark for multiple uncertainty types and tasks

    Gianni Franchi, Xuanlong Yu, Andrei Bursuc, Angel Tena, Rémi Kazmierczak, Séverine Dubuisson, Emanuel Aldea, and David Filliat. Muad: Multiple uncertainties for autonomous driving, a benchmark for multiple uncertainty types and tasks. In BMVC, 2022

    2022

  12. [20]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012

    2012

  13. [21]

    Dense open-set recognition with synthetic outliers generated by real nvp

    Matej Grci ´c, Petra Bevandi ´c, and Siniša Šegvi ´c. Dense open-set recognition with synthetic outliers generated by real nvp. In VISAPP, 2021

    2021

  14. [22]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In ICLR, 2022

    2022

  15. [23]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In CVPR, 2019

    2019

  16. [24]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017

    2017

  17. [25]

    Scaling out-of-distribution detection for real-world settings

    Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Xiaodong Song. Scaling out-of-distribution detection for real-world settings. In ICML, 2022. 10

    2022

  18. [26]

    Bevdet4d: Exploit temporal cues in multi-camera 3d object detection

    Junjie Huang and Guan Huang. Bevdet4d: Exploit temporal cues in multi-camera 3d object detection. arXiv preprint arXiv:2203.17054, 2022

    2022 arXiv

  19. [27]

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view

    Junjie Huang, Guan Huang, Zheng Zhu, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view. arXiv preprint arXiv:2112.11790, 2021

    2021 arXiv

  20. [28]

    Da-bev: Unsupervised domain adaptation for bird’s eye view perception

    Kai Jiang, Jiaxing Huang, Weiying Xie, Jie Lei, Yunsong Li, Ling Shao, and Shijian Lu. Da-bev: Unsupervised domain adaptation for bird’s eye view perception. In ECCV, 2024

    2024

  21. [29]

    Unlocking textual and visual wisdom: Open- vocabulary 3d object detection enhanced by comprehensive guidance from text and image

    Pengkun Jiao, Na Zhao, Jingjing Chen, and Yu-Gang Jiang. Unlocking textual and visual wisdom: Open- vocabulary 3d object detection enhanced by comprehensive guidance from text and image. In ECCV, 2024

    2024

  22. [30]

    Multi-modal classifiers for open-vocabulary object detection

    Prannay Kaul, Weidi Xie, and Andrew Zisserman. Multi-modal classifiers for open-vocabulary object detection. In ICML, 2023

    2023

  23. [31]

    Crash to not crash: Learn to identify dangerous vehicles using a simulator

    Hoon Kim, Kangwook Lee, Gyeongjo Hwang, and Changho Suh. Crash to not crash: Learn to identify dangerous vehicles using a simulator. In AAAI, 2019

    2019

  24. [32]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In ICCV, 2023

    2023

  25. [33]

    Coda: A real-world road corner case dataset for object detection in autonomous driving

    Kaican Li, Kai Chen, Haoyu Wang, Lanqing Hong, Chaoqiang Ye, Jianhua Han, Yukuai Chen, Wei Zhang, Chunjing Xu, Dit-Yan Yeung, et al. Coda: A real-world road corner case dataset for object detection in autonomous driving. In ECCV, 2022

    2022

  26. [34]

    Grounded language-image pre-training

    Liunian Harold Li*, Pengchuan Zhang*, Haotian Zhang*, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-image pre-training. In CVPR, 2022

    2022

  27. [35]

    Bevstereo: Enhancing depth estimation in multi-view 3d object detection with dynamic temporal stereo

    Yinhao Li, Han Bao, Zheng Ge, Jinrong Yang, Jianjian Sun, and Zeming Li. Bevstereo: Enhancing depth estimation in multi-view 3d object detection with dynamic temporal stereo. In AAAI, 2023

    2023

  28. [36]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal trans- formers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal trans- formers. In ECCV, 2022

    2022

  29. [37]

    Bevfusion: A simple and robust lidar-camera fusion framework

    Tingting Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia, Zhiwei Lin, Yongtao Wang, Tao Tang, Bing Wang, and Zhi Tang. Bevfusion: A simple and robust lidar-camera fusion framework. In NeurIPS, 2022

    2022

  30. [38]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014

    2014

  31. [39]

    Training-free open-ended object detection and segmentation via attention as prompts

    Zhiwei Lin, Yongtao Wang, and Zhi Tang. Training-free open-ended object detection and segmentation via attention as prompts. In NeurIPS, 2024

    2024

  32. [40]

    Sparsebev: High-performance sparse 3d object detection from multi-camera videos

    Haisong Liu, Yao Teng, Tao Lu, Haiguang Wang, and Limin Wang. Sparsebev: High-performance sparse 3d object detection from multi-camera videos. In ICCV, 2023

    2023

  33. [41]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  34. [42]

    Open-vocabulary point-cloud object detection without 3d annotation

    Yuheng Lu, Chenfeng Xu, Xiaobao Wei, Xiaodong Xie, Masayoshi Tomizuka, Kurt Keutzer, and Shang- hang Zhang. Open-vocabulary point-cloud object detection without 3d annotation. In CVPR, 2023

    2023

  35. [43]

    Codet: Co-occurrence guided region-word alignment for open-vocabulary object detection

    Chuofan Ma, Yi Jiang, Xin Wen, Zehuan Yuan, and Xiaojuan Qi. Codet: Co-occurrence guided region-word alignment for open-vocabulary object detection. NeurIPS, 2024

    2024

  36. [44]

    Two video data sets for tracking and retrieval of out of distribution objects

    Kira Maag, Robin Chan, Svenja Uhlemeyer, Kamil Kowol, and Hanno Gottschalk. Two video data sets for tracking and retrieval of out of distribution objects. In ACCV, 2022

    2022

  37. [45]

    One million scenes for autonomous driving: Once dataset

    Jiageng Mao, Minzhe Niu, Chenhan Jiang, Hanxue Liang, Jingheng Chen, Xiaodan Liang, Yamin Li, Chaoqiang Ye, Wei Zhang, Zhenguo Li, et al. One million scenes for autonomous driving: Once dataset. In NeurIPS Datasets and Benchmarks Track, 2021

    2021

  38. [46]

    Scaling open-vocabulary object detection

    Neil Houlsby Matthias Minderer, Alexey Gritsenko. Scaling open-vocabulary object detection. In NeurIPS, 2023

    2023

  39. [47]

    Gpt-4v(vision) system card

    OpenAI. Gpt-4v(vision) system card. cdn.openai.com/papers/GPTV_System_Card.pdf, 2023

    2023

  40. [48]

    Lost and found: detecting small road hazards for self-driving vehicles

    Peter Pinggera, Sebastian Ramos, Stefan Gehrig, Uwe Franke, Carsten Rother, and Rudolf Mester. Lost and found: detecting small road hazards for self-driving vehicles. In IROS, 2016

    2016

  41. [49]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017

    2017

  42. [50]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021. 11

    2021

  43. [51]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In ICCV, 2019

    2019

  44. [52]

    Sun rgb-d: A rgb-d scene understanding benchmark suite

    Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In CVPR, 2015

    2015

  45. [53]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, 2020

    2020

  46. [54]

    Detecting everything in the open world: Towards universal object detection

    Zhenyu Wang, Yali Li, Xi Chen, Ser-Nam Lim, Antonio Torralba, Hengshuang Zhao, and Shengjin Wang. Detecting everything in the open world: Towards universal object detection. In CVPR, 2023

    2023

  47. [55]

    Ov-uni3detr: Towards unified open-vocabulary 3d object detection via cycle-modality propagation

    Zhenyu Wang, Yali Li, Taichi Liu, Hengshuang Zhao, and Shengjin Wang. Ov-uni3detr: Towards unified open-vocabulary 3d object detection via cycle-modality propagation. In ECCV, 2024

    2024

  48. [56]

    Argoverse 2: Next generation datasets for self-driving perception and forecasting

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting. In NeurIPS Datasets and ...

    2021

  49. [57]

    Aligning bag of regions for open-vocabulary object detection

    Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, and Chen Change Loy. Aligning bag of regions for open-vocabulary object detection. In CVPR, 2023

    2023

  50. [58]

    Clim: Contrastive language-image mosaic for region representation

    Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Wentao Liu, and Chen Change Loy. Clim: Contrastive language-image mosaic for region representation. In AAAI, 2024

    2024

  51. [59]

    Henet: Hybrid encoding for end-to-end multi-task 3d perception from multi-view cameras

    Zhongyu Xia, Zhiwei Lin, Xinhao Wang, Yongtao Wang, Yun Xing, Shengxiang Qi, Nan Dong, and Ming-Hsuan Yang. Henet: Hybrid encoding for end-to-end multi-task 3d perception from multi-view cameras. In ECCV, 2024

    2024

  52. [60]

    Multi-modal queried object detection in the wild

    Yifan Xu, Mengdan Zhang, Chaoyou Fu, Peixian Chen, Xiaoshan Yang, Ke Li, and Changsheng Xu. Multi-modal queried object detection in the wild. In NeurIPS, 2023

    2023

  53. [61]

    Bevformer v2: Adapting modern image backbones to bird’s-eye- view recognition via perspective supervision

    Chenyu Yang, Yuntao Chen, Hao Tian, Chenxin Tao, Xizhou Zhu, Zhaoxiang Zhang, Gao Huang, Hongyang Li, Yu Qiao, Lewei Lu, et al. Bevformer v2: Adapting modern image backbones to bird’s-eye- view recognition via perspective supervision. In CVPR, 2023

    2023

  54. [62]

    Generalized out-of-distribution detection: A survey

    Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. IJCV, 2024

    2024

  55. [63]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, 2024

    2024

  56. [64]

    Imov3d: Learning open vocabulary point clouds 3d object detection from only 2d images

    Timing Yang, Yuanliang Ju, and Li Yi. Imov3d: Learning open vocabulary point clouds 3d object detection from only 2d images. In NeurIPS, 2024

    2024

  57. [65]

    Detclipv3: Towards versatile generative open-vocabulary object detection

    Lewei Yao, Renjie Pi, Jianhua Han, Xiaodan Liang, Hang Xu, Wei Zhang, Zhenguo Li, and Dan Xu. Detclipv3: Towards versatile generative open-vocabulary object detection. In CVPR, 2024

    2024

  58. [66]

    Center-based 3d object detection and tracking

    Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In CVPR, 2021

    2021

  59. [67]

    Open-vocabulary object detection using captions

    Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih-Fu Chang. Open-vocabulary object detection using captions. In CVPR, 2021

    2021

  60. [68]

    Fm-ov3d: Foundation model-based cross-modal knowledge blending for open-vocabulary 3d detection

    Dongmei Zhang, Chang Li, Ray Zhang, Shenghao Xie, Wei Xue, Xiaodong Xie, and Shanghang Zhang. Fm-ov3d: Foundation model-based cross-modal knowledge blending for open-vocabulary 3d detection. In AAAI, 2023

    2023

  61. [69]

    A simple framework for open-vocabulary segmentation and detection

    Hao Zhang, Feng Li, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, and Lei Zhang. A simple framework for open-vocabulary segmentation and detection. In ICCV, 2023

    2023

  62. [70]

    Detecting twenty- thousand classes using image-level supervision

    Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. Detecting twenty- thousand classes using image-level supervision. In ECCV, 2022

    2022

  63. [71]

    Detrs with collaborative hybrid assignments training

    Zhuofan Zong, Guanglu Song, and Yu Liu. Detrs with collaborative hybrid assignments training. In ICCV, 2023. 12 A. Ablation on the Annotation Pipeline. As shown in Figure 6, we conduct experiments by employing diverse visual and textual prompts, along with various MLLMs, and s...

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.