REVIEW 2 major objections 6 minor 2 cited by
ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving
T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper reports on the first W-CODA workshop at ECCV 2024 and documents its dual-track challenge on corner-case scene understanding and generation, claiming substantial improvements over organizer baselines.
desk verdict A useful workshop record, written oddly in the future tense, with self-reported challenge numbers that need a grain of salt. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing objects are the CODA-LM dataset, about 10K road images with LLM-synthesized, manually inspected textual annotations spanning global scene understanding, region-level reasoning, and driving suggestions, and the MagicDrive-based generation pipeline, a diffusion model controlled by BEV maps, 3D bounding boxes, and text prompts for weather and time. The evaluation protocol, with a composite final score for Track 1 and FVD, mAP, and mIoU for Track 2, is what turns the raw submissions into the reported percentage gains.
What would settle it
Re-run the Track 1 evaluation on a held-out set of corner-case images with annotations produced independently of the CODA-LM pipeline and check whether the winner's score remains roughly 156% above LLaVA-1.5; if the gap collapses, the reported gains depend on the organizers' data and scoring rather than on the model.
Extended reading notes
Core claim
On the paper's own terms, the finding is that a community challenge centered on corner cases can be organized and run, and that participating teams can substantially outperform existing open baselines on both understanding and generation. Track 1 evaluates multimodal large language models on the CODA-LM dataset of about 10K images with textual annotations, covering general perception, region perception, and driving suggestions; the winning team's final score exceeds the LLaVA-1.5 baseline by 156.02% and the CODA-VLM baseline by 13.36%. Track 2 asks participants to generate multi-view street videos consistent with BEV maps, 3D bounding boxes, and text descriptions of weather and time of day, building on MagicDrive; the winner improves FVD by 56.63%, mAP by 106.97%, and mIoU by 96.02% over that baseline. The paper argues these results indicate that multimodal large language models and geometrically controllable generation are promising tools for the corner-case problem, and that the event established a benchmark and community agenda for future work.
Load-bearing premise
The reported challenge results, in particular the large percentage gains over the LLaVA-1.5 and CODA-VLM baselines, depend on the evaluation protocol and on the CODA-LM dataset created by the organizers; if the dataset annotations or scoring are biased, the improvements would not reflect genuine capability gains.
Editorial extensions
If this is right
- Future corner-case understanding systems can be measured directly against the 156.02% improvement over LLaVA-1.5 and the 13.36% improvement over CODA-VLM on CODA-LM.
- The CODA-LM dataset of about 10K annotated images becomes a public benchmark for evaluating multimodal large language models on global perception, region perception, and driving suggestions in corner cases.
- Track 2 demonstrates that geometric control signals such as BEV maps, 3D boxes, and text can guide multi-view street video generation, with winning entries showing that the MagicDrive baseline can be improved substantially in quality and controllability.
- The publication of winners' reports and the ECCV workshop proceedings gives other teams reproducible starting points for future challenges.
Reading between the lines
- The large percentage gains over LLaVA-1.5 may partly reflect that this general-purpose baseline is not tuned for driving scenes, so the numbers should not be read as the winner being 156% better in absolute driving capability; a separate evaluation against a stronger driving-specific baseline would clarify the real gap.
- Because CODA-LM's annotations rely on LLM-synthetic text that was then manually inspected, the benchmark's validity depends on how well those annotations match what a human driver would identify as important; a human-annotation-only subset could test this.
- If the generation track's controllability metrics continue to improve alongside video quality, controllable world models may become practical for data augmentation and simulation of rare scenes, a direction the paper points to but does not quantify.
- The paper does not report inter-annotator or scorer variance, so it is unclear whether a different scoring rule for the final score would preserve the ranking of the top teams.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reports on the 1st W-CODA workshop held at ECCV 2024, including the call-for-papers topics and accepted contributions, the dual-track challenge (corner-case scene understanding and scene generation) with participation statistics and winners, the program schedule and invited speakers, and the organizers' background and relation to prior workshops. The paper is a retrospective event record rather than a technical research paper; its central claim is that the workshop took place as described and that the challenge outcomes listed in Section 3 constitute the official results of the event.
Significance. If accurate, the paper would serve as a useful archival reference for the autonomous-driving community: it consolidates the list of accepted full and abstract papers with a link to the official ECCV 2024 workshops proceedings, gives the workshop URL, and documents the invited talks, schedule, and organizing team. The paper also makes the challenge's dataset (CODA-LM) and baselines explicit, so the reader can trace the technical context. However, the scientific significance is confined to documentation; the quantitative challenge results are not independently verifiable from the manuscript alone, and the paper offers no algorithmic or empirical contribution beyond the event record.
major comments (2)
- [3.1, Results] The reported 'Final Score' improvements (+156.02% over LLaVA-1.5, +13.36% over CODA-VLM) are not interpretable because 'Final Score' is never defined, the test set and annotation protocol are not described, and no leaderboard snapshot or evaluation script is provided. Since these numbers are the paper's primary quantitative record and are self-reported on organizer-built baselines, the paper does not support their verification. Please provide the scoring formula, the baseline scores, the winning scores, and a persistent link to the leaderboard or evaluation repository.
- [3.2, Results] The Track 2 improvements (+56.63% FVD, +106.97% mAP, +96.02% mIoU over MagicDrive) are ambiguous: FVD is lower-is-better while mAP and mIoU are higher-is-better, and the paper does not state whether the percentages are relative or absolute improvements or give the underlying metric values. Without these details, the reader cannot judge the magnitude or direction of the reported gains. Please report the raw metric values for MagicDrive and for each ranked entry, and clarify the sign and normalization conventions.
minor comments (6)
- [3.1, Overview] The phrase 'this track 2 is dedicated' should read 'this track is dedicated' (or 'Track 1 is dedicated'), since Section 3.1 describes Track 1.
- [4] The program outline uses the future tense ('We plan a half-day event,' 'We will need a digital projector') despite the rest of the paper reporting results of the completed workshop; the tense should be made consistent to avoid confusing readers about the event's status.
- [2, Accepted papers] In the Paper ID 8 entry of the full-paper list, 'yangwangwang' appears to be an incomplete or misspelled author name; please verify and correct it.
- [2, Submission Tracks] The phrase 'F ull workshop papersnot previously published' is missing spaces; it should read 'Full workshop papers not previously published.'
- [1, footnote] The equal-contribution footnote uses both '⋆' and '∗' markers for different authors; please use a single consistent symbol set.
- [5] The claim that W-CODA has a 'peerless advantage' over FMAS is not substantiated; consider softening to a more measured comparison.
Circularity Check
No circularity: the paper is an event/challenge report with no derivation chain to reduce.
full rationale
The manuscript is a workshop report, not a technical derivation. It records that W-CODA took place at ECCV 2024, lists invited speakers and accepted papers, and reports challenge results in Section 3. There is no equation, fitted parameter, prediction, or uniqueness argument whose conclusion is equivalent to its input. The reported percentage improvements over organizer baselines (e.g., +156.02% over LLaVA-1.5 and +13.36% over CODA-VLM in Track 1; +56.63% FVD, +106.97% mAP, +96.02% mIoU over MagicDrive in Track 2) are empirical assertions about challenge submissions, not quantities derived by construction from the baselines. The CODA-LM dataset and the organizer baselines are cited from prior work, including work by overlapping authors, but citing one's own benchmark as the basis of a challenge is not circular reasoning unless the reported result is shown to be forced by the benchmark definition; no such reduction is exhibited. A concern about whether the challenge results are reproducible or whether the evaluation protocol is unbiased is a verifiability and correctness concern, not a circularity concern. The paper makes no attempt to prove or predict anything, so no load-bearing step reduces to its own inputs.
Assumptions & free parameters
assumptions (2)
- domain assumption CODA-LM dataset is a valid evaluation benchmark for corner case scene understanding.
- domain assumption MagicDrive is a valid baseline for controllable street scene generation.
Cite this review
Pith. "Pith review of ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving." pith.science (2026). https://pith.science/paper/7BWM4IKY
@misc{pith2026250701735,
author = {Pith},
title = {Pith review of: ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving},
year = {2026},
howpublished = {\url{https://pith.science/paper/7BWM4IKY}},
note = {Machine review of arXiv:2507.01735}
}
read the original abstract
In this paper, we present details of the 1st W-CODA workshop, held in conjunction with the ECCV 2024. W-CODA aims to explore next-generation solutions for autonomous driving corner cases, empowered by state-of-the-art multimodal perception and comprehension techniques. 5 Speakers from both academia and industry are invited to share their latest progress and opinions. We collect research papers and hold a dual-track challenge, including both corner case scene understanding and generation. As the pioneering effort, we will continuously bridge the gap between frontier autonomous driving techniques and fully intelligent, reliable self-driving agents robust towards corner cases.
Figures
Forward citations
Cited by 2 Pith papers
-
DriveJudge: Rethinking Autonomous Driving Evaluation with Vision-Language Models
DriveJudge combines VLM reasoning with rule functions on a new 33,577-sample human-annotated dataset, outperforming EPDMS by 21.23 AUC on quality classification and DriveCritic by 6.5% on trajectory preference.
-
AnyScene: Towards Highly Controllable Driving Scene Generation at Anywhere and Beyond
AnyScene is an occupancy-centric framework using a Spatial-Temporal Occupancy Diffusion Transformer and Geometry-Grounded View Expansion to generate controllable driving scenes and videos from BEV layouts.
Reference graph
Works this paper leans on
-
[1]
Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025) 1
arXiv 2025
-
[2]
arXiv preprint arXiv:2102.05897 (2021) 9
Breitenstein, J., Term¨ ohlen, J.A., Lipinski, D., Fingscheidt, T.: Corner cases for visual perception in automated driving: Some guidance on detection approaches. arXiv preprint arXiv:2102.05897 (2021) 9
arXiv 2021
-
[3]
Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuscenes: A multimodal dataset for autonomous driving. In: CVPR (2020) 9
work page 2020
-
[4]
arXiv preprint arXiv:2409.18042 (2024) 1
Chen, K., Gou, Y., Huang, R., Liu, Z., Tan, D., Xu, J., Wang, C., Zhu, Y., Zeng, Y., Yang, K., et al.: Emova: Empowering language models to see, hear and speak with vivid emotions. arXiv preprint arXiv:2409.18042 (2024) 1
arXiv 2024
-
[5]
Chen, K., Hong, L., Xu, H., Li, Z., Yeung, D.Y.: Multisiam: Self-supervised multi- instance siamese representation learning for autonomous driving. In: ICCV (2021) 8
work page 2021
-
[6]
Chen, K., Liu, Z., Hong, L., Xu, H., Li, Z., Yeung, D.Y.: Mixed autoencoder for self-supervised visual representation learning. In: CVPR (2023) 8
work page 2023
-
[7]
arXiv preprint arXiv:2310.10477 (2023) 4
Chen, K., Wang, C., Yang, K., Han, J., Hong, L., Mi, F., Xu, H., Liu, Z., Huang, W., Li, Z., Yeung, D.Y., Shang, L., Jiang, X., Liu, Q.: Gaining wisdom from setbacks: Aligning large language models via mistake analysis. arXiv preprint arXiv:2310.10477 (2023) 4
arXiv 2023
-
[8]
arXiv preprint arXiv:2306.04607 (2023) 5
Chen, K., Xie, E., Chen, Z., Hong, L., Li, Z., Yeung, D.Y.: Integrating geometric control into text-to-image diffusion models for high-quality detection data genera- tion via text prompt. arXiv preprint arXiv:2306.04607 (2023) 5
arXiv 2023
Show all 29 references
-
[9]
arXiv preprint arXiv:2405.14475 (2024) 5
Gao, R., Chen, K., Li, Z., Hong, L., Li, Z., Xu, Q.: Magicdrive3d: Controllable 3d generation for any-view rendering in street scenes. arXiv preprint arXiv:2405.14475 (2024) 5
2024 arXiv
-
[10]
arXiv preprint arXiv:2411.13807 (2024) 5
Gao, R., Chen, K., Xiao, B., Hong, L., Li, Z., Xu, Q.: Magicdrivedit: High- resolution long video generation for autonomous driving with adaptive control. arXiv preprint arXiv:2411.13807 (2024) 5
2024 arXiv
-
[11]
arXiv preprint arXiv:2310.02601 (2023) 5, 6
Gao, R., Chen, K., Xie, E., Hong, L., Li, Z., Yeung, D.Y., Xu, Q.: Magicdrive: Street view generation with diverse 3d geometry control. arXiv preprint arXiv:2310.02601 (2023) 5, 6
2023 arXiv
-
[12]
arXiv preprint arXiv:2506.04559 (2025) 1
Gou, Y., Chen, K., Liu, Z., Hong, L., Jin, X., Li, Z., Kwok, J.T., Zhang, Y.: Perceptual decoupling for scalable multi-modal reasoning via reward-optimized captioning. arXiv preprint arXiv:2506.04559 (2025) 1
2025
-
[13]
arXiv preprint arXiv:2403.09572 (2024) 1
Gou, Y., Chen, K., Liu, Z., Hong, L., Xu, H., Li, Z., Yeung, D.Y., Kwok, J.T., Zhang, Y.: Eyes closed, safety on: Protecting multimodal llms via image-to-text transformation. arXiv preprint arXiv:2403.09572 (2024) 1
2024 arXiv
-
[14]
arXiv preprint arXiv:2312.12379 (2023) 1
Gou, Y., Liu, Z., Chen, K., Hong, L., Xu, H., Li, A., Yeung, D.Y., Kwok, J.T., Zhang, Y.: Mixture of cluster-conditional lora experts for vision-language instruc- tion tuning. arXiv preprint arXiv:2312.12379 (2023) 1
2023 arXiv
-
[15]
arXiv preprint arXiv:2502.12635 (2025) 1
Gou, Y., Yang, H., Liu, Z., Chen, K., Zeng, Y., Hong, L., Li, Z., Liu, Q., Kwok, J.T., Zhang, Y.: Corrupted but not broken: Rethinking the impact of corrupted data in visual instruction tuning. arXiv preprint arXiv:2502.12635 (2025) 1
2025 arXiv
-
[16]
arXiv preprint arXiv:2106.11118 (2021) 1 12 Chen et al
Han, J., Liang, X., Xu, H., Chen, K., Hong, L., Ye, C., Zhang, W., Li, Z., Liang, X., Xu, C.: Soda10m: Towards large-scale object detection benchmark for autonomous driving. arXiv preprint arXiv:2106.11118 (2021) 1 12 Chen et al
2021 arXiv
-
[17]
arXiv preprint arXiv:2203.07724 (2022) 1, 4, 9
Li, K., Chen, K., Wang, H., Hong, L., Ye, C., Han, J., Chen, Y., Zhang, W., Xu, C., Yeung, D.Y., et al.: Coda: A real-world road corner case dataset for object detection in autonomous driving. arXiv preprint arXiv:2203.07724 (2022) 1, 4, 9
2022 arXiv
-
[18]
arXiv preprint arXiv:2312.00651 (2023) 5
Li, P., Liu, Z., Chen, K., Hong, L., Zhuge, Y., Yeung, D.Y., Lu, H., Jia, X.: Trackd- iffusion: Multi-object tracking data generation via diffusion models. arXiv preprint arXiv:2312.00651 (2023) 5
2023 arXiv
-
[19]
arXiv preprint arXiv:2404.10595 (2024) 1, 4, 5, 9
Li, Y., Zhang, W., Chen, K., Liu, Y., Li, P., Gao, R., Hong, L., Tian, M., Zhao, X., Li, Z., et al.: Automated evaluation of large vision-language models on self-driving corner cases. arXiv preprint arXiv:2404.10595 (2024) 1, 4, 5, 9
2024 arXiv
-
[20]
In: CVPR (2024) 5
Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning. In: CVPR (2024) 5
2024
-
[21]
arXiv preprint arXiv:2310.05873 (2023) 1
Liu, Z., Chen, K., Zhang, Y., Han, J., Hong, L., Xu, H., Li, Z., Yeung, D.Y., Kwok, J.: Geom-erasing: Geometry-driven removal of implicit concept in diffusion models. arXiv preprint arXiv:2310.05873 (2023) 1
2023 arXiv
-
[22]
arXiv preprint arXiv:2405.00557 (2024) 4
Liu, Z., Gou, Y., Chen, K., Hong, L., Gao, J., Mi, F., Zhang, Y., Li, Z., Jiang, X., Liu, Q., et al.: Mixture of insightful experts (mote): The synergy of thought chains and expert mixtures in self-alignment. arXiv preprint arXiv:2405.00557 (2024) 4
2024 arXiv
-
[23]
In: AAAI (2022) 8
Liu, Z., Han, J., Chen, K., Hong, L., Xu, H., Xu, C., Li, Z.: Task-customized self-supervised pre-training with scalable dynamic routing. In: AAAI (2022) 8
2022
-
[24]
https://openai.com/ contributions/gpt-4v (2023) 1
OpenAI: Gpt-4v(ision) technical work and authors. https://openai.com/ contributions/gpt-4v (2023) 1
2023
-
[25]
In: CVPR (2022) 1
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: CVPR (2022) 1
2022
-
[26]
arXiv preprint arXiv:2403.13304 (2024) 1
Wang, Y., Gao, R., Chen, K., Zhou, K., Cai, Y., Hong, L., Li, Z., Jiang, L., Yeung, D.Y., Xu, Q., Zhang, K.: Detdiffusion: Synergizing generative and perceptive mod- els for enhanced data generation and perception. arXiv preprint arXiv:2403.13304 (2024) 1
2024 arXiv
-
[27]
arXiv preprint arXiv:2311.05332 (2023) 1, 9
Wen, L., Yang, X., Fu, D., Wang, X., Cai, P., Li, X., Ma, T., Li, Y., Xu, L., Shang, D., et al.: On the road with gpt-4v (ision): Early explorations of visual-language model on autonomous driving. arXiv preprint arXiv:2311.05332 (2023) 1, 9
2023 arXiv
-
[28]
arXiv preprint arXiv:2410.23114 (2024) 1
Wu, J., Chung, T.T., Chen, K., Yeung, D.Y.: Unified triplet-level hallucina- tion evaluation for large vision-language models. arXiv preprint arXiv:2410.23114 (2024) 1
2024 arXiv
-
[29]
In: ICLR (2023) 8
Zhili, L., Chen, K., Han, J., Lanqing, H., Xu, H., Li, Z., Kwok, J.: Task-customized masked autoencoder via mixture of cluster-conditional experts. In: ICLR (2023) 8
2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.