REVIEW 7 cited by
Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Vision-Language Models (LVLMs) have received widespread attention for advancing the interpretable self-driving. Existing evaluations of LVLMs primarily focus on multi-faceted capabilities in natural circumstances, lacking automated and quantifiable assessment for self-driving, let alone the severe road corner cases. In this work, we propose CODA-LM, the very first benchmark for the automatic evaluation of LVLMs for self-driving corner cases. We adopt a hierarchical data structure and prompt powerful LVLMs to analyze complex driving scenes and generate high-quality pre-annotations for the human annotators, while for LVLM evaluation, we show that using the text-only large language models (LLMs) as judges reveals even better alignment with human preferences than the LVLM judges. Moreover, with our CODA-LM, we build CODA-VLM, a new driving LVLM surpassing all open-sourced counterparts on CODA-LM. Our CODA-VLM performs comparably with GPT-4V, even surpassing GPT-4V by +21.42% on the regional perception task. We hope CODA-LM can become the catalyst to promote interpretable self-driving empowered by LVLMs.
Forward citations
Cited by 7 Pith papers
-
ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
ReCogDrive unifies VLM scene understanding with a diffusion planner reinforced by DiffGRPO to reach state-of-the-art results on NAVSIM and Bench2Drive benchmarks.
-
RoboTron-Sim: Improving Real-World Driving via Simulated Hard-Case
A simulation-to-real pipeline (HASS synthetic hard cases, scenario-aware prompts, and an image-to-ego geometry encoder) improves an MLLM's open-loop planning on nuScenes, especially in hard scenarios.
-
MoSE: Skill-by-Skill Mixture-of-Experts Learning for Embodied Autonomous Machines
A skill-by-skill mixture-of-experts router lets a sub-3B vision-language model beat much larger models on autonomous-driving and robot-reasoning benchmarks.
-
SafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation
SafeDrive228K is a 228K-example multimodal QA benchmark for traffic safety, and a graph-based RAG method improves VLM accuracy on it by 4.7 to 14.6 points across five models.
-
Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving
IRR-Drive adds an adaptive multimodal reflection step (text intention plus predicted future BEV) that lets a VLA model self-correct its trajectory plan according to scene complexity and reports SOTA on NAVSIM.
-
A Survey on LLM-as-a-Judge
A survey on LLM-as-a-Judge that reviews reliability strategies, proposes evaluation methods, and introduces a novel benchmark for assessing such systems.
-
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
A survey that organizes LLMs-as-judges research into functionality, methodology, applications, meta-evaluation, and limitations.
Discussion (0). Sign in to comment.