REVIEW 9 cited by
NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We introduce a novel visual question answering (VQA) task in the context of autonomous driving, aiming to answer natural language questions based on street-view clues. Compared to traditional VQA tasks, VQA in autonomous driving scenario presents more challenges. Firstly, the raw visual data are multi-modal, including images and point clouds captured by camera and LiDAR, respectively. Secondly, the data are multi-frame due to the continuous, real-time acquisition. Thirdly, the outdoor scenes exhibit both moving foreground and static background. Existing VQA benchmarks fail to adequately address these complexities. To bridge this gap, we propose NuScenes-QA, the first benchmark for VQA in the autonomous driving scenario, encompassing 34K visual scenes and 460K question-answer pairs. Specifically, we leverage existing 3D detection annotations to generate scene graphs and design question templates manually. Subsequently, the question-answer pairs are generated programmatically based on these templates. Comprehensive statistics prove that our NuScenes-QA is a balanced large-scale benchmark with diverse question formats. Built upon it, we develop a series of baselines that employ advanced 3D detection and VQA techniques. Our extensive experiments highlight the challenges posed by this new task. Codes and dataset are available at https://github.com/qiantianwen/NuScenes-QA.
Forward citations
Cited by 9 Pith papers
-
VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models
A training-free decoding framework that adaptively reweights attention toward video tokens and erases key visual evidence per frame to suppress hallucinated predictions, achieving 72.60% accuracy on EventHallusion wit...
-
OccVLA: Vision-Language-Action Model with Implicit 3D Occupancy Supervision
OccVLA trains a vision-language-action model to predict 3D occupancy as an auxiliary output, improving nuScenes trajectory planning and 3D VQA from camera images only, with the occupancy branch disabled at inference.
-
DriveQA: Passing the Driving Knowledge Test
DriveQA is a new multimodal driving-knowledge benchmark showing that LLMs and MLLMs struggle with right-of-way, numerical traffic rules, and sign variations, with modest transfer gains to nuScenes and BDD.
-
MMHU: A Massive-Scale Multimodal Benchmark for Human Behavior Understanding
MMHU introduces a large-scale multimodal benchmark with 57k human instances and rich annotations for motion, trajectory, text, behavior labels, and VQA in driving scenes.
-
VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understanding
The new VRU-Accident benchmark (1K videos, 6K QA pairs, 1K dense captions) shows the best evaluated MLLM reaches 66.9% on VRU-accident VQA versus 94.7% for human experts, with the weakest performance on causal and pre...
-
CoopReflect: Towards Natural Language Communication for Cooperative Autonomous Driving via Multi-Agent Learning
Post-episode multi-agent debriefing lets LLM driving agents learn concise natural-language coordination protocols that avoid collisions and merge traffic, and distillation makes the policy fast enough for near-real-time use.
-
Embodied Scene Understanding for Vision Language Models via MetaVQA
Fine-tuning on the auto-generated MetaVQA VQA corpus improves VLMs' spatial reasoning accuracy and partially improves their closed-loop driving safety in simulation.
-
ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Grounding
ViGiL3D is a 350-prompt diagnostic dataset showing that existing 3D visual grounding models lose 20 or more points on linguistically diverse prompts compared to ScanRefer.
-
Generative AI for Autonomous Driving: Frontiers and Opportunities
A comprehensive, structured survey of generative AI for autonomous driving, covering model families, sensor modalities, real-world applications, and open research challenges.
Discussion (0). Continue with ORCID to comment.