REVIEW 4 cited by
Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In autonomous driving tasks, scene understanding is the first step towards predicting the future behavior of the surrounding traffic participants. Yet, how to represent a given scene and extract its features are still open research questions. In this study, we propose a novel text-based representation of traffic scenes and process it with a pre-trained language encoder. First, we show that text-based representations, combined with classical rasterized image representations, lead to descriptive scene embeddings. Second, we benchmark our predictions on the nuScenes dataset and show significant improvements compared to baselines. Third, we show in an ablation study that a joint encoder of text and rasterized images outperforms the individual encoders confirming that both representations have their complementary strengths.
Forward citations
Cited by 4 Pith papers
-
Embodied Scene Understanding for Vision Language Models via MetaVQA
Fine-tuning on the auto-generated MetaVQA VQA corpus improves VLMs' spatial reasoning accuracy and partially improves their closed-loop driving safety in simulation.
-
VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision
VLM-AD uses GPT-4o-generated reasoning and action annotations as auxiliary supervision to improve end-to-end autonomous driving planning without VLM inference.
-
Large Foundation Models for Trajectory Prediction in Autonomous Driving: A Comprehensive Survey
A structured survey of LLM-based trajectory prediction methods, organized into trajectory-language mapping, multimodal fusion, and constraint-based reasoning, with benchmarks, metrics, and future directions.
-
A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach
A position/review paper argues data-driven model predictive control is the best route to safe, adaptive, human-like autonomous-driving motion planning, but provides no new derivation or experiment.
Discussion (0). Continue with ORCID to comment.