Pith. sign in

REVIEW 3 major objections 5 minor 57 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

31,398 high-risk driving events paired with LLM reasoning and simulator checks

2026-07-09 20:01 UTC pith:KZU4IC32

load-bearing objection Dataset of 31K high-risk driving events with LLM annotations — solid resource, but closed-loop validation is in-sample and underspecified the 3 major comments →

arxiv 2607.07103 v1 pith:KZU4IC32 submitted 2026-07-08 cs.LG cs.DB

A knowledge-augmented dataset of high-risk driving scenarios with LLM annotations for autonomous driving

Heye Huang , Jingguang Li , Zhiyuan Zhou , Paul Liang , Mingyu Wu , Kitae Jang , Jianqiang Wang This is my paper
classification cs.LG cs.DB
keywords drivingannotationshigh-riskk-riskscenariosautonomouseventlanguage
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces K-Risk, a dataset built to address a gap in autonomous driving research: existing driving datasets are dominated by routine, uneventful driving, while the rare safety-critical events that matter most for safe deployment are almost absent. The authors aggregate 20 trajectory datasets from Europe, China, and the United States, then apply a three-stage filtering pipeline — a driver risk field scorer, calibrated hard-maneuver detectors, and a two-second trajectory-conflict predictor — to extract 31,398 high-risk events, including 1,036 extreme near-collision cases. Each event is stored as a synchronized triple: raw trajectory data in CSV, structured metadata in JSON, and natural language annotations in text. The language layer includes rule-based scenario descriptions and abnormal-behavior alerts for every event, and for a representative subset, an LLM generates causal risk analyses and discrete action recommendations from a five-option schema (keep, turn left, turn right, accelerate, decelerate). These recommendations are validated in a collision-free simulator with iterative reflection, producing verified safety signals. The paper's central claim is that by combining multi-dimensional physical risk modeling, interpretable language supervision, and closed-loop decision verification in a single standardized format, K-Risk provides the missing foundation for training and evaluating risk-aware autonomous driving agents that can reason about why a situation is dangerous and what to do about it.

Core claim

The paper demonstrates that it is feasible to construct a large-scale, multi-source dataset of safety-critical driving events where each event carries not just trajectory data but also structured semantic annotations and verifiable decision labels. The key mechanism is a layered pipeline: first, a driver risk field filter retains only the top 10% of frames by perceived danger; second, calibrated behavioral thresholds detect hard maneuvers that measurably reduce time-to-collision; third, a two-second trajectory-conflict predictor flags imminent collisions. On top of this physical filtering, an LLM annotation layer converts the numerical records into natural language risk analyses and action推荐

What carries the argument

The annotation protocol has three physical filters: (1) a Driver Risk Field (DRF) that scores each frame by ego-vehicle dynamics, surrounding-vehicle configuration, and spatial proximity, retaining only the top 10% per source; (2) a hard-maneuver detector using ±3 m/s² acceleration thresholds and dataset-specific lateral-speed thresholds within a 0.7-second window; (3) a two-second trajectory-conflict predictor under constant-acceleration and constant-steering assumptions. Events are graded moderate (76.8%), high (19.7%), or extreme (3.5%). The semantic layer adds rule-based scenario descriptions and abnormal-behavior notifications for all events, plus LLM-generated causal risk analyses and

Load-bearing premise

The annotation protocol assumes that a single set of calibrated thresholds — DRF top-10%, ±3 m/s² acceleration, TTC below 5 seconds — correctly identifies safety-critical events across all 20 heterogeneous source datasets spanning different countries, road types, and vehicle behaviors. If these thresholds are miscalibrated for specific driving cultures or road geometries, the curated events may not represent genuine high-risk scenarios.

What would settle it

If a substantial fraction of the 31,398 curated events, when reviewed by independent traffic-safety experts, turn out to be routine driving situations that do not require evasive or defensive maneuvers — or conversely, if a large number of genuine high-risk events in the source datasets are missed by the three-stage filter — then the extraction pipeline does not accurately identify safety-critical driving, and the dataset's value for training risk-aware agents would be undermined.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • K-Risk's synchronized trajectory-metadata-language format could become a standard template for other safety-critical domains where rare events must be paired with interpretable reasoning, such as aviation near-misses or medical emergency response.
  • The closed-loop validation approach — where LLM recommendations are tested in simulation and failure cases are fed back as reflection — could be generalized as a training methodology for any decision-making agent where verifiable outcomes exist.
  • The trial-1 vs trial-3 preference pairs generated through iterative reflection provide a ready-made dataset for preference optimization methods without additional human labeling, which could lower the cost of aligning language models to safety-critical domains.
  • The multi-source aggregation across three continents demonstrates that heterogeneous trajectory data can be unified under a common risk framework, suggesting that similar unification could be attempted for other fragmented driving data collections.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the DRF thresholds and ±3 m/s² hard-maneuver cutoffs are not equally valid across all 20 source datasets — which span different road geometries, traffic cultures, and vehicle types — then the curated events may systematically over- or under-represent certain risk categories, biasing downstream models trained on K-Risk.
  • The constant-acceleration, constant-steering trajectory prediction used in the conflict detector is a simplification that may miss conflicts arising from non-linear driver behavior, particularly in complex intersection or roundabout scenarios where vehicles follow curved paths.
  • The 5% human-driver collision rate on the extreme subset, compared to the LLM's 1.91% after three trials, could be interpreted as evidence that LLM-based reasoning outperforms human drivers in these scenarios — but the comparison may be unfair because the human drivers operated without the structured risk reminders and eight-agent situational context provided to the LLM.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript presents K-Risk, a knowledge-augmented dataset of 31,398 high-risk driving scenarios curated from 20 public trajectory datasets across Europe, China, and the United States. Each event is released as a synchronized triple (CSV trajectory, JSON metadata, text annotation) containing structured scenario descriptions, abnormal-behavior notifications, and, for a representative subset, LLM-generated causal risk analyses and action recommendations. A closed-loop validation framework iteratively refines LLM-generated decisions through a collision-free simulator. The dataset addresses a genuine gap: existing driving datasets rarely combine event-level risk labels, semantic annotations, and verifiable safety signals. The curation pipeline (DRF filtering, hard-maneuver detection, TTC/trajectory-conflict prediction) is clearly described, and the release of code and standardized file formats is commendable.

Significance. The dataset contribution is substantial and timely. Aggregating 20 heterogeneous sources into a unified event-level format with multi-layer annotations fills a real need for training and evaluating risk-aware autonomous driving agents. The synchronized triple format (CSV/JSON/text) is well-designed for bridging trajectory-level and language-level supervision. The release of processing code, benchmark protocols, and preference pairs from closed-loop trials adds practical value. The five-action schema and the alignment with standard LLM post-training stages (CPT, SFT, RLHF, RLVR) make the dataset immediately usable for the community.

major comments (3)
  1. Table 2 and surrounding text (Technical Validation, 'Closed-loop usability of the extreme subset'): The three-trial closed-loop validation uses the same 262 scenarios across all trials, with each trial's failures fed back as reflection prompts for the next. There is no held-out test set. The reported 58.3% relative reduction in collision rate (4.58% → 1.91%) therefore cannot distinguish generalizable safety improvement from scenario-specific prompt memorization. The abstract's claim of 'verifiable decision supervision' and 'verifiable decisions' rests on this result. The dataset contribution stands independently, but the decision-supervision claim needs either (a) a held-out evaluation on unseen events, or (b) a clear reframing stating that the closed-loop loop produces training data (preference pairs) rather than demonstrating generalizable decision quality. As written, the text implies
  2. Appendix A: The DRF formulation depends on six parameters (p, t_la, m, c, k_1, k_2) whose numerical values are never specified. The text states 'the DRF threshold is set per source' (Annotation Protocol) and that only the top 10% of frames are retained, but without the parameter values the DRF computation is not reproducible from the mathematical description alone. The code release may contain these values, but the manuscript should either state them or explicitly direct the reader to the specific code file and configuration where they are defined.
  3. Annotation Protocol, third filter, and Stage 5 of Appendix C: The closed-loop simulator uses constant-acceleration and constant-steering assumptions to roll trajectories forward for two seconds. While this model is cited as well-validated for short-term motion prediction (Refs 34, 35), it is also used to evaluate discrete action recommendations that may include lane changes (actions 2 and 3 in the five-action schema). A lane change executed under constant-steering is physically inconsistent. The manuscript should discuss whether this simplification systematically biases the evaluation of lateral actions, or at minimum state it as a known limitation.
minor comments (5)
  1. The manuscript mentions 'domain experts' reviewed LLM-generated annotations (Annotation Protocol, last paragraph) but does not specify how many experts, their qualifications, inter-rater agreement, or the fraction of annotations revised. A brief statement would strengthen the validation.
  2. Figure 1 contains a placeholder citation mark '?,' in the Background & Summary text ('systems that are at once safer and more interpretable?, 4'). This should be corrected.
  3. The event distribution across sources is highly uneven (highD: 8,807; FreewayB: 5,304; vs. aggregated AV sources: 2,570). While the text acknowledges this, it would help to state the number of events per AV source individually, since the 14 AV sources collectively contribute fewer events than a single HV source.
  4. Table 1: The 'Post-training stages' column for K-Risk lists 'CPT + SFT + RLHF + RLVR,' which is aspirational rather than demonstrated. The paper shows SFT data and preference pairs, but CPT and RLVR are only suggested in Usage Notes. Consider clarifying which stages are supported by released data versus proposed as future work.
  5. Reference 33 (FHWA surrogate safety measures) includes the note 'placeholder pending verification' in the reference list. This should be resolved before publication.

Circularity Check

0 steps flagged

No significant circularity; one minor self-citation for the DRF model that is not load-bearing for the dataset's core contribution

full rationale

The paper is primarily a dataset construction contribution. Its main claim—31,398 curated high-risk events with multi-layered annotations from 20 sources—does not involve a derivation chain that could be circular. The Driver Risk Field (DRF) is adopted from SafeDrive (Ref 4, co-authored by Zhou), which is a self-citation, but the DRF is a parameterized kinematic model applied to external trajectory data; it is not defined in terms of the risk labels it produces, so there is no self-definitional loop. The closed-loop validation (Table 2: collision rate dropping from 4.58% to 1.91% across three trials on the same 262 events) raises a legitimate generalization concern—iterative prompt refinement on the evaluation set without a held-out test—but this is an overfitting risk, not circularity in the strict sense: the LLM's recommended actions are not defined in terms of the simulator's collision outcomes, and the simulator independently evaluates them. The improvement could reflect memorization, but it is not a case where the 'prediction reduces to the input by construction.' No uniqueness theorem is invoked, no fitted parameter is renamed as a prediction, and no ansatz is smuggled through self-citation. The self-citation to SafeDrive for the DRF formulation is minor and not load-bearing for the dataset's standalone value.

Axiom & Free-Parameter Ledger

6 free parameters · 3 axioms · 0 invented entities

The paper relies on established models and thresholds rather than inventing new entities. The main burden is on the DRF model parameters and the assumption that LLM annotations are reliable.

free parameters (6)
  • DRF top-percentile threshold = 10%
    Used to filter frames; set per source but the calibration process is not detailed.
  • Hard maneuver acceleration threshold = ±3m/s²
    Used to detect hard acceleration/braking; cited as widely used in traffic safety research.
  • Lane change lateral speed threshold = 75% of source average
    Used to detect lane changes; chosen to tolerate normal variability.
  • TTC thresholds = 5s, 3s, 2s
    Used to grade risk levels; standard thresholds in traffic safety.
  • DRF model parameters (p, tla, m, c, k1, k2) = Not specified
    Parameters of the SafeDrive DRF model; values not given in the paper.
  • Detection window = 0.7s
    Used for maneuver detection; based on human brake-reaction time.
axioms (3)
  • domain assumption The driver risk field (DRF) model from SafeDrive accurately quantifies perceived driving risk.
    The entire event extraction pipeline depends on this model, which is adopted without independent validation in this paper.
  • domain assumption Constant-acceleration and constant-steering assumptions are valid for 2-second trajectory forecasting.
    Used in the third filter and the closed-loop simulator; cited as well-validated but is a strong simplification.
  • ad hoc to paper LLM-generated risk analyses and action recommendations are factually correct and contextually consistent.
    The paper states expert review was used, but the scale and rigor of this review is not quantified.

pith-pipeline@v1.1.0-glm · 27018 in / 2568 out tokens · 210665 ms · 2026-07-09T20:01:24.464412+00:00 · methodology

0 comments
read the original abstract

Safe autonomous driving requires both rapid responses to common high-risk events and deeper reasoning over rare, extreme long-tail scenarios in traffic safety. These scenarios are severely under-represented in naturalistic driving data, and existing trajectory and language-augmented datasets seldom provide high-risk event labels, semantic annotations, and verifiable safety signals. Here we present K-Risk, a knowledge-augmented dataset that combines structured driving trajectories with large language model generated semantic annotations for safety-critical driving scenarios. K-Risk integrates 20 human-driven and autonomous-vehicle trajectory datasets from Europe, China, and the United States, covering highways, urban freeways, intersections, and roundabouts. Using a unified risk-centric extraction pipeline, K-Risk curates 31,398 high-risk events, together with a 1,036-event extreme subset of near-collision cases. Each event is released as a synchronized trajectory, metadata, and language triplet containing structured scenario descriptions, abnormal-behavior notifications, and, for a representative subset, causal risk analyses and action recommendations validated through a closed-loop simulator with iterative reflection. By combining multi-dimensional risk annotations, interpretable language supervision, and verifiable decisions, K-Risk bridges structured traffic trajectories, semantic reasoning, and decision supervision, providing a standardized foundation for developing and evaluating next-generation risk-aware autonomous driving agents.

Figures

Figures reproduced from arXiv: 2607.07103 by Heye Huang, Jianqiang Wang, Jingguang Li, Kitae Jang, Mingyu Wu, Paul Liang, Zhiyuan Zhou.

Figure 1
Figure 1. Figure 1: Motivation for K-Risk. Naturalistic driving data are dominated by routine cases, whereas safety-critical long-tail events such as cut-ins, hard braking, VRU conflicts, low-TTC situations and roundabout conflicts are rare but important for autonomous driving safety. Existing datasets mainly provide perception or trajectory records for normal driving and often lack event-level risk objects, multi-dimensional… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the K-Risk dataset and the LLM annotation framework. (a) Multi-source curation aggregates 20 human-driven (HV) and autonomous-vehicle (AV) trajectory sources across the United States, Europe and China, spanning highways, urban freeways, intersections and roundabouts. (b) Risk-centric event extraction screens the raw records through a driver risk field filter, calibrated hard-maneuver thresholds… view at source ↗
Figure 3
Figure 3. Figure 3: Detection of risk-relevant maneuvers and the resulting graded events. (a) The driver risk field retains interaction-rich candidate frames and removes routine cruising; the ego vehicle and its surrounding agents are shown with their TTC and time headway. (b) Each maneuver is detected within a 0.7 s window: longitudinal acceleration beyond ±3m/s2 marks hard acceleration or hard braking, and a peak in lateral… view at source ↗
Figure 4
Figure 4. Figure 4: Data processing workflow of K-Risk. Raw records from the 20 sources first pass through lane context augmentation, which reconstructs the lane topology and attaches up to eight surrounding agents to each ego vehicle (a). Risk event extraction then applies the driver risk field filter, the hard-maneuver detector and the TTC and trajectory-conflict test, and grades the retained events as moderate (76.8%), hig… view at source ↗
Figure 5
Figure 5. Figure 5: Layered structure of a single K-Risk event, stored as one synchronized triple. The CSV layer holds the raw per-frame trajectories of the ego and conflict vehicles, including position, velocity, acceleration, lane, TTC and behavioral flags (a). The JSON layer encodes the structured metadata, with agent roles, lane relationships and the risk level (b). The text layer provides the natural language annotation,… view at source ↗
Figure 6
Figure 6. Figure 6: Composition and safety-critical properties of K-Risk. (a) Curated events per source, stacked by risk level (moderate, high, extreme) and grouped by driving environment; the autonomous-vehicle (AV) subset carries no human-driver risk stratification. (b) Joint ego-speed × time-to-collision (TTC) density for K-Risk (filled) versus the original HighD data (blue contours), with speed and TTC marginals; K-Risk c… view at source ↗
Figure 7
Figure 7. Figure 7: Closed-loop LLM annotation framework. A structured input describes the ego vehicle, up to eight surrounding agents, the lane topology and legal maneuvers, and the TTC (a). The LLM agent reasons over the scenario description, the abnormal-behavior notifications and the causal risk cues (b) and recommends an action from the five-action schema (c). The action is checked against a collision-free simulator, and… view at source ↗
Figure 8
Figure 8. Figure 8: Worked example of a single K-Risk event, drawn from the ExpresswayA source. The text layer holds the environmental description, the LLM risk analysis and the decision for the ego vehicle (top left). Time-stamped snapshots show the preceding, critical and resolution frames of the event window (middle). The JSON layer lists the per-frame trajectory records, position, speed, acceleration, heading, lane and th… view at source ↗
Figure 9
Figure 9. Figure 9: Worked example of the closed-loop LLM annotation on an ExpresswayA event. The structured scenario description of ego vehicle 822 and its neighbors, together with the domain-specific system prompt and its five-action schema, form the LLM input (left). The trajectory visualization marks the critical frame and the TTC of about 1.2 s, with the safe and unsafe action regions indicated (top right). The LLM produ… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 57 canonical work pages · 13 internal anchors

  1. [1]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2446–2454 (2020)

    Sun, P.et al.Scalability in perception for autonomous driving: Waymo open dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2446–2454 (2020)

    work page 2020

  2. [2]

    Wilson, B.et al.Argoverse 2: Next generation datasets for self-driving perception and forecasting.arXiv preprint arXiv:2301.00493(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  3. [3]

    Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

    Fu, D.et al.Drive like a human: Rethinking autonomous driving with large language models. InarXiv preprint arXiv:2307.07162(2023). Demonstrates LLMs’ reasoning, interpretation, and memorization abilities in long-tail driving scenarios

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  4. [4]

    SafeDrive: Knowledge- and Data-Driven Risk-Sensitive Decision-Making for Autonomous Vehicles with Large Language Models

    Zhou, Z.et al.Safedrive: Knowledge- and data-driven risk-sensitive decision-making for autonomous vehicles with large language models (2024). 2412.13238

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  5. [5]

    & Eckstein, L

    Krajewski, R., Bock, J., Kloeker, L. & Eckstein, L. The highd dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems. In2018 21st International Conference on Intelligent Transportation Systems (ITSC), 2118–2125 (IEEE, 2018)

    work page 2018

  6. [6]

    In2020 IEEE Intelligent Vehicles Symposium (IV), 1929–1934 (IEEE, 2020)

    Bock, J.et al.The ind dataset: A drone dataset of naturalistic road user trajectories at german intersections. In2020 IEEE Intelligent Vehicles Symposium (IV), 1929–1934 (IEEE, 2020). 13/22

    work page 1929

  7. [7]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2446–2454 (2020)

    Sun, P.et al.Scalability in perception for autonomous driving: Waymo open dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2446–2454 (2020)

    work page 2020

  8. [8]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11621–11631 (2020)

    Caesar, H.et al.nuscenes: A multimodal dataset for autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11621–11631 (2020). 9.Chang, M.-F.et al.Argoverse: 3d tracking and forecasting with rich maps. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8748–87...

    work page 2020

  9. [9]

    OnSiteVRU: A High-Resolution Trajectory Dataset for High-Density Vulnerable Road Users

    Yan, Z., Li, J., Hang, P. & Sun, J. Onsitevru: A high-resolution trajectory dataset for high-density vulnerable road users. arXiv preprint arXiv:2503.23365(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps

    Zhan, W.et al.Interaction dataset: An international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps. InarXiv preprint arXiv:1910.03088(2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  11. [11]

    Jiang, X.et al.Interhub: A naturalistic trajectory dataset with dense interaction for autonomous driving.arXiv preprint arXiv:2411.18302(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  12. [12]

    & Farooq, B

    Mokhtarian, H. & Farooq, B. Time-to-collision and collision probability in traffic safety analysis: A review.Accid. Analysis & Prev.165, 106528 (2022)

    work page 2022

  13. [13]

    Arai, H.et al.Covla: Comprehensive vision-language-action dataset for autonomous driving.arXiv preprint arXiv:2408.10845(2024)

    work page arXiv 2024

  14. [14]

    Wang, S.et al.Omnidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning.arXiv preprint arXiv:2504.04348(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    InProceedings of the European Conference on Computer Vision (ECCV)(2024)

    Sima, C.et al.DriveLM: Driving with graph visual question answering. InProceedings of the European Conference on Computer Vision (ECCV)(2024)

    work page 2024

  16. [16]

    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)(2025)

    Li, Y .et al.Automated evaluation of large vision-language models on self-driving corner cases (CODA-LM). In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)(2025)

    work page 2025

  17. [17]

    Gao, Y .et al.NuRisk: A visual question answering dataset for agent-level risk assessment in autonomous driving.arXiv preprint arXiv:2509.13869(2025)

    work page arXiv 2025

  18. [18]

    Chi, H.et al.Impromptu VLA: Open weights and open data for driving vision-language-action models.arXiv preprint arXiv:2505.23757(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  19. [19]

    Towards Safe Mobility: A Unified Transportation Foundation Model enabled by Open-Ended Vision-Language Dataset

    Huang, W.et al.Towards safe mobility: Large-scale traffic data (LTD) and the UniVLT traffic vision-language transformer. arXiv preprint arXiv:2604.22260(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  20. [20]

    & Eckstein, L

    Krajewski, R., Moers, T., Bock, J., Vater, L. & Eckstein, L. The rounD dataset: A drone dataset of road user trajectories at roundabouts in Germany. In2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 1–6 (IEEE, 2020). Dataset available at https://www.round-dataset.com

    work page 2020

  21. [21]

    Zheng, O.et al.CitySim: A drone-based vehicle trajectory dataset for safety-oriented research and digital twins.Transp. Res. Rec.2678, 606–621 (2024). Dataset available at https://github.com/UCF-SST-Lab/UCF-SST-CitySim1-Dataset

    work page 2024

  22. [22]

    Next generation simulation (NGSIM) vehicle trajectories and supporting data

    Federal Highway Administration. Next generation simulation (NGSIM) vehicle trajectories and supporting data. U.S. Department of Transportation, Intelligent Transportation Systems Joint Program Office (2006). I-80 recording, Emeryville, CA. https://ops.fhwa.dot.gov/trafficanalysistools/ngsim.htm

    work page 2006

  23. [23]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 9710–9719 (2021)

    Ettinger, S.et al.Large scale interactive motion forecasting for autonomous driving: The Waymo open motion dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 9710–9719 (2021)

    work page 2021

  24. [24]

    A: Transp

    Yang, M.et al.Microsimacc: an open database for field experiments on the potential capacity impact of commercial adaptive cruise control (acc).Transp. A: Transp. Sci.1–30 (2024)

    work page 2024

  25. [25]

    & Ciuffo, B

    Makridis, M., Mattas, K., Anesiadou, A. & Ciuffo, B. Openacc. an open database of car-following experiments to study the properties of commercial acc systems.Transp. Res. Part C: Emerg. Technol.125, 103047 (2021)

    work page 2021

  26. [26]

    Zhou, H., Ma, L., Liang, S., Li, X. & Qu, X. A unified longitudinal trajectory dataset for automated vehicle.Sci. Data11, 1123 (2024). Dataset available at https://github.com/CATS-Lab/Filed-Experiment-Data-ULTra-A V

    work page 2024

  27. [27]

    An analysis of vehicle acceleration behavior in urban traffic flow.Transp

    Ishikawa, H. An analysis of vehicle acceleration behavior in urban traffic flow.Transp. Res. Part A: Policy Pract.25, 265–278 (1991)

    work page 1991

  28. [28]

    Liu, H., Wang, W. & Li, K. Traffic safety and comfort evaluation based on longitudinal acceleration thresholds.Accid. Analysis & Prev.80, 220–229 (2015). 14/22

    work page 2015

  29. [29]

    & Onieva, E

    Milanés, V ., Pérez, J., Godoy, J. & Onieva, E. Lane-change fuzzy control in autonomous vehicles for lateral maneuvers. IEEE Transactions on Intell. Transp. Syst.11, 132–141 (2010)

    work page 2010

  30. [30]

    How long does it take to stop? methodological analysis of driver perception-brake times.Transp

    Green, M. How long does it take to stop? methodological analysis of driver perception-brake times.Transp. Hum. Factors 2, 195–216 (2000). 32.Treiber, M. & Kesting, A.Traffic Flow Dynamics: Data, Models and Simulation(Springer, 2013)

    work page 2000

  31. [31]

    Surrogate safety measures from traffic simulation models: Final Report

    Federal Highway Administration. Surrogate safety measures from traffic simulation models: Final Report. U.S. Department of Transportation, FHWA-RD-03-050 (2003). Reference inherited from the original K-Risk manuscript; placeholder pending verification

    work page 2003

  32. [32]

    & Knoll, A

    Schöller, C., Aravantinos, V ., Lay, S. & Knoll, A. What the constant velocity model can teach us about pedestrian motion prediction.IEEE Robotics Autom. Lett.5, 1696–1703 (2020). 35.Gilles, T., Schöller, C., Duthon, P., Stanciulescu, B. & Nashashibi, F. A review on motion prediction and risk assessment for intelligent vehicles.IEEE Transactions on Intell...

    work page 2020

  33. [33]

    Marcu, A.-M.et al.LingoQA: Video question answering for autonomous driving.arXiv preprint arXiv:2312.14115(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  34. [34]

    & Guo, F

    Zeng, T., Wu, L., Shi, L., Zhou, D. & Guo, F. Are vision LLMs road-ready? A comprehensive benchmark for safety-critical driving video understanding (DVBench). InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)(2025)

    work page 2025

  35. [35]

    Godbole, M., Gao, X. & Tu, Z. DRAMA-X: A fine-grained intent prediction and risk reasoning benchmark for driving. arXiv preprint arXiv:2506.08188(2025)

    work page arXiv 2025

  36. [36]

    InProceedings of the European Conference on Computer Vision (ECCV)(2024)

    Ljungbergh, W.et al.NeuroNCAP: Photorealistic closed-loop safety testing for autonomous driving. InProceedings of the European Conference on Computer Vision (ECCV)(2024)

    work page 2024

  37. [37]

    Zhang, T.et al.Bench2ADVLM: A closed-loop benchmark for vision-language models in autonomous driving.arXiv preprint arXiv:2510.15793(2025)

    work page arXiv 2025

  38. [38]

    InAdvances in Neural Information Processing Systems (NeurIPS)(2023)

    Rafailov, R.et al.Direct preference optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems (NeurIPS)(2023)

    work page 2023

  39. [39]

    & Chen, D

    Meng, Y ., Xia, M. & Chen, D. SimPO: Simple preference optimization with a reference-free reward. InAdvances in Neural Information Processing Systems (NeurIPS)(2024)

    work page 2024

  40. [40]

    Wang, H.et al.CrashAgent: Crash report driven scenario reconstruction with multi-agent large language models.arXiv preprint arXiv:2505.18341(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  41. [41]

    Zhang, J., Xu, C. & Li, B. ChatScene: Knowledge-enabled safety-critical scenario generation for autonomous vehicles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2024)

    work page 2024

  42. [42]

    InAdvances in Neural Information Processing Systems (NeurIPS)(2022)

    Ouyang, L.et al.Training language models to follow instructions with human feedback. InAdvances in Neural Information Processing Systems (NeurIPS)(2022)

    work page 2022

  43. [43]

    Lambert, N.et al.Tülu 3: Pushing frontiers in open language model post-training.arXiv preprint arXiv:2411.15124 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  44. [44]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Shao, Z.et al.DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024). Acknowledgements This research was conducted by KAIST as part of a joint research project under the Korea Institute of Science and Technology Information (KISTI) R&D program, “Development of the Next-Generation Integrated...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  45. [45]

    IDLE: Remain in the current lane with the current speed (Action ID: 1)

    work page

  46. [46]

    Turn Left: Change to the lane on the left of the current lane (Action ID: 2)

    work page

  47. [47]

    Turn Right: Change to the lane on the right of the current lane (Action ID: 3)

    work page

  48. [48]

    Acceleration: Increase vehicle speed (Action ID: 4)

    work page

  49. [49]

    car_id": 134,

    Deceleration: Reduce vehicle speed (Action ID: 5) A clear action ID must be chosen at the end of your reasoning process. Stage 2: Trajectory record (CSV → JSON event slice).The raw labeled trajectory stores per-frame kinematics to- gether with the eight-surrounding-vehicle role columns (preceding_id, following_id, left_preceding_id, . . ., right_following...

    work page

  50. [50]

    Description of the environment

    work page

  51. [51]

    Your own vehicle's position, dynamic information, and legally permissible actions in that lane

    work page

  52. [52]

    This scenario takes place at an expressway

    Vehicles in your current lane and adjacent lanes, including their position, dynamic information, and legally permissible actions in that lane. This scenario takes place at an expressway. The positive x-coordinate points east, while the positive y-coordinate points south. The top-left corner is the origin (0,0). The velocity components follow the same conv...

    work page

  53. [53]

    IDLE: Remain in the current lane with the current speed

    work page

  54. [54]

    Turn Left: Change to the lane on the left of the current lane

    work page

  55. [55]

    Turn Right: Change to the lane on the right of the current lane

    work page

  56. [56]

    Acceleration: Increase vehicle speed

    work page

  57. [57]

    The structured scenario description of ego vehicle 822 and its neighbors, together with the domain-specific system prompt and its five-action schema, form the LLM input (left)

    Deceleration: Reduce vehicle speed K-Risk Risk Analysis Input Backin Data Wrong Correct Textual Description Input LLM Annotation Figure 9.Worked example of the closed-loop LLM annotation on an ExpresswayA event. The structured scenario description of ego vehicle 822 and its neighbors, together with the domain-specific system prompt and its five-action sch...

    work page