{"id":"0499b09b-af87-4b04-addc-b141f399e71b","arxiv_id":"2602.20575","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A fused, interaction-labeled dataset of 7.31M driving segments with synthetic BEV videos and VQA pairs for training/evaluating driving VLMs.","lead":"This paper builds IEDD, a 7.31-million-segment driving interaction dataset by merging and re-labeling five public trajectory datasets, with synthetic bird's-eye-view videos and question-answer pairs for vision-language driving models. It is a large new resource for training and testing autonomous-driving VLA models, though its interaction labels depend on hand-picked thresholds and its benchmark is self-generated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8) path-consistency metric is dimensionally inconsistent, so the claimed [0,1]-normalized efficiency E_Veh is invalid for all 7.31M segments; this undermines the dataset's efficiency labels before any threshold calibration is considered.","rationale":"The reader's weakest assumption concerns unvalidated thresholds and weights (Table 1), a valid external-calibration issue. However, I identify a more fundamental internal problem: Eq. (8) as written is dimensionally inconsistent, producing E_pi values not bounded to [0,1] for trivial trajectories. This directly affects the claimed efficiency metrics and the VQA ground truth that depends on them. The reader's concern about threshold sensitivity is still important, but the formula error is independently load-bearing because no amount of calibration of Dsearch or weight coefficients can fix a malformed metric. I recommend keeping the CONDITIONAL verdict: the dataset may be usable for interaction mining, but the efficiency labels require correction or verification against the released code. The concrete test would settle whether the manuscript typo or the implementation is at fault.","tokens_in":19927,"tokens_out":5856,"duration_ms":58275,"concrete_test":"Implement Eq. (8) exactly as written and evaluate on a straight-line constant-speed trajectory from (0,0) to (100,0). The numerator is 10,000 and denominator is 100, giving E_pi = 100, not in [0,1]. Then run the same exact formula over a random sample of 1,000 IEDD segments; if any E_pi > 1 or not dimensionless, the E_Veh labels are not normalized. If the released code instead uses sqrt((xi0−xiT)^2 + (yi0−yiT)^2), the manuscript text is erroneous and the paper must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim includes trajectory-derived efficiency metrics (E_Veh) for every IEDD segment. In the Methods section, Eq. (8) defines the path consistency metric as E_pi = [(xi0−xiT)^2 + (yi0−yiT)^2] / ∫_0^T vi(t) dt. As written, the numerator is squared Euclidean distance (units m^2) and the denominator is path length (units m), making E_pi dimensionally inconsistent and not dimensionless. For a straight 100 m trajectory, E_pi = 100, far outside the claimed normalized range [0,1] for E_i (Eq. 7). This implies either the formula is missing a square root (a typo) or the computed E_Veh values are not normalized and cannot support the paper's statements about efficiency scores. Since L3 physical-quantization tasks use these values as ground truth, every such label inherits this error. This is an internal inconsistency independent of external calibration concerns about Table 1 thresholds; even if thresholds are perfectly validated, the efficiency metric as written is invalid.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents IEDD, a large-scale interaction-oriented driving dataset mined from five public trajectory datasets (Lyft Level 5, Waymo, nuPlan, INTERACTION, and SIND), with 7.31 million ego-centric interaction segments, including 6.66 million multi-agent cases. Each segment is annotated with trajectory-derived interaction intensity Q_i and efficiency E_i metrics, and a subset, IEDD-VQA, provides trajectory-reconstructed BEV videos, structured interaction semantics, and multi-turn question-answer pairs. The authors also report a hierarchical VLM evaluation benchmark (L1–L4) and fine-tuning experiments on Qwen2.5-VL-7B.","tokens_in":20336,"tokens_out":3698,"duration_ms":40357,"significance":"If the metric system and thresholds are accepted, IEDD would be a substantial and reusable resource: it integrates five heterogeneous trajectory sources, provides a transparent mining pipeline, and ships public code and data on GitHub/Zenodo. The internal counts in Table 2 are consistent, and the availability of scripts for trajectory-to-VQA generation is a clear strength. However, the claimed quantitative ground truth rests on two unverified pillars: the efficiency metric in Eq. (8) is dimensionally inconsistent as written, and the interaction thresholds/weights in Table 1 are hand-set with no sensitivity analysis or calibration. Because L3 benchmark answers and the fine-tuning experiments are validated against labels generated by the same formulas, the empirical evidence is self-referential at present. The dataset concept is valuable, but the central quantitative labels need correction and independent validation before the descriptor can be accepted.","major_comments":[{"comment":"The path consistency metric is dimensionally inconsistent. Eq. (8) defines E_pi = [(xi0−xiT)^2 + (yi0−yiT)^2] / ∫_0^T vi(t) dt. The numerator has units m^2 and the denominator has units m, so E_pi is not dimensionless and cannot lie in the claimed [0,1] range of Eq. (7). For a straight 100 m trajectory, E_pi = 100. This appears to be a missing square root: the intended ratio d_ref/d_actual would be sqrt((xi0−xiT)^2 + (yi0−yiT)^2) / ∫_0^T vi(t) dt. As written, every E_Veh value in the released dataset and every L3 quantification ground truth derived from Eq. (8) is invalid. The authors must correct the formula and, if E_Veh values were computed with the published formula, regenerate the dataset or explicitly confirm the implemented definition.","section":"Methods, Eq. (8)"},{"comment":"All interaction labels and quantitative metrics depend on hand-set thresholds and weights: Dsearch=2 m, Tsearch=3 s, Twindow=5 s, θmerge=30°, θcross=160°, dinter=50 m, tinter=3 s, scenario-specific weights (e.g., 0.25/0.35/0.40 for merging), and the intensity/efficiency parameters in Table 1. The paper provides no sensitivity analysis, no comparison against human judgments of interaction, and no calibration against established surrogate safety measures. Consequently, the 7.31M count, the type distribution, and every Q_i and E_i label inherit this arbitrariness. At minimum, the authors should report sensitivity of the counts and metric distributions to plausible variations in the thresholds, and provide some external validation (e.g., human-rated interaction samples or comparison with TTC/PET-based conflict classification) to support the claim that these values are physically meaningful g","section":"Table 1 and Methods"},{"comment":"The benchmark and fine-tuning results are self-referential with respect to the dataset's quantitative annotations. L3 MAE is computed against Q_i and E_i values generated by the same formulas (Eqs. 3–10) used to create the training labels, and the L2/L4 evaluation relies on LLM-as-a-Judge (GLM-4.7) with no human validation or inter-annotator agreement reported. The claim that fine-tuning 'learns an internal representation mechanism that maps visual features to physical parameters' is therefore only evidence that the model fits the dataset's internally generated labels. This does not invalidate the dataset's utility, but it does not yet establish transferable physical understanding. The authors should add an independent evaluation component, such as human-annotated held-out samples, external conflict metrics, or a cross-dataset transfer test.","section":"Technical Validation / IEDD-VQA generation"}],"minor_comments":[{"comment":"Several threshold symbols render as '?search' or '??????' in the text (e.g., in the interaction mining paragraph and the classification description). These should be replaced with the actual symbols defined in Table 1.","section":"Methods, threshold symbols"},{"comment":"The rows for ws, wr, wp list three value sets but do not label which row corresponds to which scenario; add a 'Scenario' column or explicit row labels.","section":"Table 1"},{"comment":"The E_Veh field is defined only as 'E values of involved vehicles'. State the formula, units (or normalized range), and which vehicle's efficiency is recorded before the dataset is released.","section":"Data Records / Table 3"},{"comment":"The MAE-to-normalized-score conversion in L3 uses a tolerance threshold of 0.5, but the relevant MAE values in Table 4 are far larger (e.g., 1855.5); explain how the normalization behaves outside the tolerance and whether the L3 score saturates at zero for most models.","section":"Technical Validation, WIS"},{"comment":"Several references are dated 2026 (e.g., References 7 and 57), which is after the current manuscript year; verify the bibliographic data and publication status.","section":"References"},{"comment":"The column headers are split across lines in a way that makes the table hard to read, and MAE values have inconsistent decimal formatting. A cleaner layout with units and consistent precision would help reproducibility.","section":"Tables 4–6"}],"recommendation":"major_revision","confidential_remarks":"This is a potentially useful data resource, but the Eq. (8) dimension error is a hard technical defect in the central quantitative claims, and the absence of threshold sensitivity/calibration further weakens the 'ground truth' status of the labels. The authors should be asked to fix the metric formula, regenerate or confirm released values, and add at least a basic sensitivity analysis and a human- or external-metric validation component. If those are added, the manuscript could become acceptable for a data descriptor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nIEDD is a large, carefully described interaction dataset: 7.31M ego-centric segments mined from five public trajectory datasets, with type labels, intensity/efficiency metrics, and a VQA subset with BEV videos and QA pairs. The counts are internally consistent, the pipeline is transparent, and the authors ship code and data. That is a real contribution — an aggregate artifact, not a new method, but a useful resource for VLA training and evaluation.\n\nThe strongest part is the multi-source fusion and the hierarchical benchmark (L1–L4). The fine-tuning results show concrete numbers, and the zero-shot evaluations of ten VLMs are reproducible in principle. Credit is also due for open-sourcing the generation scripts.\n\nNow the soft spots, in proportion. The biggest is Eq. (8). The path consistency E_pi is defined as squared Euclidean distance divided by path length — numerator has units m^2, denominator m, so E_pi is not dimensionless and not in [0,1]. For a straight 100 m trajectory it evaluates to 100, not 1. Since E_Veh is the product of E_pi, E_ti, E_si (Eq. 7) and is claimed to be normalized to [0,1], every efficiency label used in L3 physical quantization is on the wrong scale. This is not a calibration or threshold issue; it is internal to the formula. If this is a typo (missing square root), the paper and code need to be corrected and the dataset regenerated. As written, the efficiency ground truth is invalid.\n\nSecond, the interaction thresholds in Table 1 are hand-set with no sensitivity analysis or human validation. That affects the interaction counts and type distribution but, importantly, not the dimensional issue. It is a common weakness in this kind of mining paper, and a sensitivity analysis would help.\n\nThird, the VQA ground-truth answers are generated from the same templates and formulas used to validate the models, so the benchmark measures consistency with the pipeline rather than external truth. Acceptable for a resource, but the wording overstates independence.\n\nThe L4 counterfactual test set is only 100 scenarios, which is small for ranking models, though the authors are transparent about that.\n\nWho is this for? Researchers who want a large, interaction-dense trajectory dataset with language supervision and are willing to verify metric correctness themselves. The paper deserves a serious referee: the dataset artifact is potentially valuable, but Eq. (8) must be fixed before the efficiency labels can be used. I would send it out with a request for correction and re-release.\n\nBest.","headline":"A large, transparently built interaction dataset worth a referee's time, but the efficiency metric in Eq. (8) is dimensionally wrong and every E_Veh label inherits the error.","tokens_in":20810,"tokens_out":2680,"would_cite":false,"duration_ms":27535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper presents IEDD, a mining and synthesis pipeline that extracts 7.31 million ego-centric interaction segments from five public trajectory datasets, labels each with physics-based intensity and efficiency scores, and turns a subset i","keywords":["interaction mining","autonomous driving dataset","trajectory analysis","vision-language-action models","bird's-eye-view video","visual question answering","driving interaction intensity","multi-agent scenarios"],"falsifier":"Take a random sample of 200 interaction segments from IEDD, show them to human drivers (blinded to the dataset's labels), and ask them to mark whether an interaction occurred, what type it was, and how intense it felt; if inter-rater agreement with IEDD's labels is low, the mining thresholds do not capture human notions of interaction and the quantitative labels would not support the paper's claimed semantic alignment.","tokens_in":19823,"feed_emoji":"🚗","tokens_out":2672,"duration_ms":29803,"temperature":0.7,"pith_summary":"The paper tries to solve a data bottleneck for autonomous driving: real-world driving logs are dominated by boring, non-interactive driving, while the rare negotiation-heavy events (merging, crossing, head-on encounters, forced yielding) are exactly what Vision-Language-Action models need. It builds a scalable pipeline that homogenizes five heterogeneous trajectory datasets, detects spatio-temporal overlaps between vehicles, classifies each overlap into four interaction types, and aggregates overlapping pairwise events into multi-agent groups. For each extracted segment it computes a two-axis interaction score: an intensity metric combining pose adjustment, risk variation, and potential-field pressure, and an efficiency metric combining path, time, and smoothness. A further stage renders bird's-eye-view videos from the real trajectories and fills pre-defined language templates with the structured semantics, producing the IEDD-VQA instruction set whose language descriptions are strictly tied to physical ground truth. A sympathetic reader would care because the paper offers a low-cost way to turn existing datasets into a large, reusable interaction corpus for training and evaluating driving VLMs.","feed_headline":"7.31 million driving interactions mined from public logs","feed_subtitle":"Trajectories become labeled, VQA-style interaction clips that quantify intensity and efficiency for training driving AI.","key_machinery":"The load-bearing mechanism is the interaction mining and quantification cascade: trajectory cleaning and resampling, spatio-temporal intersection search with a double-pointer sliding window (Dsearch = 2 m, Tsearch = 3 s), two-stage classification into car-following versus merging/crossing/head-on using heading thresholds (30° and 160°), and recursive multi-agent aggregation around anchor vehicles. Intensity is computed as a weighted sum of pose adjustment, TTC/PET variation, and an artificial potential-field term, with scenario-specific weight triples; efficiency is the product of path-, time-, and smoothness-based scores. The synthesis stage converts continuous trajectories into discrete be","core_discovery":"The central claim is that a trajectory-only pipeline can automatically transform heterogeneous naturalistic driving data into a large-scale, interaction-focused multimodal corpus without new sensor collection or manual annotation. The paper reports 7.31 million extracted ego-centric interaction segments, of which 6.66 million involve more than two agents, with a deliberately balanced distribution across head-on, car-following, merging, and crossing behaviors. It further claims that the trajectory-derived intensity and efficiency metrics provide a defensible physical ground truth for language generation, and that IEDD-VQA's four-level structure (perception, description, quantification, reason","pith_inferences":["The hand-set thresholds and metric weights are never validated against human judgments, so the 7.31M count and every VQA label inherit their arbitrariness; a sensitivity analysis varying Dsearch, Tsearch, Twindow, and the intensity weights would reveal whether the interaction distribution is stable or an artifact of parameter choice.","The counterfactual reasoning questions are generated from real trajectories by hypothesizing different ego actions, so they test whether a model can produce linguistically plausible alternatives, not whether those alternatives would physically unfold as described; a causal simulator or trajectory predictor would be needed to make L4 a rigorous physical counterfactual test.","The 'open-source models outperform closed-source models' finding is based on only 100 evaluation scenarios and could shift with a larger, more diverse test set; a stratified sample across the four interaction types and five source datasets would make that claim more robust.","The intensity-efficiency dual metric could be repurposed as a reward signal for closed-loop driving simulation or as a data filter to select high-value segments for human review, but both uses require the metric to first be calibrated against human risk perception."],"forward_implications":["If the pipeline works as described, any trajectory dataset can be turned into interaction-rich VLA training data without new hardware or manual annotation, making large-scale interaction corpora substantially cheaper to produce.","The IEDD-VQA benchmark's four-level structure gives a concrete way to measure where driving VLMs fail: near-perfect language description but severely inaccurate physical quantification, as shown by the large MAE values.","The dataset's 91% multi-agent share directly addresses the sparsity of negotiation scenarios in existing trajectory datasets, potentially enabling models to learn group interactions rather than only pairwise ones.","Fine-tuning on IEDD-VQA can slash physical-quantification error from 1855.55 to 0.30 while raising logical accuracy from 0.15 to 0.53, demonstrating that trajectory-derived metrics are learnable supervision signals.","The observed drop in counterfactual reasoning after fine-tuning warns that domain specialization without task replay trades away general reasoning ability, informing future instruction-tuning designs."],"fun_headline_variants":["7.31M driving interactions mined from public logs","Trajectory-only pipeline yields 7.31M labeled clips","7.31M interaction VQA pairs from existing drives","No new sensors: 7.31M ego clips from public data"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The hand-set thresholds and metric weights in Table 1 determine which trajectory pairs count as interactions and how intense they are, so if those values are arbitrary or miscalibrated, every derived count, type distribution, and VQA ground-truth label inherits that arbitrariness.","fun_headline_variants_meta":{"raw":{"variants":["7.31M driving interactions mined from public logs","Trajectory-only pipeline yields 7.31M labeled clips","7.31M interaction VQA pairs from existing drives","No new sensors: 7.31M ego clips from public data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1110,"prompt_tokens":688,"completion_tokens":422,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":351}},"tokens_in":432,"tokens_out":422,"duration_ms":5012,"temperature":1.0,"reasoning_tokens":351,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T21:16:28.484955+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of 200 interaction segments from IEDD, show them to human drivers (blinded to the dataset's labels), and ask them to mark whether an interaction occurred, what type it was, and how intense it felt; if inter-rater agreement with IEDD's labels is low, the mining thresholds do not capture human notions of interaction and the quantitative labels would not support the paper's claimed semantic alignment.","supporting_citations":[],"review_version":1}