Pith. sign in

REVIEW 3 major objections 5 minor 48 references

A video-based GAN can generate realistic traffic trajectories for larger scenes at under 20 ms inference, the authors show.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:52 UTC pith:BMIUQHEF

load-bearing objection A useful incremental extension of the authors' videoGAN trajectory pipeline, but the central scalability claim is undercut by boundary-excluded disappearing rates in Table 1 that the paper never addresses. the 3 major comments →

arxiv 2608.02289 v1 pith:BMIUQHEF submitted 2026-08-03 cs.CV cs.LG

Extended Field of View Analysis for VideoGAN-based Trajectory Generation

Annajoyce Mariani , Kira Maag , Hanno Gottschalk This is my paper
classification cs.CV cs.LG
keywords trajectory generationvideo GANbird's-eye viewautonomous drivingobject permanencehallucination metricsfield of viewsemantic rasterization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that video generative adversarial networks trained on semantic bird's-eye-view traffic videos are not limited to small scene crops: they can scale to larger and more complex fields of view while still producing statistically realistic, safe-feeling multi-agent trajectories. To get there, the authors replace the previous HSV color encoding with a Lab-space encoding, switch trajectory recovery to a graph-based overlap association, and add quantitative metrics for hallucinations such as vehicles appearing, disappearing, merging, or splitting. Across 15 m, 20 m, and 25 m longitudinal fields of view, the distributions of relative speed, acceleration, inter-vehicle distance, and time-to-collision in generated scenes align with real scenes, with only slightly worsened statistics at the largest FOV. If true, this makes video-GAN scene generation a practical low-latency engine for automated-driving simulation, prediction, and planning.

Core claim

On the paper's own terms, the central discovery is that the extended pipeline—Lab-color semantic rasterization, a low-resolution video GAN, and graph-based trajectory extraction—generalizes from the previously studied small field of view to longitudinal ranges of 15 m, 20 m, and 25 m. Over 200 generated and 200 real scenes, the model produces vehicle counts, object sizes, nearest-neighbor distances, relative speed and acceleration distributions, and time-to-collision statistics that match the training data closely; generated videos rarely show merging or splitting vehicles, and most appearing/disappearing events occur at frame boundaries. Inference stays below 20 ms for 20-second scenes. The

What carries the argument

The load-bearing objects are: (1) a Lab color-space encoding that separates empty space, road markings, traffic-light states, ego vehicle, and other vehicles into distinct, easily filterable color ranges; (2) the low-resolution branch of a long-video GAN, which generates dynamically coherent semantic videos in a single forward pass; and (3) a graph-based trajectory extractor in which each detected object is a node and edges connect overlapping same-type objects in adjacent frames, with node in/out degree anomalies used to count appearing, disappearing, merging, and splitting hallucinations. The graph carries the argument because every trajectory statistic—speeds, accelerations, spacings, tim

Load-bearing premise

The paper assumes the hand-chosen Lab color ranges used to rasterize training videos will still cleanly identify vehicles and traffic lights in generated videos, even though the authors observe traffic-light morphing and vehicle color shifts in generated frames—if that color code drifts, every trajectory statistic inherits detection errors.

What would settle it

Manually annotate vehicle bounding boxes in a sample of generated 25 m-FOV videos, run the paper's Lab-threshold and graph-association pipeline, and compare the resulting speed, acceleration, and time-to-collision distributions to those computed from the manual boxes; a material divergence—especially around the color-shifted vehicles the authors report—would show the reported realism is an artifact of the extraction pipeline rather than the generator.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the central claim holds, video-GAN scene generation is a viable low-latency alternative to diffusion-based world models for trajectory proposal: roughly 20 ms per 20-second scene versus seconds to tens of seconds.
  • The graph-based extraction and hallucination metrics give a concrete way to audit object permanence in generated traffic videos, not just visual fidelity.
  • The statistical alignment of speed, acceleration, and time-to-collision distributions suggests the generated scenes can serve as training or testing data for downstream prediction and planning modules without introducing systematic unsafe-biased trajectories.
  • Scaling from 15 m to 25 m fields of view works with only mildly degraded statistics, indicating that larger-context modeling is within reach of the same architecture.
  • Traffic-light interactions—stopping at red, crossing green—are reproduced well enough that generated scenes preserve rule-relevant behavior.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the paper's logic: because the trajectory statistics are extracted by color thresholding, the reported distributional matches upper-bound the generator's realism; a more robust instance-aware detector could either confirm or raise the current numbers.
  • The same graph-based hallucination accounting could be applied to other generative world models, giving a common yardstick for object permanence across video-generation paradigms.
  • The authors' observation that parked-car scenes are never generated hints at a mode-coverage limitation; conditioning the generator on map or scenario context is a natural next step suggested by, but not tested in, this paper.
  • If the approach transfers to denser urban datasets beyond the one studied here, it would strengthen the scalability claim; that transfer is an open empirical question rather than a demonstrated result.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper extends a videoGAN-based trajectory proposal framework to larger bird's-eye-view fields of view (15 m, 20 m, 25 m) using Waymo Open Motion Dataset. It introduces a Lab color-space semantic representation, a graph-based trajectory extraction method, and a quantitative hallucination metric measuring appearing, disappearing, merging, and splitting events. The authors claim that the framework generalizes to larger scenes while maintaining statistically realistic trajectories and coherent spatial relationships, with inference times below 20 ms. The evaluation compares distributions of object sizes, counts, distances, speeds, accelerations, trajectory lengths, and time-to-collision between real and generated videos.

Significance. If the central claim holds, this is a useful demonstration that lightweight video GANs can generate scalable, semantically grounded traffic scenes for AD simulation and prediction. The paper's strengths include public release of code and trained models, low-latency inference, and a concrete evaluation framework for object permanence. However, the quantitative evaluation is built entirely on a color-threshold detection pipeline whose reliability on generated frames is not established, and the reported hallucination numbers contain internal inconsistencies. The paper is therefore a promising empirical contribution but currently does not fully support its headline claims of statistically realistic and scalable trajectory generation.

major comments (3)
  1. [§4.3, Table 1] The text states that splitting/merging rates are 'lower than 1% at FOV 15m and 20m, and marginally higher rates for FOV 25m', but Table 1 lists Merging rates of 0%, 1%, 4% and Splitting rates of 0%, 1%, 6% for generated videos. More seriously, the boundary-excluded Disappearing rates are 33%, 45.5%, and 55% for generated videos versus 2%, 1%, and 0% for real videos. The authors attribute disappearing events to agents leaving the FOV, but the boundary-excluded column is specifically designed to remove that explanation. These numbers are not consistent with the claim that object permanence is maintained, and they indicate that the tracker loses objects inside the scene, most plausibly due to color drift in generated frames. Because all trajectory statistics in §4.4–§4.6 are computed from this tracker, the reported distributions may be based on incomplete object sets.
  2. [§3.3, §4.2] The object detector uses the same hand-chosen Lab color ranges that were used to rasterize the training data. The paper itself acknowledges that non-ego vehicles exhibit occasional color shifts and that traffic lights morph when vehicles pass under them. No quantitative validation of detection accuracy on generated frames is provided; Figure 5 shows only hand-picked examples. The boundary-excluded disappearing behavior in Table 1 is consistent with generated objects drifting outside the detection thresholds, which would make the trajectory statistics in §4.4–§4.6 artifacts of the extraction pipeline rather than properties of the generator. The authors should demonstrate detection reliability on generated frames—for example, by comparing extracted trajectories from real videos against ground-truth object lists, or by using a held-out detection method—or substantially soften claims about t
  3. [§4.4–§4.6] The central claim of 'statistically realistic trajectories' is supported only by visual overlap of histograms. No distributional distance (e.g., KL or Wasserstein), confidence interval, significance test, or error bar is reported. The paper uses 100–200 scenes per condition but does not state variance across model checkpoints or seeds. Given that the abstract and conclusions make explicit statistical claims, the authors should add quantitative distributional comparisons and uncertainty estimates. This is especially important because the detection-pipeline issues above could differentially affect the generated and real distributions.
minor comments (5)
  1. [§1] Typo: 'could could not guarantee' should be 'could not guarantee'.
  2. [§4.5] Typos: 'logaritmic' should be 'logarithmic', and 'A verage' in §2.4 should be 'Average'.
  3. [Abstract vs. §4.1] The abstract reports '150 GPU hours' of training while §4.1 reports '140 GPU hours'; these should be reconciled.
  4. [References] References [34] and [35] appear to be the same paper by Saxena and Cao; duplicate references should be removed or distinguished.
  5. [§4.3] The term 'rates' is ambiguous: Table 1 reports the percentage of videos containing at least one event, while the text speaks of rates as if they were per-frame or per-trajectory frequencies. Please define the denominator explicitly.

Circularity Check

0 steps flagged

No significant circularity: the central claims are supported by distributional comparisons against external Waymo data, not by self-referential construction.

full rationale

The paper's central claim is that a videoGAN-based BEV trajectory generation framework generalizes to larger fields of view while maintaining statistically realistic trajectories and coherent spatial relationships. The support for this claim consists of distributional comparisons of speeds, accelerations, inter-vehicle distances, time-to-collision, and traffic-light-relative speeds between generated videos and real videos rasterized from the external Waymo Open Motion Dataset. No parameter is fitted to a target metric and then re-reported as a prediction: the quantitative evaluation is a standard generative-model fidelity check against an external dataset. The use of the same Lab color ranges for rasterization (Sec. 3.1) and object detection (Sec. 3.3) is a shared encoding choice, not a definitional reduction; it could affect evidence quality if generated colors drift, but that is a robustness/correctness concern, not circularity. The self-citation of prior work [28] supplies the baseline pipeline and training recipe, but the new contributions—larger FOVs, Lab encoding, graph-based trajectory extraction, and hallucination metrics—are evaluated independently against real data and against the externally developed LongVideoGAN backbone [3]. No uniqueness theorem is imported from the authors, no ansatz is smuggled in via citation, and no known result is merely renamed. The paper even discloses its own limitations, such as occasional color shifts and traffic-light morphing (Sec. 4.2), and Table 1's elevated boundary-excluded disappearing rates are an empirical evidence concern rather than a case of conclusions being assumed in the inputs. I find no step in the derivation chain that reduces by definition or by self-citation to its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

This is an empirical engineering paper, so the central claim rests on domain assumptions and hand-set parameters rather than mathematical axioms. No new physical entities are introduced. The most consequential ad hoc degrees of freedom are the color codebook, the checkpoint selection, and the boundary-exclusion threshold, all of which directly change the reported quality numbers.

free parameters (4)
  • Semantic Lab color ranges per category = L∈[30,90] centerline; black ego; blue-pink others; ellipsoidal light ranges
    Chosen by hand to maximize category separation (§3.1) and used identically for rasterization and detection (§3.3); extraction accuracy is tied to these arbitrary ranges.
  • Centerline thickness and traffic light size = 1.5 m; 2×2 m
    Arbitrary rasterization choices in §3.1 that affect object footprints and inpainting artifacts; footprint distributions partly reflect these choices.
  • Boundary-exclusion fraction = outer 10%
    Post-hoc threshold used to redefine hallucination counts in §4.3/Table 1; moving this threshold changes real-vs-generated rates substantially.
  • Checkpoint selection epoch = 20/23/24 epochs (per FOV)
    Best checkpoints selected by visual inspection (§4.2); all reported metrics come from these selected checkpoints rather than a pre-registered criterion.
axioms (5)
  • domain assumption Waymo Open Motion Dataset is representative of real traffic for the purpose of trajectory realism.
    Both training and reference distributions are Waymo-derived (§4.1); no external traffic data validates generality.
  • domain assumption Low-resolution semantic BEV videos (54×96 px, up to 25 m) retain enough information for realistic trajectory extraction.
    The pipeline's ceiling is set by the rasterization; any loss of geometry or dynamics is invisible to both generator and evaluator (§3.1).
  • domain assumption Graph association via reciprocal bounding-box overlap in adjacent frames recovers true object identities.
    Relies on 100 ms frame spacing and per-frame displacement less than vehicle length (§3.4); failure would corrupt all trajectory statistics.
  • domain assumption Traffic lights are static reference points in generated videos.
    Explicitly assumed in §4.6 to compute vehicle speeds relative to lights; authors also observe light morphing, making this partially fragile.
  • ad hoc to paper Histogram matching of marginal distributions is sufficient evidence of statistical realism.
    The evaluation framework defines realism as visual alignment of generated/real distributions; no formal metric or significance test is used.

pith-pipeline@v1.3.0-daily-deepseek · 15232 in / 12657 out tokens · 103498 ms · 2026-08-04T09:52:45.819007+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Extended Field of View Analysis for VideoGAN-based Trajectory Generation." pith.science (2026). https://pith.science/paper/BMIUQHEF

@misc{pith2026260802289,
  author       = {Pith},
  title        = {Pith review of: Extended Field of View Analysis for VideoGAN-based Trajectory Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BMIUQHEF}},
  note         = {Machine review of arXiv:2608.02289}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Realistic and diverse trajectory generation is central to enabling higher levels of vehicle automation. While rule-based and classical learning-based methods may struggle to capture the complexity of traffic behavior, generative models have already demonstrated in other fields that they can handle a comparable level of complexity. In this paper, we build upon previous work on generative adversarial network (GAN)-based semantic bird's-eye-view traffic generation and extend the proposed framework in several key aspects. We improve the semantic representation, replace the trajectory extraction procedure with a graph-based association method, and systematically investigate increasingly larger fields of view. In addition, we introduce a quantitative evaluation framework to assess hallucinations and object permanence in generated videos. Our experiments demonstrate that the framework generalizes to larger and more complex traffic scenes while maintaining statistically realistic trajectories and coherent spatial relationships between traffic participants. Within 150GPU hours of training and with inference times below 20ms for scenes of up to 20s, our results demonstrate that video-based GANs remain an efficient and scalable approach for realistic trajectory generation, even in substantially larger traffic scenes, making them well suited for downstream tasks such as prediction, planning, and simulation in automated driving.

Figures

Figures reproduced from arXiv: 2608.02289 by Annajoyce Mariani, Hanno Gottschalk, Kira Maag.

Figure 1
Figure 1. Figure 1: 3D representation of a scene from the Waymo Open Motion Dataset [38]. Image is extracted from [28]. AD system in the perception module [12]. However, the real decision making center of an automated vehicle, where the versatility of DL can make a differ￾ence between stiff and hesitant driving in complex and interactive environments, lies in the prediction and planning subtasks (see [PITH_FULL_IMAGE:figures… view at source ↗
Figure 2
Figure 2. Figure 2: A schematic illustration of the pipeline. The abstract trajectory data is ras￾terized into low-resolution BEV semantic videos of traffic scenes. A videoGAN model, which consists of a generator (G) that attempts to generate realistic data, and a dis￾criminator (D) that distinguishes real data from generated data, is trained on the semantic scenes. Last, abstract trajectories are extracted from the generated… view at source ↗
Figure 3
Figure 3. Figure 3: Four slices of the Lab space represented in RGB. we do not identify a compelling advantage of using a non-color encoding, and the employed videoGAN has demonstrated strong performance on RGB video generation [3], we adopt an RGB-based representation throughout this work. To preserve the semantic identity of the different scene elements, we employ a color encoding that maximizes the separation between seman… view at source ↗
Figure 4
Figure 4. Figure 4: Some examples of frames from real traffic scenes (a,b,c) and from generated ones (d,e,f) for the three FOVs investigated in this work. The scenes are represented as occupancy grid videos in BEV perspective, with road users as 2D Bounding Boxes, traffic lights as green, yellow or red circles, and road as gray center lines. [28]) provide the same degree of separation, whereas Lab enables a more distinct cate… view at source ↗
Figure 5
Figure 5. Figure 5: Two sets of frames displaying the frame by frame extraction process and accu￾racy. The first set is from a real video, the second from a generated video. For each set, moving left to right, we see: the original frame, the mask for traffic lights, the mask for vehicles, and original image overlaid with the pipeline prediction of the center of the object. The object detection pipeline is fairly accurate even… view at source ↗
Figure 6
Figure 6. Figure 6: An example of trajectory graph extracted from a real video. The y-axis rep￾resents a combination of longitudinal and lateral position, such that higher distances between two objects are represented by higher distances in position. The black line in the center corresponds to the ego vehicle, whose position remains stable in the center of the scene across time. In the beginning of the scene, three green ligh… view at source ↗
Figure 7
Figure 7. Figure 7: Examples from progressing training iterations at FOV 20 m (a) 15 m (b) 20 m (c) 20 m (d) 15 m (e) 25 m [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Significant frames from generated videos of traffic scenes of different fields of view. The agents are observed waiting at a red light (a), crossing (b) and passing (c) a green light, approaching a yellow light (d) and moving in dense traffic (e) 4.2 Quality of the Generated Videos At https://www.youtube.com/watch?v=7BAulb8rGsU we provide a selection of real and generated video clips for the three differen… view at source ↗
Figure 9
Figure 9. Figure 9: Left: Relevant frames from an example of merging hallucination observed at an early training stage. The frames are extracted around timestep t*. Right: the extracted graph of the objects in the scene. Notice that around t* two trajectories merge into one, causing one node to have two input nodes. In addition to this, the generated videos feature diverse and accurate behavior of the ego agent with respect t… view at source ↗
Figure 10
Figure 10. Figure 10: Distribution of appearing and disappearing events for FOV 15 m, calculated over 100 real and 100 generated scenes. (a) FOV 15 m (b) FOV 20 m (c) FOV 25 m [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Distribution of object footprint areas in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m. The peak around 12 m2 corresponds to the average area of a car. 4.4 Quantitative Results: Frame-level We quantitatively evaluate the observations above by analyzing the distribution alignment for different geometrical parameters extracted from the generated and real videos… view at source ↗
Figure 12
Figure 12. Figure 12: Distribution of the distances between the two nearest vehicles in a scene, in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m (a) FOV 15 m (b) FOV 20 m (c) FOV 25 m [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Distribution of the amounts of vehicles in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m. The amount of vehicles in scenes of FOV 15 ranges between 1 and 5, while it reaches up to 10 for scenes of FOV 25. above. As observed above, this is mostly significant for vehicles waiting at a red traffic light [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Distribution of the amounts of traffic lights in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m. In all three cases, most of the frames do not display traffic lights, and the ones who do have a preference for clustering in groups of around 3. 4.5 Quantitative Results: Dynamics [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Distribution of vehicle speeds in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m, presented in logaritmic scale. The significant peak around 0 correspond to vehicles moving at the same speed as the ego vehicle (comoving traffic). (a) FOV 15 m (b) FOV 20 m (c) FOV 25 m [PITH_FULL_IMAGE:figures/full_fig_p018_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Distribution of vehicle accelerations in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m, presented in logaritmic scale. 4.6 Quantitative Results: Traffic Lights Since the model is unconditional, the generated map geometry may exhibit small local variations over time. To avoid introducing unnecessary uncertainty when estimating vehicle motion, we compute vehicle… view at source ↗
Figure 17
Figure 17. Figure 17: Distribution of the lengths of real and generated trajectories as extracted from the videos for the three different investigated FOVs of 15 m, 20 m and 25 m. (a) FOV 15 m (b) FOV 20 m (c) FOV 25 m [PITH_FULL_IMAGE:figures/full_fig_p019_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Distribution of time-to-collision values for real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m. This is particularly true for lower FOVs, while the higher FOVs seem to display a slightly higher probability for the agent to be moving. Also for the distribution of speeds near green traffic lights, in [PITH_FULL_IMAGE:figures/full_fig_p019_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Distribution of vehicle speeds in the proximity of a red light, in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m. (a) FOV 15 m (b) FOV 20 m (c) FOV 25 m [PITH_FULL_IMAGE:figures/full_fig_p020_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Distribution of vehicle speeds in the proximity of a green light, in real and generated scenes, for the three different investigated FOVs of 15 m, 20 m and 25 m. 5 Conclusions and Outlook In this work, we expanded upon [28] to investigate a trajectory proposal pipeline based on a video generative model which is effective at learning the dynamics of traffic scenes in a purely visual fashion, without extern… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 4 canonical work pages · 1 internal anchor

  1. [1]

    https://doi.org/10.48550/arXiv.2307.07527

    Bharilya, V., Kumar, N.: Machine Learning for Autonomous Vehicle’s Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions (Jul 2023). https://doi.org/10.48550/arXiv.2307.07527

    Pith/arXiv arXiv doi:10.48550/arxiv.2307.07527 2023

  2. [2]

    arXiv preprint arXiv:2311.15127 (2023)

    Blattmann, A., Dockhorn, T., Kulal, S., Mendelevitch, D., Kilian, M., Lorenz, D., Levi, Y., English, Z., Voleti, V., Letts, A., et al.: Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 (2023)

    Pith/arXiv arXiv 2023

  3. [3]

    In: 36th Conference on Neural Information Processing Systems (2022)

    Brooks, T., Hellsten, J., Aittala, M., Wang, T.C., Aila, T., Lehtinen, J., Liu, M.Y., Efros, A.A., Karras, T.: Generating Long Videos of Dynamic Scenes. In: 36th Conference on Neural Information Processing Systems (2022)

    2022

  4. [4]

    Geoinformatica (Apr 2025)

    Chen, X., Huang, C., Wang, C., Chen, L.: Trajectory generation: A survey on methods and techniques. Geoinformatica (Apr 2025). https://doi.org/10.1007/ s10707-025-00545-z

    2025

  5. [5]

    Clark, A., Donahue, J., Simonyan, K.: Adversarial video generation on complex datasets (2019), https://arxiv.org/abs/1907.06571

    Pith/arXiv arXiv 2019

  6. [6]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Cui, A., Casas, S., Sadat, A., Liao, R., Urtasun, R.: LookOut: Diverse Multi- Future Prediction and Planning for Self-Driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 16107–16116 (2021)

    2021

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Fang, L., Jiang, Q., Shi, J., Zhou, B.: TPNet: Trajectory Proposal Network for Motion Prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    2020

  8. [8]

    In: IEEE Int

    Gilles, T., Sabatini, S., Tsishkou, D., Stanciulescu, B., Moutarde, F.: HOME: Heatmap Output for future Motion Estimation. In: IEEE Int. Intell. Transp. Syst. Conf. pp. 500–507 (2021). https://doi.org/10.1109/ITSC48978.2021.9564944

    arXiv 2021

  9. [9]

    Journal of Field Robotics 37(3), 362–386 (Apr 2020)

    Grigorescu, S., Trasnea, B., Cocias, T., Macesanu, G.: A Survey of Deep Learning Techniques for Autonomous Driving. Journal of Field Robotics 37(3), 362–386 (Apr 2020). https://doi.org/10.1002/rob.21918

    doi:10.1002/rob.21918 2020

  10. [10]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Gu, T., Chen, G., Li, J., Lin, C., Rao, Y., Zhou, J., Lu, J.: Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17113–17122 (2022)

    2022

  11. [11]

    In: Proceedings of 22 A

    Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., Alahi, A.: Social GAN: Socially Acceptable Trajectories With Generative Adversarial Networks. In: Proceedings of 22 A. Mariani et al. the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2255–2264 (2018)

    2018

  12. [12]

    In: Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation Systems

    Horgan, J., Hughes, C., McDonald, J., Yogamani, S.: Vision-Based Driver As- sistance Systems: Survey, Taxonomy and Advances. In: Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation Systems. pp. 2032–2039. ITSC ’15, IEEE Computer Society, USA (Sep 2015). https://doi.org/ 10.1109/ITSC.2015.329

    work page doi:10.1109/itsc.2015.329 2015

  13. [13]

    In: 2021 IEEE/CVF International Conference on Com- puter Vision (ICCV)

    Hu, A., Murez, Z., Mohan, N., Dudas, S., Hawke, J., Badrinarayanan, V., Cipolla, R., Kendall, A.: FIERY: Future Instance Prediction in Bird’s-Eye View from Sur- round Monocular Cameras. In: 2021 IEEE/CVF International Conference on Com- puter Vision (ICCV). pp. 15253–15262. IEEE, Montreal, QC, Canada (Oct 2021). https://doi.org/10.1109/ICCV48922.2021.01499

    arXiv 2021

  14. [14]

    In: A vidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T

    Hu, S., Chen, L., Wu, P., Li, H., Yan, J., Tao, D.: ST-P3: End-to-End Vision- Based Autonomous Driving via Spatial-Temporal Feature Learning. In: A vidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. pp. 533–549. Springer Nature Switzerland, Cham (2022). https: //doi.org/10.1007/978-3-031-19839-7_31

    doi:10.1007/978-3-031-19839-7_31 2022

  15. [15]

    Sustainability 15(20), 14716 (Oct 2023)

    Huang, R., Zhuo, G., Xiong, L., Lu, S., Tian, W.: A Review of Deep Learning- Based Vehicle Motion Prediction for Autonomous Driving. Sustainability 15(20), 14716 (Oct 2023). https://doi.org/10.3390/su152014716

    work page doi:10.3390/su152014716 2023

  16. [16]

    Vision-based Multi-future Trajectory Prediction: A Survey

    Huang, R., Xue, H., Pagnucco, M., Salim, F., Song, Y.: Multimodal Trajectory Prediction: A Survey (Feb 2023). https://doi.org/10.48550/arXiv.2302.10463

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2302.10463 2023

  17. [17]

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., Wang, Y., Chen, X., Wang, L., Lin, D., Qiao, Y., Liu, Z.: Vbench: Comprehensive benchmark suite for video generative models (2023), https: //arxiv.org/abs/2311.17982

    Pith/arXiv arXiv 2023

  18. [18]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., Wang, Y., Chen, X., Wang, L., Lin, D., Qiao, Y., Liu, Z.: Vbench: Comprehensive benchmark suite for video generative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 21807–21818 (June 2024)

    2024

  19. [19]

    IEEE Trans

    Jia, X., Wu, P., Chen, L., Liu, Y., Li, H., Yan, J.: HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encoding. IEEE Trans. Pattern Anal. Mach. Intell. 45(11), 13860–13875 (Nov 2023). https: //doi.org/10.1109/TPAMI.2023.3298301

    arXiv 2023

  20. [20]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

    Jiang, C.M., Cornman, A., Park, C., Sapp, B., Zhou, Y., Anguelov, D.: MotionDif- fuser: Controllable Multi-Agent Motion Prediction Using Diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

    2023

  21. [21]

    In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems

    Jiao, R., Wang, Y., Liu, X., Zhan, S.S., Huang, C., Zhu, Q.: Kinematics-aware Trajectory Generation and Prediction with Latent Stochastic Differential Mod- eling. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 565–572. IEEE, Abu Dhabi, United Arab Emirates (Oct 2024). https://doi.org/10.1109/IROS58592.2024.10802438

    arXiv 2024

  22. [22]

    ACM Comput

    Khan, M.A., Sayed, H.E., Malik, S., Zia, T., Khan, J., Alkaabi, N., Ignatious, H.: Level-5 Autonomous Driving—Are We There Yet? A Review of Research Lit- erature. ACM Comput. Surv. 55(2), 1–38 (Feb 2023). https://doi.org/10.1145/ 3485767

    2023

  23. [23]

    Mathematics 9(6), 660 (Jan 2021)

    Leon, F., Gavrilescu, M.: A Review of Tracking and Trajectory Prediction Methods for Autonomous Driving. Mathematics 9(6), 660 (Jan 2021). https://doi.org/10. 3390/math9060660 Extended Field of View Analysis for VideoGAN-based Trajectory Generation 23

    2021

  24. [24]

    Accident Analysis and Prevention 204, 107649 (2024)

    Li, S., Anis, M., Lord, D., Zhang, H., Zhou, Y., Ye, X.: Beyond 1d and over- simplified kinematics: A generic analytical framework for surrogate safety mea- sures. Accident Analysis and Prevention 204, 107649 (2024). https://doi.org/https: //doi.org/10.1016/j.aap.2024.107649

    arXiv 2024

  25. [25]

    https://doi.org/10.48550/arXiv.2502.17100

    Li, Y., Shao, X., Zhang, J., Wang, H., Brunswic, L.M., Zhou, K., Dong, J., Guo, K., Li, X., Chen, Z., Wang, J., Hao, J.: Generative Models in Decision Making: A Survey (Mar 2025). https://doi.org/10.48550/arXiv.2502.17100

    doi:10.48550/arxiv.2502.17100 2025

  26. [26]

    In: Yue, Y., Garg, A., Peng, N., Sha, F., Yu, R

    Lyu, Z., Si, C., Song, J., Yang, Z., Qiao, Y., Liu, Z., Wong, K.Y.K.: Fastercache: Training-free video diffusion model acceleration with high quality. In: Yue, Y., Garg, A., Peng, N., Sha, F., Yu, R. (eds.) International Conference on Learning Representations. vol. 2025, pp. 33132–33156 (2025)

    2025

  27. [27]

    IEEE Robot

    Mahjourian, R., Kim, J., Chai, Y., Tan, M., Sapp, B., Anguelov, D.: Occupancy Flow Fields for Motion Forecasting in Autonomous Driving. IEEE Robot. Autom. Lett. 7(2), 5639–5646 (Apr 2022). https://doi.org/10.1109/LRA.2022.3151613

    arXiv 2022

  28. [28]

    In: Proceedings of the 18th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART

    Mariani, A., Maag, K., Gottschalk, H.: Videogan-based trajectory proposal for automated vehicles. In: Proceedings of the 18th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART. pp. 1149–1156. INSTICC, SciTePress (2026). https://doi.org/10.5220/0014229000004052

    work page doi:10.5220/0014229000004052 2026

  29. [29]

    In: International Conference on Learning Representations (Oct 2021)

    Ngiam, J., Vasudevan, V., Caine, B., Zhang, Z., Chiang, H.T.L., Ling, J., Roelofs, R., Bewley, A., Liu, C., Venugopal, A., Weiss, D.J., Sapp, B., Chen, Z., Shlens, J.: Scene Transformer: A unified architecture for predicting future trajectories of multiple agents. In: International Conference on Learning Representations (Oct 2021)

    2021

  30. [30]

    In: International Conference on Learning Representations (2023)

    Park, D., Ryu, H., Yang, Y., Cho, J., Kim, J., Yoon, K.J.: Leveraging Future Rela- tionship Reasoning for Vehicle Trajectory Prediction. In: International Conference on Learning Representations (2023)

    2023

  31. [31]

    In: International Conference on Learning Repre- sentations (Sep 2019)

    Rhinehart, N., McAllister, R., Levine, S.: Deep Imitative Models for Flexible In- ference, Planning, and Control. In: International Conference on Learning Repre- sentations (Sep 2019)

    2019

  32. [32]

    IEEE Robotics and Automation Let- ters 5(2), 2816–2823 (Apr 2020)

    Ridel, D., Deo, N., Wolf, D., Trivedi, M.: Scene Compliant Trajectory Forecast With Agent-Centric Spatio-Temporal Grids. IEEE Robotics and Automation Let- ters 5(2), 2816–2823 (Apr 2020). https://doi.org/10.1109/LRA.2020.2974393

    arXiv 2020

  33. [33]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)

    2022

  34. [35]

    ACM Computing Surveys 54(3), 1–42 (2022)

    Saxena, D., Cao, J.: Generative Adversarial Networks (GANs): Challenges, So- lutions, and Future Directions. ACM Computing Surveys 54(3), 1–42 (2022). https://doi.org/10.1145/3446374

    doi:10.1145/3446374 2022

  35. [36]

    Advances in Neural Information Processing Systems 35, 6531–6543 (Dec 2022)

    Shi, S., Jiang, L., Dai, D., Schiele, B.: Motion Transformer with Global Intention Localization and Local Movement Refinement. Advances in Neural Information Processing Systems 35, 6531–6543 (Dec 2022)

    2022

  36. [37]

    Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., Parikh, D., Gupta, S., Taigman, Y.: Make-a-video: Text-to- video generation without text-video data (2022), https://arxiv.org/abs/2209.14792

    Pith/arXiv arXiv 2022

  37. [38]

    Mariani et al

    Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., Vasudevan, V., Han, W., Ngiam, J., Zhao, H., Timofeev, A., Ettinger, S., Krivokon, M., Gao, A., Joshi, A., Zhang, Y., Shlens, J., 24 A. Mariani et al. Chen, Z., Anguelov, D.: Scalability in perception for autonomous driving: Waymo open data...

    2020

  38. [40]

    Unterthiner, T., van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., Gelly, S.: Towards accurate generative models of video: A new metric and challenges (2019), https://arxiv.org/abs/1812.01717

    Pith/arXiv arXiv 2019

  39. [41]

    IEEE Transactions on Image Processing 35, 4947–4960 (2026)

    Wang, L., Zheng, W., Ren, Y., Jiang, H., Cui, Z., Yu, H., Lu, J.: Occsora: 4d occupancy generation models as world simulators for autonomous driving. IEEE Transactions on Image Processing 35, 4947–4960 (2026). https://doi.org/10.1109/ TIP.2026.3687468

    arXiv 2026

  40. [42]

    arXiv preprint arXiv:2505.15863 (2025)

    Winter, K., Vivekanandan, A., Polley, R., Shen, Y., Schlauch, C., Bouzidi, M.K., Derajic, B., Grabowsky, N., Mariani, A., Rochau, D., et al.: Generative ai for autonomous driving: A review. arXiv preprint arXiv:2505.15863 (2025)

    Pith/arXiv arXiv 2025

  41. [43]

    In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wu, P., Chen, S., Metaxas, D.N.: MotionNet: Joint Perception and Motion Predic- tion for Autonomous Driving Based on Bird’s Eye View Maps. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11382–11392. IEEE, Seattle, W A, USA (Jun 2020). https://doi.org/10.1109/CVPR42600.2020.01140

    arXiv 2020

  42. [44]

    ACM Comput

    Xing, Z., Feng, Q., Chen, H., Dai, Q., Hu, H., Xu, H., Wu, Z., Jiang, Y.G.: A Survey on Video Diffusion Models. ACM Comput. Surv. 57(2), 1–42 (Feb 2025). https://doi.org/10.1145/3696415

    doi:10.1145/3696415 2025

  43. [45]

    https://doi.org/10.48550/arXiv.2503

    Xiong, W., Chen, J., Qi, Z.: Fine-Grained Behavior and Lane Constraints Guided Trajectory Prediction Method (Apr 2025). https://doi.org/10.48550/arXiv.2503. 21477

    doi:10.48550/arxiv.2503 2025

  44. [46]

    In: Kim, B., Yue, Y., Chaudhuri, S., Fragkiadaki, K., Khan, M., Sun, Y

    Yu, S., Nie, W., Huang, D.A., Li, B., Shin, J., anandkumar, a.: Efficient video diffu- sion models via content-frame motion-latent decomposition. In: Kim, B., Yue, Y., Chaudhuri, S., Fragkiadaki, K., Khan, M., Sun, Y. (eds.) International Conference on Learning Representations. vol. 2024, pp. 9642–9670 (2024)

    2024

  45. [47]

    In: Belgrave, D., Zhang, C., Lin, H., Pascanu, R., Koniusz, P., Ghassemi, M., Chen, N

    Zhang, P., Chen, Y., Huang, H., Lin, W., Liu, Z., Stoica, I., Xing, E., Zhang, H.: Faster video diffusion with trainable sparse attention. In: Belgrave, D., Zhang, C., Lin, H., Pascanu, R., Koniusz, P., Ghassemi, M., Chen, N. (eds.) Advances in Neural Information Processing Systems. vol. 38, pp. 152509–152534. Curran Associates, Inc. (2025)

    2025

  46. [48]

    IEEE Trans

    Zhao, C., Zhu, Y., Du, Y., Liao, F., Chan, C.Y.: A Novel Direct Trajectory Planning Approach Based on Generative Adversarial Networks and Rapidly-Exploring Ran- dom Tree. IEEE Trans. Intell. Transport. Syst. 23(10), 17910–17921 (Oct 2022). https://doi.org/10.1109/TITS.2022.3164391

    arXiv 2022

  47. [49]

    org/abs/2412.20404

    Zheng, Z., Peng, X., Yang, T., Shen, C., Li, S., Liu, H., Zhou, Y., Li, T., You, Y.: Open-sora: Democratizing efficient video production for all (2024), https://arxiv. org/abs/2412.20404

    Pith/arXiv arXiv 2024

  48. [50]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022)

    Zhou, Z., Ye, L., Wang, J., Wu, K., Lu, K.: HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022)

    2022