Pith. sign in

REVIEW 2 major objections 54 references

Converting motion prediction modes into ordered sequences with explicit dependencies reduces collapse and improves ranking.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 17:15 UTC pith:WEEWHRIO

load-bearing objection The sequence reframing plus EMTA loss produced real leaderboard wins on Waymo, but without an ablation isolating the causal dependency the central claim stays under-supported. the 2 major comments →

arxiv 2605.24037 v1 pith:WEEWHRIO submitted 2026-05-21 cs.CV cs.AI

Mode-as-Sequence: Translating Multimodal Motion Prediction into Unified Sequential Mode Modeling

Zikang Zhou , Haibo Hu , Xinhong Chen , Yifan Zhang , Nan Guan , Yung-Hui Li , Chun Jason Xue , Jianping Wang This is my paper
classification cs.CV cs.AI
keywords multimodal motion predictionmode collapsesequence modelingmotion forecastingautonomous drivingWaymo Open Datasetrecurrent decodingmasked attention
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that multimodal motion forecasting is under-supervised because each scene supplies only one realized future, which causes mode collapse into redundant hypotheses and unreliable confidence scores. It addresses this by reframing the unordered set of predicted modes as an ordered sequence so that mode-to-mode dependencies can be modeled directly. Two implementations follow: recurrent decoding in ModeSeq conditions each new mode on prior ones, while Parallel ModeSeq uses masked self-attention to keep the same causal structure but decode all modes at once. Early-Match-Take-All matching and a lightweight ranking regularizer further ensure the modes are representative and properly ordered under sparse labels. These changes produce measurable gains in both diversity and ranking metrics on large benchmarks and secure first-place results in recent Waymo challenges.

Core claim

Mode-as-Sequence is a unified decoding framework that translates an unordered mode set into an ordered mode sequence and explicitly models mode-to-mode dependency; ModeSeq realizes this via recurrent generation while Parallel ModeSeq uses masked mode-to-mode self-attention, both paired with Early-Match-Take-All matching and a ranking regularizer to learn calibrated modes from single-realization supervision.

What carries the argument

Mode-as-Sequence framework, which imposes sequential order on modes to enable explicit dependency modeling through recurrence or masked attention.

Load-bearing premise

That imposing ordered mode-to-mode dependencies through recurrence or masked attention, together with Early-Match-Take-All matching, will increase diversity and ranking accuracy without creating new failure modes when only one future is observed per scene.

What would settle it

A controlled comparison on held-out scenes where the sequential models produce lower best-of-K accuracy or higher confidence inversion rates than a non-sequential multimodal baseline that uses the same backbone and matching loss.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Recurrent and parallel variants both improve ranking-oriented metrics and best-of-K accuracy across datasets, prediction horizons, and object types.
  • Parallel ModeSeq removes the autoregressive bottleneck, enabling efficient inference for large numbers of modes and joint-scene prediction.
  • MA-EMTA extends the matching strategy to multi-agent scenes while preserving the same dependency modeling.
  • The ranking regularizer directly reduces confidence inversions under the single-label regime.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same sequence-ordering idea could be tested on other under-supervised multimodal tasks such as future video frame prediction or multi-hypothesis object detection.
  • If the causal dependency structure generalizes, it may offer a template for stabilizing diversity in other generative models trained with sparse supervision.
  • Large-scale deployment would require checking whether the added ordering constraints remain beneficial when scene complexity or sensor noise increases beyond the Waymo distribution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The manuscript proposes Mode-as-Sequence, a unified decoding framework that reformulates unordered multimodal motion prediction outputs as ordered mode sequences to explicitly capture mode-to-mode dependencies. It instantiates this via recurrent decoding (ModeSeq) and parallel masked self-attention (Parallel ModeSeq), introduces Early-Match-Take-All (EMTA) and MA-EMTA losses plus a ranking regularizer to mitigate mode collapse under single-realization supervision, and reports consistent gains in ranking metrics and best-of-K accuracy, culminating in first-place finishes on the 2024 Waymo LiDAR-free track and 2025 Interaction Prediction Challenge.

Significance. If the central empirical claims are substantiated, the work offers a practical and scalable approach to improving mode diversity and confidence calibration in motion forecasting, directly evidenced by challenge leaderboard leadership; the parallel variant additionally addresses inference efficiency for large K and joint-scene settings.

major comments (2)
  1. [Abstract] Abstract (final paragraph) and method description: the headline 1st-place claims on Waymo challenges are presented as validation of the Mode-as-Sequence thesis, yet no ablation is described that holds EMTA/MA-EMTA, capacity, and ranking regularizer fixed while removing only the ordered mode-to-mode dependency (recurrent or masked-attention); without this isolation it remains possible that the reported gains are driven by the matching/ranking components rather than the sequential modeling itself.
  2. [Abstract] Abstract (paragraph 1) and results sections: the claim of 'consistent improvements ... across datasets, horizons, and object types' is stated without reference to error bars, statistical significance tests, or the number of random seeds; given the low-confidence soundness assessment arising from absent ablation tables, this weakens the ability to attribute gains specifically to the proposed dependency modeling.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the empirical validation of Mode-as-Sequence. We respond to each major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract (final paragraph) and method description: the headline 1st-place claims on Waymo challenges are presented as validation of the Mode-as-Sequence thesis, yet no ablation is described that holds EMTA/MA-EMTA, capacity, and ranking regularizer fixed while removing only the ordered mode-to-mode dependency (recurrent or masked-attention); without this isolation it remains possible that the reported gains are driven by the matching/ranking components rather than the sequential modeling itself.

    Authors: We agree that a controlled ablation isolating the ordered mode-to-mode dependency (while holding EMTA/MA-EMTA, model capacity, and the ranking regularizer fixed) would more directly attribute gains to the sequential modeling. The manuscript compares ModeSeq/Parallel ModeSeq to non-sequential baselines, but these differ in multiple respects. We will add the requested ablation (e.g., independent per-mode decoding with identical losses and regularizer) in the revised version. revision: yes

  2. Referee: [Abstract] Abstract (paragraph 1) and results sections: the claim of 'consistent improvements ... across datasets, horizons, and object types' is stated without reference to error bars, statistical significance tests, or the number of random seeds; given the low-confidence soundness assessment arising from absent ablation tables, this weakens the ability to attribute gains specifically to the proposed dependency modeling.

    Authors: We acknowledge that the absence of error bars, seed counts, and significance tests limits the strength of the consistency claims. Experiments were run across multiple seeds, but these details were omitted. We will revise the results sections and tables to report standard deviations, the number of random seeds, and any applicable statistical tests. revision: yes

Circularity Check

0 steps flagged

No circularity; empirical claims rest on external benchmarks

full rationale

The paper introduces Mode-as-Sequence as a modeling framework with recurrent/attention-based mode decoding and EMTA matching, then reports leaderboard results on Waymo Open Dataset. No equations, fitted parameters, or self-citations are presented in the abstract or described claims that reduce any performance number or ranking to a quantity defined inside the same paper. The derivation chain consists of architectural choices whose outputs are evaluated on held-out external data rather than by construction or self-referential fitting. This is the common case of a self-contained empirical ML contribution.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Abstract-only review supplies no equations or implementation details, so the ledger is limited to the explicit domain assumption stated in the first sentence.

axioms (1)
  • domain assumption Multimodal motion forecasting is inherently under-supervised because each training scene provides only one realized future.
    Opening sentence of the abstract; this premise motivates the entire framework.
invented entities (1)
  • Early-Match-Take-All (EMTA) loss and its joint-scene extension MA-EMTA no independent evidence
    purpose: To learn representative modes and calibrated confidence under sparse labels
    Introduced as a new training component in the abstract; no independent evidence supplied.

pith-pipeline@v0.9.1-grok · 5844 in / 1411 out tokens · 29614 ms · 2026-06-30T17:15:13.527640+00:00 · methodology

0 comments
read the original abstract

Multimodal motion forecasting is inherently under-supervised: each training scene provides only one realized future, yet multiple plausible futures exist. This sparse supervision often leads to mode collapse (redundant hypotheses and insufficient mode coverage) and unreliable confidence ranking when predicting a small set of trajectories. We propose Mode-as-Sequence, a unified decoding framework that translates an unordered mode set into an ordered mode sequence and explicitly models mode-to-mode dependency. Under this framework, we develop two complementary instantiations. ModeSeq performs recurrent mode decoding, where each mode is generated conditioned on the previously generated modes, encouraging diverse, non-redundant hypotheses with calibrated confidence ordering. To remove the mode-by-mode autoregressive bottleneck, we further propose Parallel ModeSeq, which preserves the same causal dependency using masked mode-to-mode self-attention while decoding all modes in a single forward pass, enabling efficient large-$K$ inference and scalable joint-scene prediction. To learn representative modes and calibrated confidence under sparse labels, we introduce Early-Match-Take-All (EMTA) and its joint-scene extension MA-EMTA, together with a lightweight ranking regularizer that reduces confidence inversions. Extensive experiments on large-scale benchmarks demonstrate consistent improvements in both ranking-oriented metrics and best-of-K accuracy across datasets, horizons, and object types. In the Waymo Open Dataset challenges, ModeSeq achieves 1st place in the 2024 LiDAR-free motion prediction track, and Parallel ModeSeq achieves 1st place in the 2025 Interaction Prediction Challenge, validating the effectiveness of Mode-as-Sequence for both accuracy and efficiency.

Figures

Figures reproduced from arXiv: 2605.24037 by Chun Jason Xue, Haibo Hu, Jianping Wang, Nan Guan, Xinhong Chen, Yifan Zhang, Yung-Hui Li, Zikang Zhou.

Figure 1
Figure 1. Figure 1: Illustration of three different ways to model future [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the Mode-as-Sequence framework. The scene embedding contains the ego agent’s state, surrounding agent [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Details of Parallel ModeSeq layer. Multiple modes are [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization on the WOMD. The agents in purple are predicted with blue trajectories, with the opacity indicating [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The results of generating more than 6 modes on the validation set of the WOMD. rearrangement explicitly prioritizes the refinement of the more probable trajectories in the next layer by manually placing the less confident modes at the end of the sequence. e) Capability of Representative Mode Learning.: We demonstrate Mode-as-Sequence’s ability to produce represen￾tative modes in Table VI. While training mo… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages · 2 internal anchors

  1. [1]

    Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset,

    S. Ettinger, S. Cheng, B. Caine, C. Liu, H. Zhao, S. Pradhan, Y . Chai, B. Sapp, C. R. Qi, Y . Zhouet al., “Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9710–9719

    work page 2021

  2. [2]

    Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting

    B. Wilson, W. Qi, T. Agarwal, J. Lambert, J. Singh, S. Khandelwal, B. Pan, R. Kumar, A. Hartnett, J. K. Ponteset al., “Argoverse 2: Next generation datasets for self-driving perception and forecasting,”arXiv preprint arXiv:2301.00493, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  3. [3]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Kr- ishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” inCVPR, 2020, pp. 11 621–11 631

    work page 2020

  4. [4]

    INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps

    W. Zhan, L. Sun, D. Wang, H. Shi, A. Clausse, M. Naumann, J. Kum- merle, H. Konigshof, C. Stiller, A. de La Fortelleet al., “Interaction dataset: An international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps,” inarXiv preprint arXiv:1910.03088, 2019

    work page Pith review arXiv 1910

  5. [5]

    Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction,

    Y . Chai, B. Sapp, M. Bansal, and D. Anguelov, “Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction,” in CoRL, 2019, pp. 86–99

    work page 2019

  6. [6]

    Covernet: Multimodal behavior prediction using trajectory sets,

    T. Phan-Minh, E. C. Grigore, F. A. Boulton, O. Beijbom, and E. M. Wolff, “Covernet: Multimodal behavior prediction using trajectory sets,” inCVPR, 2020, pp. 14 074–14 083. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 14

    work page 2020

  7. [7]

    The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs,

    B. Ivanovic and M. Pavone, “The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs,” inICCV, 2019, pp. 2375–2384

    work page 2019

  8. [8]

    Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,

    T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone, “Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,” in European conference on computer vision. Springer, 2020, pp. 683–700

    work page 2020

  9. [9]

    Social gan: Socially acceptable trajectories with generative adversarial networks,

    A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, “Social gan: Socially acceptable trajectories with generative adversarial networks,” in CVPR, 2018, pp. 2255–2264

    work page 2018

  10. [10]

    Overcoming limitations of mixture density networks: A sampling and fitting framework for multimodal future prediction,

    O. Makansi, E. Ilg, O. Casser, and T. Brox, “Overcoming limitations of mixture density networks: A sampling and fitting framework for multimodal future prediction,” inCVPR, 2019, pp. 7144–7153

    work page 2019

  11. [11]

    Accurate and diverse sampling of sequences based on a “best of many

    A. Bhattacharyya, B. Schiele, and M. Fritz, “Accurate and diverse sampling of sequences based on a “best of many” sample objective,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8485–8493

    work page 2018

  12. [12]

    Densetnt: End-to-end trajectory prediction from dense goal sets,

    H. Zhao, J. Gao, T. Lan, C. Sun, B. Sapp, B. Balakrishnan, Y . Shen, and H. Zhao, “Densetnt: End-to-end trajectory prediction from dense goal sets,” inICCV, 2021, pp. 15 403–15 412

    work page 2021

  13. [13]

    Densetnt: Waymo open dataset motion prediction challenge 1st place solution,

    J. Gu, C. Sun, and H. Zhao, “Densetnt: Waymo open dataset motion prediction challenge 1st place solution,” inarXiv preprint arXiv:2106.13240, 2021

    work page arXiv 2021

  14. [14]

    Hivt: Hierarchical vector transformer for multi-agent motion prediction,

    Z. Zhou, L. Ye, J. Wang, K. Wu, and K. Lu, “Hivt: Hierarchical vector transformer for multi-agent motion prediction,” inCVPR, 2022, pp. 8823–8833

    work page 2022

  15. [15]

    Motion transformer with global intention localization and local movement refinement,

    S. Shi, L. Jiang, D. Deng, and J. Yan, “Motion transformer with global intention localization and local movement refinement,” inNeurIPS, vol. 35, 2022, pp. 6531–6543

    work page 2022

  16. [16]

    Wayformer: Motion forecasting via simple & efficient attention net- works,

    N. Nayakanti, R. Al-Rfou, A. Zhou, K. Goel, K. S. Refaat, and B. Sapp, “Wayformer: Motion forecasting via simple & efficient attention net- works,” inICRA, 2023, pp. 2980–2987

    work page 2023

  17. [17]

    Query-centric trajectory prediction,

    Z. Zhou, J. Wang, Y .-H. Li, and Y .-K. Huang, “Query-centric trajectory prediction,” inCVPR, 2023, pp. 17 863–17 873

    work page 2023

  18. [18]

    Modeseq: Taming sparse multimodal motion prediction with sequential mode modeling,

    Z. Zhou, H. Zhou, H. Hu, Z. Wen, J. Wang, Y .-H. Li, and Y .-K. Huang, “Modeseq: Taming sparse multimodal motion prediction with sequential mode modeling,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 1612–1621

    work page 2025

  19. [19]

    Multimodal trajectory predictions for autonomous driving using deep convolutional networks,

    H. Cui, V . Radosavovic, T.-Y . Chou, T.-H. Lin, T. Tanet al., “Multimodal trajectory predictions for autonomous driving using deep convolutional networks,” inICRA, 2019, pp. 2090–2096

    work page 2019

  20. [20]

    Uncertainty-aware short-term motion prediction of traffic actors for autonomous driving,

    N. Djuric, V . Radosavovic, H. Cui, T. Nguyen, T.-Y . Chou, T.-H. Lin, N. Singh, and J. Schneider, “Uncertainty-aware short-term motion prediction of traffic actors for autonomous driving,” inWACV, 2020, pp. 2095–2104

    work page 2020

  21. [21]

    Rules of the road: Predicting driving behavior with a convolutional model of semantic interactions,

    B. Hong, Joey andSapp and J. Philbin, “Rules of the road: Predicting driving behavior with a convolutional model of semantic interactions,” inCVPR, 2019, pp. 8454–8462

    work page 2019

  22. [22]

    Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst,

    M. Bansal, A. Krizhevsky, and A. Ogale, “Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst,” inRSS, 2019

    work page 2019

  23. [23]

    Home: Heatmap output for future motion estimation,

    T. Gilles, S. Sabatini, D. Tsishkou, B. Stanciulescu, and F. Moutarde, “Home: Heatmap output for future motion estimation,” inITSC, 2021, pp. 500–507

    work page 2021

  24. [24]

    Gohome: Graph-oriented heatmap output for future motion estimation,

    ——, “Gohome: Graph-oriented heatmap output for future motion estimation,” inICRA, 2022, pp. 9107–9114

    work page 2022

  25. [25]

    Vectornet: Encoding hd maps and agent dynamics from vectorized representation,

    J. Gao, C. Sun, H. Zhao, Y . Shen, D. Anguelov, C. Li, and C. Schmid, “Vectornet: Encoding hd maps and agent dynamics from vectorized representation,” inCVPR, 2020, pp. 11 525–11 533

    work page 2020

  26. [26]

    Learning lane graph representations for motion forecasting,

    M. Liang, B. Yang, R. Hu, Y . Chen, R. Liao, S. Feng, and R. Urtasun, “Learning lane graph representations for motion forecasting,” inECCV, 2020, pp. 541–556

    work page 2020

  27. [27]

    Tnt: Target-driven trajectory prediction,

    H. Zhao, J. Gao, T. Lan, C. Sun, B. Sapp, B. Balakrishnan, Y . Shen, and H. Zhao, “Tnt: Target-driven trajectory prediction,” inCoRL, 2020, pp. 895–904

    work page 2020

  28. [28]

    Mul- tipath++: Efficient information fusion and trajectory aggregation for behavior prediction,

    B. Varadarajan, A. Hefny, A. Srivastava, K. S. Refaatet al., “Mul- tipath++: Efficient information fusion and trajectory aggregation for behavior prediction,” inICRA, 2022

    work page 2022

  29. [29]

    Social lstm: Human trajectory prediction in crowded spaces,

    A. Alahi, K. Goel, V . Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “Social lstm: Human trajectory prediction in crowded spaces,” inCVPR, 2016, pp. 961–971

    work page 2016

  30. [30]

    Convolutional social pooling for vehicle trajectory prediction,

    N. Deo and M. M. Trivedi, “Convolutional social pooling for vehicle trajectory prediction,” inCVPR Workshops, 2018, pp. 1468–1476

    work page 2018

  31. [31]

    Desire: Distant future prediction in dynamic scenes with interacting agents,

    N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. Torr, and M. Chandraker, “Desire: Distant future prediction in dynamic scenes with interacting agents,” inCVPR, 2017, pp. 336–345

    work page 2017

  32. [32]

    Stochastic trajectory prediction via motion indeterminacy diffusion,

    T. Gu, G. Chen, J. Li, C. Lin, Y . Rao, J. Zhou, and J. Lu, “Stochastic trajectory prediction via motion indeterminacy diffusion,” inCVPR, 2022, pp. 17 113–17 122

    work page 2022

  33. [33]

    Motiondiffuser: Controllable multi-agent motion prediction using dif- fusion,

    C. Jiang, A. Cornman, C. Park, B. Sapp, Y . Zhou, and D. Anguelov, “Motiondiffuser: Controllable multi-agent motion prediction using dif- fusion,” inCVPR, 2023, pp. 14 499–14 508

    work page 2023

  34. [34]

    Leapfrog diffusion model for stochastic trajectory prediction,

    W. Mao, C. Chen, Y . Liuet al., “Leapfrog diffusion model for stochastic trajectory prediction,” inCVPR, 2023

    work page 2023

  35. [35]

    Guided conditional diffusion for controllable traffic simulation,

    Z. Zhong, D. Rempe, Davis andxu, Y . Chen, S. Veer, T. Che, B. Ray, and M. Pavone, “Guided conditional diffusion for controllable traffic simulation,” inICRA, 2023, pp. 3560–3566

    work page 2023

  36. [36]

    Scene transformer: A unified architecture for predicting multiple agent trajectories,

    J. Ngiam, B. Caine, V . Vasudevan, Z. Zhang, H.-T. L. Chiang, J. Ling, R. Roelofs, A. Bewley, C. Liuet al., “Scene transformer: A unified architecture for predicting multiple agent trajectories,” inICLR, 2022

    work page 2022

  37. [37]

    Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting,

    Y . Yuan, X. Weng, Y . Ou, and K. Kitani, “Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting,” inICCV, 2021, pp. 9813–9823

    work page 2021

  38. [38]

    Rain: Reinforced hybrid attention inference network for motion forecasting,

    J. Li, F. Yang, H. Ma, S. Malla, M. Tomizuka, and C. Choi, “Rain: Reinforced hybrid attention inference network for motion forecasting,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 16 096–16 106

    work page 2021

  39. [39]

    Hdgt: Heterogeneous driving graph transformer for multi-agent trajectory prediction,

    X. Jia, L. Wong, Y . Chen, J. Wu, and J. Yan, “Hdgt: Heterogeneous driving graph transformer for multi-agent trajectory prediction,” inIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    work page 2023

  40. [40]

    Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving,

    Z. Huang, H. Liu, and C. Lv, “Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3903–3913

    work page 2023

  41. [41]

    Planning-oriented autonomous driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wanget al., “Planning-oriented autonomous driving,” in CVPR, 2023, pp. 17 853–17 862

    work page 2023

  42. [42]

    Vad: Vectorized scene representation for efficient autonomous driving,

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” inICCV, 2023

    work page 2023

  43. [43]

    Perceive, predict, and plan: Safe motion planning through interpretable semantic representations,

    A. Sadat, M. Ren, A. Pokrovsky, Y .-C. Lin, E. Yumer, and R. Urtasun, “Perceive, predict, and plan: Safe motion planning through interpretable semantic representations,” inECCV, 2020, pp. 414–430

    work page 2020

  44. [44]

    Precog: Predic- tion conditioned on goals in visual multi-agent settings,

    N. Rhinehart, R. McAllister, K. Kitani, and S. Levine, “Precog: Predic- tion conditioned on goals in visual multi-agent settings,” inICCV, 2019, pp. 2821–2830

    work page 2019

  45. [45]

    Planning and decision- making for autonomous vehicles,

    W. Schwarting, J. Alonso-Mora, and D. Rus, “Planning and decision- making for autonomous vehicles,” inAnnual Review of Control, Robotics, and Autonomous Systems, vol. 1, 2018, pp. 187–210

    work page 2018

  46. [46]

    Mtr v3: 1st place solution for 2024 waymo open dataset challenge - motion prediction,

    C. Shi, S. Shi, and L. Jiang, “Mtr v3: 1st place solution for 2024 waymo open dataset challenge - motion prediction,” Waymo Open Dataset Challenges (Technical Report), Technical Report, 2024

    work page 2024

  47. [47]

    Rmp: 3rd place solution for 2024 waymo open dataset challenge - motion prediction,

    J. Sun, J. Li, T. Liu, C. Yuan, S. Sun, Y . Han, A. Wong, K. P. Tee, and M. H. J. Ang, “Rmp: 3rd place solution for 2024 waymo open dataset challenge - motion prediction,” Waymo Open Dataset Challenges (Technical Report), Technical Report, 2024

    work page 2024

  48. [48]

    Scalability in perception for autonomous driving: Waymo open dataset,

    P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V . Patnaik, P. Tsui, J. Guo, Y . Zhou, Y . Chai, B. Caineet al., “Scalability in perception for autonomous driving: Waymo open dataset,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2446–2454

    work page 2020

  49. [49]

    Causal- planner: Causal interaction disentangling with episodic memory gating for autonomous planning,

    Y . Yuan, J. Fang, Y . Zhou, Z. Yang, C. Lv, and J. Xue, “Causal- planner: Causal interaction disentangling with episodic memory gating for autonomous planning,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 335–341

    work page 2025

  50. [50]

    Reasoning multi-agent behavioral topology for interactive autonomous driving,

    H. Liu, L. Chen, Y . Qiao, C. Lv, and H. Li, “Reasoning multi-agent behavioral topology for interactive autonomous driving,”Advances in Neural Information Processing Systems, vol. 37, pp. 92 605–92 637, 2024

    work page 2024

  51. [51]

    Impact: Behavioral intention-aware multimodal trajectory prediction with adaptive context trimming,

    J. Sun, X. Yue, J. Li, T. Shen, C. Yuan, S. Sun, S. Guo, Q. Zhou, and M. H. Ang Jr, “Impact: Behavioral intention-aware multimodal trajectory prediction with adaptive context trimming,”IEEE Robotics and Automation Letters, vol. 11, no. 1, pp. 610–617, 2025

    work page 2025

  52. [52]

    Trajflow: Multi-modal motion prediction via flow matching,

    Q. Yan, B. Zhang, Y . Zhang, D. Yang, J. White, D. Chen, J. Liu, L. Liu, B. Zhuang, S. Shiet al., “Trajflow: Multi-modal motion prediction via flow matching,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 3923–3928

    work page 2025

  53. [53]

    Rmp-yolo: A robust motion predictor for partially observable scenarios even if you only look once,

    J. Sun, J. Li, T. Liu, C. Yuan, S. Sun, Z. Huang, A. Wong, K. P. Tee, and M. H. Ang, “Rmp-yolo: A robust motion predictor for partially observable scenarios even if you only look once,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 1024–1031

    work page 2025

  54. [54]

    RetroMotion: Retrocausal Motion Forecasting Models are Instructable

    R. Wagner, O. S. Tas, F. Hauser, M. Steiner, D. Strutz, A. Vivekanan- dan, C. Fernandez, and C. Stiller, “Retromotion: Retrocausal motion forecasting models are instructable,”arXiv preprint arXiv:2505.20414, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025