Pith. sign in

REVIEW 3 major objections 5 minor 56 references

DONUT: A Decoder-Only Model for Trajectory Prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that trajectory forecasting can be done with one decoder-only Transformer that unrolls future motion step by step, and reports state-of-the-art results on Argoverse 2.

desk verdict Solid decoder-only variant for Argoverse 2 with a defensible non-ensemble SOTA claim; the load-bearing comparisons rest on single runs, so the gains need seed-level evidence before I'd call them settled. read the letter →

arxiv 2506.06854 v2 pith:XC4RKD7K submitted 2025-06-07 cs.CV

Markus Knoche , Daan de Geus , Bastian Leibe This is my paper
classification cs.CV
keywords motionforecastingdecoder-onlytransformerautoregressivetrajectorypredictionoverpredictionArgoverse2multi-tokenautonomousdrivingquery-centricencoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that motion forecasting should work like language generation: one decoder-only Transformer consumes the observed history, then predicts the future one short segment at a time, instead of a separate encoder producing a fixed summary that a decoder turns into a full trajectory. The proposed model, DONUT, unrolls predictions autoregressively and, after each segment, moves its coordinate reference point to the segment's endpoint, so its attention to road geometry and other agents always reflects the most recent position. Training also asks the model to 'overpredict' the segment after the one it is predicting, borrowing multi-token prediction from language modeling. On the Argoverse 2 single-agent benchmark, DONUT reports 1.79 on the main b-minFDE6 metric (endpoint error plus a penalty for low confidence in the best mode), the best published non-ensemble score, and its ablation beats the encoder-decoder baseline on endpoint-focused metrics while the gap grows with prediction horizon. This matters because correctly anticipating the far future—turns, intersections, conflicts—is exactly what lets a self-driving car act early rather than react late.

What carries the argument

The mechanism is autoregressive trajectory unrolling in a query-centric Transformer. Future time is divided into sub-trajectories of $T_{\mathrm{sub}}=10$ steps (one second); a proposer module tokenizes the previous sub-trajectory, applies temporal, map, social, and mode attention with relative positional encodings, and detokenizes to predict the next sub-trajectory plus an overpredicted one. The reference frame is then moved to the endpoint of the proposed segment before a refiner predicts offsets, and moved again after refinement, so every relative encoding is recomputed against the latest predicted position. A mixture-of-Laplace loss supervises positions, a von Mises distribution supervises headings, and mode probabilities are trained by the joint negative log-likelihood.

What would settle it

Run the same training configuration multiple times with different random seeds and measure the spread of b-minFDE6 on the Argoverse 2 validation set. If the observed gap between DONUT (1.807) and the encoder-decoder baseline (1.874), or the test-set gap to the nearest non-ensemble competitor (1.79 vs 1.84), falls within one standard deviation of that spread, the state-of-the-art and ablation claims would not be distinguishable from training noise.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the encoder-decoder separation used by most motion-forecasting models is not required, and is in fact a bottleneck for long-horizon accuracy. A single decoder-only Transformer that processes observed and future trajectory segments in the same way, always attending to scene elements from the endpoint of its latest prediction, predicts more accurate endpoints than the encoder-decoder baseline, and the advantage grows with prediction length. Adding the overprediction objective and a refinement module gives b-minFDE6 of 1.79 on the Argoverse 2 test set, ahead of every published non-ensemble method, with the largest gains on turn-heavy scenes where the baseline performs worst.

Load-bearing premise

The load-bearing premise is that the reported single-training-run numbers on one benchmark are stable enough to rank configurations; no repeated seeds, confidence intervals, or significance tests are reported, while several decisive differences are small.

Editorial extensions

If this is right

  • If DONUT's results hold, a separate agent encoder is not needed for state-of-the-art single-agent forecasting: the same decoder can read history and generate future.
  • Per-step reference-point updates are what buy far-future accuracy: the decoder-only model's endpoint-error advantage over the encoder-decoder baseline grows over the 60 prediction steps, and the relative gain is larger (14.6% vs 6.1% in minFDE) on scenes with turns above 45 degrees.
  • Overprediction and refinement each do little alone, but together they give the best configuration, and overprediction also removes the training instabilities seen with refinement alone.
  • DONUT establishes a new non-ensemble state of the art on Argoverse 2 (b-minFDE6 1.79), ahead of the nearest non-ensemble competitor (1.84) and close behind the best ensemble model (1.73).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the decoder-only recipe—short segments, endpoint-anchored attention, and overprediction—could transfer to other multimodal sequence-prediction problems, such as pedestrian intent or marine traffic, where stale context also degrades far-future forecasts.
  • The supplement's efficiency numbers point to a trade-off: decoder-only unrolling roughly triples inference time versus the baseline (65.7 ms vs 23.7 ms) and refinement roughly doubles it again (129 ms), so reducing latency through sparser attention or caching is a natural next step for deployment.
  • A testable extension would be to evaluate calibration of the six mode probabilities separately from endpoint accuracy, since b-minFDE6 merges the two and the paper does not report them independently.
  • Because all comparisons come from single training runs, repeated-seed experiments on the validation set would show whether the small decisive margins (1.807 vs 1.874 in the full ablation, 1.79 vs 1.84 on the test set) are robust to training noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DONUT, a decoder-only Transformer for motion forecasting on Argoverse 2. In place of the usual encoder-decoder structure, DONUT processes historical and future trajectory sub-steps with a single autoregressive decoder, updates the query-centric reference point after each predicted segment, adds an auxiliary overprediction loss inspired by multi-token prediction, and combines this with a refinement module. The authors report ablation results on the Argoverse 2 validation set against a QCNet-based encoder-decoder baseline, plus one submission to the Argoverse 2 hidden test leaderboard, claiming non-ensemble state-of-the-art b-minFDE6 of 1.79 and arguing that the decoder-only formulation improves far-future predictions.

Significance. If the empirical claims hold, the contribution is meaningful: DONUT offers a conceptually clean architectural shift for trajectory prediction, with a plausible mechanism (regular reference-point updates and autoregressive consistency) for improving long-horizon endpoints, and it adapts multi-token prediction to a forecasting setting. The paper has several strengths: evaluation is on the official hidden test set with standard benchmark metrics, comparisons are made against published leaderboard entries, ablations are reasonably complete, and the supplementary material includes additional qualitative results, failure cases, and an efficiency analysis. The main reservations are statistical: every configuration is evaluated with a single training run, the encoder-decoder baseline is not retrained under the same protocol, and the headline margins are small, so the central claims are not yet established at the level of rigor expected for a benchmark-state-of-the-art assertion.

major comments (3)
  1. [§4.2, Tables 1 and 4; §4.3, Table 2] All load-bearing comparisons rest on single runs: Table 1 and supplementary Table 4 report one training run per configuration, and Table 2 is based on a single test submission, with no repeated seeds, confidence intervals, or significance tests. The decisive margins are small (decoder-only vs baseline b-minFDE6 1.874 vs 1.838; adding overprediction plus refinement 1.838 vs 1.807; test-set DONUT vs DeMo 1.79 vs 1.84). The authors also report 'minor training instabilities' when refinement is used alone (§4.2), which indicates sensitivity to optimization randomness. On this evidence, the reported ordering could be a single-luck draw, and the decoder-only advantage and the state-of-the-art claim do not follow until seed-level variability or equivalent statistical evidence is provided.
  2. [§4.1, Table 1] The comparison against the encoder-decoder baseline is not controlled: the baseline uses the publicly released code and weights of QCNet (§4.1) and is therefore not retrained under the same training recipe as DONUT (epochs, batch size, learning-rate schedule, data pipeline). The claimed 0.036 b-minFDE6 gain from switching to decoder-only thus conflates architecture with training differences. The authors should retrain QCNet under their own training protocol, or otherwise demonstrate that the reported baseline number is comparable under identical conditions.
  3. [Abstract and §4.3] The abstract's unqualified statement 'achieves new state-of-the-art results on the Argoverse 2 single-agent motion forecasting benchmark' is stronger than what Table 2 supports: QCNet*, SEPT*, and DeMo* (all ensemble methods) achieve lower b-minFDE6 values of 1.78, 1.74, and 1.73, respectively. The claim is defensible only when restricted to non-ensemble methods, as the conclusion itself phrases it. This wording should be corrected consistently throughout the paper.
minor comments (5)
  1. [§4.1] The sentence 'Each scene is 11 slong, sampled at 10 Hz' appears to have a missing space; it should read '11 s long'.
  2. [Table 1] The header of Table 1 uses '✓' and '✗' for the decoder-only, overprediction, and refinement rows, but the encoder-decoder baseline row uses 'N/A' for overprediction and refinement. Using the same cross/check symbols throughout would make the row more readable.
  3. [Figure 2] In Figure 2, the labels for the reference-point updates are partially blank ('from to ', 'from to '), which makes the figure hard to parse; the intended coordinate frames should be labeled explicitly.
  4. [§4.3] The statement that unimodal metrics are 'not suitable' because an optimal mean prediction would stop in the middle of an intersection is not fully justified; a strong unimodal prediction can still be a meaningful indicator of trajectory quality, and the comparison against DeMo on minADE1 and minFDE1 should be discussed more carefully rather than dismissed.
  5. [§4.2] The sentence 'the benefit of overprediction is larger when using refinement because ... effectively reducing the overprediction horizon from 20 steps to 10' is unclear; the intended meaning of 'overprediction horizon' in this context should be explained.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DONUT's claims are empirical benchmark results with auxiliary supervision, not derivations from fitted parameters or self-citations.

full rationale

The paper's central claims are empirical: a decoder-only architecture is compared against an encoder-decoder baseline (QCNet) on the Argoverse 2 validation set, and the full model is submitted to the external hidden test leaderboard. The overprediction and refinement losses are supervised with ground-truth future positions via the Laplace and von-Mises likelihoods in Eqs. (1) and (2), and the overpredictions are explicitly discarded at inference time, so no evaluation metric is forced by a fitted parameter. The decoder-only design is an architectural modification relative to QCNet and is tested in Table 1; the state-of-the-art claim rests on Table 2 against published external results. The only reused prior work is QCNet [55], which is cited as a baseline and codebase, not as a self-citation or as a load-bearing uniqueness theorem. The abstract's phrase 'new state-of-the-art results' is slightly broader than the conclusion's accurate 'non-ensemble' qualification, since ensemble entries in Table 2 have lower b-minFDE6 values, but that is a wording imprecision, not circularity. No equation is equivalent to its input by construction, and no fitted quantity is renamed as a prediction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The model has no closed-form constants fitted in a derivation; all quantitative claims come from a trained neural network evaluated on an external benchmark. The items listed are the assumptions the evaluation inherits, including the benchmark protocol, training stability, and the reuse of QCNet's query-centric encoding.

free parameters (4)
  • Sub-trajectory length Tsub = 10 time steps (1 second)
    Chosen by hand; sets the temporal granularity of the unrolling and the overprediction horizon. Section 4.1.
  • Interaction radius r = 50 m
    Radius for map and social attention; follows the QCNet baseline and is not swept. Section 4.1.
  • Embedding dimension D and dropout = D=128, dropout=10%
    Model capacity and regularization hyperparameters chosen without reported ablations. Section 4.1.
  • Training schedule hyperparameters = lr=5e-4, 60 epochs, batch size 64
    Standard AdamW with cosine decay; no sensitivity analysis is reported. Section 4.1.
assumptions (4)
  • domain assumption Argoverse 2 train/val/test splits, K=6 mode evaluation, and the b-minFDE/minFDE/minADE/MR metrics measure motion-forecasting quality as claimed.
    All comparisons, including the SOTA claim, use the benchmark protocol in Section 4.1; the paper does not validate transfer to other datasets.
  • domain assumption Each reported configuration is deterministic or stable enough that single-run differences reflect true ordering.
    No repeated seeds, error bars, or significance tests accompany Tables 1-4; the unstated assumption appears in Section 4.2.
  • ad hoc to paper Query-centric relative positional encodings and the map encoder from QCNet can be reused in DONUT without losing validity.
    Section 3.3 says DONUT uses the same map encoder and query-centric formulation; this transfers an external design without independent justification in the paper.
  • domain assumption The mixture of Laplace and von Mises likelihoods with winner-mode optimization is a suitable training objective.
    Equations (1)-(2) and Section 4.1 adopt this objective from HiVT/QCNet; the paper does not compare objective choices.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DONUT: A Decoder-Only Model for Trajectory Prediction." pith.science (2026). https://pith.science/paper/XC4RKD7K

@misc{pith2026250606854,
  author       = {Pith},
  title        = {Pith review of: DONUT: A Decoder-Only Model for Trajectory Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XC4RKD7K}},
  note         = {Machine review of arXiv:2506.06854}
}
read the original abstract

Predicting the motion of other agents in a scene is highly relevant for autonomous driving, as it allows a self-driving car to anticipate. Inspired by the success of decoder-only models for language modeling, we propose DONUT, a Decoder-Only Network for Unrolling Trajectories. Unlike existing encoder-decoder forecasting models, we encode historical trajectories and predict future trajectories with a single autoregressive model. This allows the model to make iterative predictions in a consistent manner, and ensures that the model is always provided with up-to-date information, thereby enhancing performance. Furthermore, inspired by multi-token prediction for language modeling, we introduce an 'overprediction' strategy that gives the model the auxiliary task of predicting trajectories at longer temporal horizons. This allows the model to better anticipate the future and further improves performance. Through experiments, we demonstrate that our decoder-only approach outperforms the encoder-decoder baseline, and achieves new state-of-the-art results on the Argoverse 2 single-agent motion forecasting benchmark.

Figures

Figures reproduced from arXiv: 2506.06854 by the authors.

Figure 1
Figure 1. Encoder-decoder vs. decoder-only methods for mo￾tion forecasting. In contrast to existing works, which use an encoder-decoder architecture, DONUT uses a unified, autoregres￾sive model to process agents’ historical and future trajectories. This allows it to predict trajectories at different time steps in a consistent manner and receive up-to-date information of relevant scene elements, improving its performance. mode… view at source ↗
Figure 2
Figure 2. DONUT architecture overview. Previously predicted sub-trajectories are fed through a proposer module to make a proposal prediction ( ) and an overprediction ( ). The reference point for all relative encodings is then moved to the endpoint of the proposed trajectory (from to ). Next, the refiner predicts offsets which are added to the proposed trajectories to obtain the final predicted sub￾trajectory and overpredicti… view at source ↗
Figure 3
Figure 3. Proposer architecture. The input sub-trajectory is first tokenized relative to the reference point ( ). Then, the tokens attend to (1) sub-trajectory tokens from previous decoder steps, (2) map tokens, (3) nearby agents, and (4) other modes of the same agent. All attention operations use relative positional encodings based on the current reference point ( ). Finally, a detokenizer outputs the next sub-trajectory and… view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: Performance at different prediction horizons. Com￾pared to the encoder-decoder baseline, our decoder-only approach makes more accurate predictions at longer prediction horizons. thereby unlocking the refiner’s potential. At the same time, the benefit of overprediction …
Figure 5
Figure 5. Figure 5: Qualitative results. Green trajectories ( ) show the ground-truth historical and future trajectory of the agent of interest, red trajectories ( ) visualize the K = 6 future predictions. Boxes show the location of the agents at the start of the predicted trajectory. Ove…
Figure 6
Figure 6. Figure 6: Detailed tokenizer architecture. E. Failure Cases We manually examined 100 scenes with a minFDE > 5 m. Most errors are caused by predictions being too slow (27%) or too fast (19%), or missing a turn (19%). Additionally, 27% had rare ground-truth events, e.g., vehicles …
Figure 7
Figure 7. Figure 7: Failure cases of DONUT. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Additional qualitative results. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Additional qualitative results. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: Additional qualitative results. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Additional qualitative results. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]
Figure 17
Figure 17. Figure 17: Additional qualitative results. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]
Figure 18
Figure 18. Figure 18: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p018_18.png]
Figure 19
Figure 19. Figure 19: Additional qualitative results. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]
Figure 21
Figure 21. Figure 21: Additional qualitative results. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_21.png]
Figure 22
Figure 22. Figure 22: Additional qualitative results. (a) Encoder-decoder (b) DONUT Overp. ✗ Ref. ✗ (c) DONUT Overp. ✓ Ref. ✗ (d) DONUT Overp. ✗ Ref. ✓ (e) DONUT Overp. ✓ Ref. ✓ [PITH_FULL_IMAGE:figures/full_fig_p020_22.png]
Figure 23
Figure 23. Figure 23: Additional qualitative results. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_23.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 51 canonical work pages

  1. [1]

    GPT-4 Technical Report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [2]

    Social LSTM: Human Trajectory Prediction in Crowded Spaces

    Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. Social LSTM: Human Trajectory Prediction in Crowded Spaces. In CVPR, 2016. 3

    work page 2016

  3. [3]

    MultiPath: Multiple Probabilistic Anchor Trajec- tory Hypotheses for Behavior Prediction

    Yuning Chai, Benjamin Sapp, Mayank Bansal, and Dragomir Anguelov. MultiPath: Multiple Probabilistic Anchor Trajec- tory Hypotheses for Behavior Prediction. In CoRL, 2020. 2

    work page 2020

  4. [4]

    Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders

    Jie Cheng, Xiaodong Mei, and Ming Liu. Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders. In ICCV, 2023. 8

    work page 2023

  5. [5]

    Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

    Kyunghyun Cho, Bart van Merri ¨enboer, C ¸ a˘glar G ˙ulc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In EMNLP, 2014. 4

    work page 2014

  6. [6]

    R-Pred: Two-Stage Motion Prediction Via Tube- Query Attention-Based Trajectory Refinement

    Sehwan Choi, Jungho Kim, Junyong Yun, and Jun Won Choi. R-Pred: Two-Stage Motion Prediction Via Tube- Query Attention-Based Trajectory Refinement. In ICCV,

    work page

  7. [7]

    GoRela: Go Relative for Viewpoint- Invariant Motion Forecasting

    Alexander Cui, Sergio Casas, Kelvin Wong, Simon Suo, and Raquel Urtasun. GoRela: Go Relative for Viewpoint- Invariant Motion Forecasting. In ICRA, 2023. 8

    work page 2023

  8. [8]

    Multimodal Trajectory Predic- tions for Autonomous Driving using Deep Convolutional Networks

    Henggang Cui, Vladan Radosavljevic, Fang-Chieh Chou, Tsung-Han Lin, Thi Nguyen, Tzu-Kuo Huang, Jeff Schnei- der, and Nemanja Djuric. Multimodal Trajectory Predic- tions for Autonomous Driving using Deep Convolutional Networks. In ICRA, 2019. 2

    work page 2019

Show all 56 references
  1. [9]

    BERT: Pre-training of Deep Bidirectional Trans- formers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Trans- formers for Language Understanding. In NAACL, 2019. 3

    2019

  2. [10]

    Mac- Former: Map-Agent Coupled Transformer for Real-Time and Robust Trajectory Prediction

    Chen Feng, Hangning Zhou, Huadong Lin, Zhigang Zhang, Ziyao Xu, Chi Zhang, Boyu Zhou, and Shaojie Shen. Mac- Former: Map-Agent Coupled Transformer for Real-Time and Robust Trajectory Prediction. IEEE RA-L, 2023. 8

    2023

  3. [11]

    VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation

    Jiyang Gao, Chen Sun, Hang Zhao, Yi Shen, Dragomir Anguelov, Congcong Li, and Cordelia Schmid. VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation. In CVPR, 2020. 2

    2020

  4. [12]

    Dynamic Scenario Representation Learning for Motion Forecasting With Heterogeneous Graph Convolutional Re- current Networks

    Xing Gao, Xiaogang Jia, Yikang Li, and Hongkai Xiong. Dynamic Scenario Representation Learning for Motion Forecasting With Heterogeneous Graph Convolutional Re- current Networks. IEEE RA-L, 8(5), 2023. 2, 8

    2023

  5. [13]

    HOME: Heatmap Out- put for future Motion Estimation

    Thomas Gilles, Stefano Sabatini, Dzmitry Tsishkou, Bogdan Stanciulescu, and Fabien Moutarde. HOME: Heatmap Out- put for future Motion Estimation. In IEEE ITSC, 2021. 2

    2021

  6. [14]

    THOMAS: Trajec- tory Heatmap Output with learned Multi-Agent Sampling

    Thomas Gilles, Stefano Sabatini, Dzmitry Tsishkou, Bog- dan Stanciulescu, and Fabien Moutarde. THOMAS: Trajec- tory Heatmap Output with learned Multi-Agent Sampling. In ICLR, 2022. 2, 8

    2022

  7. [15]

    Better & Faster Large Language Models via Multi-token Prediction

    Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Roziere, David Lopez-Paz, and Gabriel Synnaeve. Better & Faster Large Language Models via Multi-token Prediction. In ICML,

  8. [16]

    Long Short-term Memory

    Sepp Hochreiter and J ¨urgen Schmidhuber. Long Short-term Memory. Neural computation, 1997. 3

    1997

  9. [17]

    Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions

    Joey Hong, Benjamin Sapp, and James Philbin. Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions. In CVPR, 2019. 2

    2019

  10. [18]

    SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout

    Max Jiang, Yijing Bai, Andre Cornman, Christopher Davis, Xiukun Huang, Hong Jeon, Sakshum Kulshrestha, John Lambert, Shuangyu Li, Xuanyu Zhou, et al. SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout. In NeurIPS, 2024. 3

    2024

  11. [19]

    SEPT: Towards Efficient Scene Repre- sentation Learning for Motion Prediction

    Zhiqian Lan, Yuxuan Jiang, Yao Mu, Chen Chen, and Shengbo Eben Li. SEPT: Towards Efficient Scene Repre- sentation Learning for Motion Prediction. In ICLR, 2024. 1, 8

    2024

  12. [20]

    Desire: Distant future prediction in dynamic scenes with interacting agents

    Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher B Choy, Philip HS Torr, and Manmohan Chandraker. Desire: Distant future prediction in dynamic scenes with interacting agents. In CVPR, 2017. 2

    2017

  13. [21]

    Learning Lane Graph Represen- tations for Motion Forecasting

    Ming Liang, Bin Yang, Rui Hu, Yun Chen, Renjie Liao, Song Feng, and Raquel Urtasun. Learning Lane Graph Represen- tations for Motion Forecasting. In ECCV, 2020. 2

    2020

  14. [22]

    EDA: Evolving and Distinct Anchors for Multimodal Motion Prediction

    Longzhong Lin, Xuewu Lin, Tianwei Lin, Lichao Huang, Rong Xiong, and Yue Wang. EDA: Evolving and Distinct Anchors for Multimodal Motion Prediction. In AAAI, 2024. 1, 2

    2024

  15. [23]

    DeepSeek-V3 Technical Report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. DeepSeek-V3 Technical Report. arXiv preprint arXiv:2412.19437, 2024. 3

    2024 arXiv

  16. [24]

    Multimodal Motion Prediction with Stacked Transformers

    Yicheng Liu, Jinghuai Zhang, Liangji Fang, Qinhong Jiang, and Bolei Zhou. Multimodal Motion Prediction with Stacked Transformers. In CVPR, 2021. 2

    2021

  17. [25]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2017. 6

    2017

  18. [26]

    Social-STGCNN: A Social Spatio- Temporal Graph Convolutional Neural Network for Human Trajectory Prediction

    Abduallah Mohamed, Kun Qian, Mohamed Elhoseiny, and Christian Claudel. Social-STGCNN: A Social Spatio- Temporal Graph Convolutional Neural Network for Human Trajectory Prediction. In CVPR, 2020. 2

    2020

  19. [27]

    The Waymo Open Sim Agents Challenge

    Nico Montali, John Lambert, Paul Mougin, Alex Kuefler, Nicholas Rhinehart, Michelle Li, Cole Gulino, Tristan Em- rich, Zoey Yang, Shimon Whiteson, et al. The Waymo Open Sim Agents Challenge. In NeurIPS, 2023. 3

    2023

  20. [28]

    Wayformer: Motion Forecasting via Simple & Efficient Attention Net- works

    Nigamaa Nayakanti, Rami Al-Rfou, Aurick Zhou, Kratarth Goel, Khaled S Refaat, and Benjamin Sapp. Wayformer: Motion Forecasting via Simple & Efficient Attention Net- works. In ICRA, 2023. 2, 3

    2023

  21. [29]

    Scene Transformer: A unified architecture for predicting multiple agent trajectories

    Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zheng- dong Zhang, Hao-Tien Lewis Chiang, Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, et al. Scene Transformer: A unified architecture for predicting multiple agent trajectories. In ICLR, 2022. 2

    2022

  22. [30]

    CoverNet: Multimodal 9 Behavior Prediction using Trajectory Sets

    Tung Phan-Minh, Elena Corina Grigore, Freddy A Boulton, Oscar Beijbom, and Eric M Wolff. CoverNet: Multimodal 9 Behavior Prediction using Trajectory Sets. In CVPR, 2020. 2

    2020

  23. [31]

    Trajeglish: Traffic Modeling as Next-Token Prediction

    Jonah Philion, Xue Bin Peng, and Sanja Fidler. Trajeglish: Traffic Modeling as Next-Token Prediction. In ICLR, 2024. 3

    2024

  24. [32]

    DySeT: A Dynamic Masked Self-distillation Approach for Robust Trajectory Prediction

    Mozhgan Pourkeshavarz, Junrui Zhang, and Amir Rasouli. DySeT: A Dynamic Masked Self-distillation Approach for Robust Trajectory Prediction. In ECCV, 2024. 8

    2024

  25. [33]

    Improving Language Understanding by Generative Pre-Training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving Language Understanding by Generative Pre-Training. OpenAI Blog, 2018. 2, 3

    2018

  26. [34]

    FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphs

    Luke Rowe, Martin Ethier, Eli-Henry Dykhne, and Krzysztof Czarnecki. FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphs. In CVPR, 2023. 1

    2023

  27. [35]

    Learning Cooperative Trajectory Representations for Motion Forecasting

    Hongzhi Ruan, Haibao Yu, Wenxian Yang, Siqi Fan, and Za- iqing Nie. Learning Cooperative Trajectory Representations for Motion Forecasting. In NeurIPS, 2024. 1

    2024

  28. [36]

    Trajectron++: Dynamically-feasible trajec- tory forecasting with heterogeneous data

    Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, and Marco Pavone. Trajectron++: Dynamically-feasible trajec- tory forecasting with heterogeneous data. In ECCV, 2020. 2

    2020

  29. [37]

    MotionLM: Multi-Agent Motion Forecast- ing as Language Modeling

    Ari Seff, Brian Cera, Dian Chen, Mason Ng, Aurick Zhou, Nigamaa Nayakanti, Khaled S Refaat, Rami Al-Rfou, and Benjamin Sapp. MotionLM: Multi-Agent Motion Forecast- ing as Language Modeling. In ICCV, 2023. 1

    2023

  30. [38]

    Motion Transformer with Global Intention Localization and Local Movement Refinement

    Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Motion Transformer with Global Intention Localization and Local Movement Refinement. In NeurIPS, 2022. 2, 8

    2022

  31. [39]

    Fourier features let networks learn high frequency functions in low dimen- sional domains

    Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ra- mamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimen- sional domains. In NeurIPS, 2020. 3

    2020

  32. [40]

    Multiple Futures Prediction

    Charlie Tang and Russ R Salakhutdinov. Multiple Futures Prediction. In NeurIPS, 2019. 2

    2019

  33. [41]

    HPNet: Dynamic Trajectory Fore- casting with Historical Prediction Attention

    Xiaolong Tang, Meina Kan, Shiguang Shan, Zhilong Ji, Jin- feng Bai, and Xilin Chen. HPNet: Dynamic Trajectory Fore- casting with Historical Prediction Attention. In CVPR, 2024. 1, 2

    2024

  34. [42]

    Gem- ini: A Family of Highly Capable Multimodal Models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gem- ini: A Family of Highly Capable Multimodal Models. arXiv preprint arXiv:2312.11805, 2023. 3

    2023 arXiv

  35. [43]

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288, 2023. 3

    2023 arXiv

  36. [44]

    Attention is All you Need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In NeurIPS, 2017. 3

    2017

  37. [45]

    GANet: Goal Area Network for Motion Forecasting

    Mingkun Wang, Xinge Zhu, Changqian Yu, Wei Li, Yuexin Ma, Ruochun Jin, Xiaoguang Ren, Dongchun Ren, Mingxu Wang, and Wenjing Yang. GANet: Goal Area Network for Motion Forecasting. In ICRA, 2023. 1, 8

    2023

  38. [46]

    ProphNet: Efficient Agent-Centric Motion Forecasting With Anchor-Informed Proposals

    Xishun Wang, Tong Su, Fang Da, and Xiaodong Yang. ProphNet: Efficient Agent-Centric Motion Forecasting With Anchor-Informed Proposals. In CVPR, 2023. 1, 2, 8

    2023

  39. [47]

    Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lam- bert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Rat- nesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Argoverse 2: Next Generation Datasets for Self-Driving Perception an...

    2021

  40. [48]

    SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction

    Wei Wu, Xiaoxin Feng, Ziyan Gao, and Yuheng Kan. SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction. In NeurIPS, 2024. 3

    2024

  41. [49]

    Spatio-Temporal Graph Transformer Networks for Pedes- trian Trajectory Prediction

    Cunjun Yu, Xiao Ma, Jiawei Ren, Haiyu Zhao, and Shuai Yi. Spatio-Temporal Graph Transformer Networks for Pedes- trian Trajectory Prediction. In ECCV, 2020. 2, 3

    2020

  42. [50]

    Lanercnn: Distributed representations for graph-centric motion forecasting

    Wenyuan Zeng, Ming Liang, Renjie Liao, and Raquel Urta- sun. Lanercnn: Distributed representations for graph-centric motion forecasting. In IROS, 2021. 2

    2021

  43. [51]

    DeMo: Decou- pling Motion Forecasting into Directional Intentions and Dy- namic States

    Bozhou Zhang, Nan Song, and Li Zhang. DeMo: Decou- pling Motion Forecasting into Directional Intentions and Dy- namic States. In NeurIPS, 2024. 1, 2, 7, 8

    2024

  44. [52]

    TNT: Target-driveN Trajectory Pre- diction

    Hang Zhao, Jiyang Gao, Tian Lan, Chen Sun, Ben Sapp, Balakrishnan Varadarajan, Yue Shen, Yi Shen, Yuning Chai, Cordelia Schmid, et al. TNT: Target-driveN Trajectory Pre- diction. In CoRL, 2021. 2

    2021

  45. [53]

    Waslan- der, Hongsheng Li, and Yu Liu

    Yang Zhou, Hao Shao, Letian Wang, Steven L. Waslan- der, Hongsheng Li, and Yu Liu. SmartRefine: A Scenario- Adaptive Refinement Framework for Efficient Motion Pre- diction. In CVPR, 2024. 2, 8

    2024

  46. [54]

    HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction

    Zikang Zhou, Luyao Ye, Jianping Wang, Kui Wu, and Kejie Lu. HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction. In CVPR, 2022. 2, 6

    2022

  47. [55]

    Query-Centric Trajectory Prediction

    Zikang Zhou, Jianping Wang, Yung-Hui Li, and Yu-Kai Huang. Query-Centric Trajectory Prediction. In CVPR,

  48. [56]

    BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction

    Zikang Zhou, HU Haibo, Xinhong Chen, Jianping Wang, Nan Guan, Kui Wu, Yung-Hui Li, Yu-Kai Huang, and Chun Jason Xue. BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction. In NeurIPS, 2024. 3 10 D NUT: A Decoder-Only Model for Trajectory Predict...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.