Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Scaling Laws of Motion Forecasting and Planning -- Technical Report

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Joint motion forecasting and planning follows a language-model-style power-law scaling law, with compute-optimal driving models about 50 times smaller than language models.

desk verdict First credible scaling-law study for AV motion forecasting; the exponents are plausible but more fragile than the quoted error bars suggest. read the letter →

arxiv 2506.08228 v2 pith:GKLJYD2F submitted 2025-06-09 cs.LG cs.AIcs.RO

Mustafa Baniodeh , Kratarth Goel , Scott Ettinger , Carlos Fuertes , Ari Seff , Tim Shen , Cole Gulino , Chenjie Yang
show 9 more authors
Ghassen Jerfel Dokook Choe Rui Wang Benjamin Charrow Vinutha Kallem Sergio Casas Rami Al-Rfou Benjamin Sapp Dragomir Anguelov
This is my paper · ORCID
classification cs.LGcs.AIcs.RO
keywords scalinglawsmotionforecastingautonomousdrivingcompute-optimaltrainingclosed-loopevaluationinference-timecomputepower-lawlossjointtrajectoryprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This report argues that joint motion forecasting and planning for autonomous vehicles obeys the same empirical scaling laws that have been documented for language models. It shows that the cross-entropy loss of an encoder-decoder autoregressive transformer, trained on roughly 447,000 hours of driving data, falls as a power law of training compute, and that the compute-optimal allocation grows model size about 1.5 times as fast as dataset size. The report also claims that these loss improvements transfer to closed-loop driving, with the number of failed simulated scenarios decreasing as a power law of pretraining compute, and that sampling more trajectories at inference time lets smaller models compete with larger ones up to a crossover point. If the claims hold, training compute for driving should be spent on relatively small, data-hungry models, and open-loop loss can serve as a practical proxy for simulation safety.

What carries the argument

The argument is carried by a single parametric ansatz: the loss surface is $L(N,D)=E+A/N^\alpha+B/D^\beta$, where $N$ counts non-embedding transformer parameters, $D$ counts training examples, $A$ and $B$ are fitted constants, and $E$ is the irreducible data entropy. The machinery around it is an iso-FLOP study: 84 models from 900K to 118M parameters, trained in seven compute bands, with final validation losses fit to parabolas in $\log N$ and $\log D$ to locate each band's optimum, then power-law fits over the optima to extract the exponents. The models themselves are MotionLM-style encoder-decoder autoregressive transformers that predict discrete Verlet-wrapped displacement tokens for eight agents jointly, with a single flattened agent-time self-attention pass; the FLOPs count in Appendix A determines how parameter and data sizes map to compute. The loss ansatz does the work: it converts a vague belief that 'bigger is better' into two concrete exponents with error bars, which are the paper's main quantitative claims.

What would settle it

Retrain the same model family at a compute budget at least ten times larger than the largest budget used here and compare the measured validation loss with the extrapolated $L(C)=aC^b+L_\infty$ curve; if the data point lands more than three propagated standard deviations above the curve, the power-law extrapolation fails. A complementary check is to vary encoder and decoder depth independently while holding FLOPs fixed: if the loss-minimizing architecture shifts, the Appendix A FLOPs model is mis-ranking models and the $N_{\mathrm{opt}}$ and $D_{\mathrm{opt}}$ exponents are artifacts of that formula.

Watch

Extended reading notes

Core claim

On its own terms, the report's central discovery is a scaling law: for an encoder-decoder autoregressive transformer trained to jointly predict the future trajectories of eight road agents, validation cross-entropy follows $L(N,D) = E + A/N^\alpha + B/D^\beta$, with compute-optimal exponents $N_{\mathrm{opt}} \propto C^{0.63\pm0.08}$ and $D_{\mathrm{opt}} \propto C^{0.44\pm0.06}$. This means that at a given training-compute budget, the best driving model is roughly 50 times smaller than a compute-optimal language model at the same budget, while requiring far more data. The same compute-optimal series shows consistent power-law improvement in open-loop distance metrics (minADE and wADE) and, after short fine-tuning to route-conditioned single-agent planning, a power-law decrease in closed-loop simulation failures. Finally, the report shows that increasing inference-time sampling improves small-model metrics until a crossover, beyond which a larger model is more inference-compute efficient, and that models trained without the ego vehicle zero-shot predict ego-agent trajectories at a rate of about ten observed miles per two to three demonstrated miles.

Load-bearing premise

The load-bearing premise is that the true validation loss is exactly $L(N,D)=E+A/N^\alpha+B/D^\beta$ with the paper's FLOPs formula correctly ranking architectures, so the exponents fitted on 84 runs across a two-order-of-magnitude compute range remain valid when extrapolated beyond it.

Editorial extensions

If this is right

  • At a fixed training-compute budget, the optimal driving model is about 50 times smaller than the optimal language model, so compute invested in driving models should buy data, not a much larger network.
  • As the compute budget grows, model parameters should increase roughly 1.5 times faster than training examples, which means the model-size axis grows somewhat faster than the data axis.
  • Open-loop loss, minADE, and wADE improve together with compute, so loss-based hill climbing is a reliable stand-in for forecast quality.
  • Closed-loop simulation failures fall as a power law of pretraining compute, meaning open-loop scaling studies can be used to predict simulator safety rather than only offline accuracy.
  • Increasing inference-time samples makes smaller models competitive with larger ones up to a crossover, after which the larger model is the better use of inference FLOPs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: if the fitted entropy constant $E$ reflects the limited perception feature set (the paper names this as a hypothesis), then adding richer inputs such as camera or LiDAR tokens should lower the floor of the loss curve and could change the exponents; rerunning the smallest iso-FLOP band with vision inputs would test this directly.
  • The zero-shot transfer from other agents' trajectories suggests a data-collection strategy the paper only gestures at: passive multi-platform driving logs could substitute for ego-demonstrated miles, and the observed 10 observed miles per 2 to 3 demonstrated miles ratio gives a rough cost-equivalence to test on larger, truly passive corpora.
  • The inference-compute crossover implies an adaptive serving policy: pick the smallest model whose sampling curve has not yet saturated and increase samples per scenario up to that model's crossover; this is a direct, testable extension of the paper's inference-scaling analysis.
  • If the 1.5x model-to-data growth ratio carries over to other continuous-control planning tasks, robotics teams should bias fixed compute budgets toward data collection over model capacity, but that generalization is not established by this report.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper reports an empirical scaling-law study of an encoder-decoder autoregressive transformer for joint motion forecasting and planning, trained on a proprietary Waymo dataset of roughly 447,000 driving hours. Following the IsoFLOP methodology of Hoffmann et al. (2022), the authors train 84 models across 7 compute budgets, fit parabolic loss curves in log-parameter and log-data space within each iso-FLOP band, and extract compute-optimal exponents N_opt ∝ C^0.63±0.08 and D_opt ∝ C^0.44±0.06 (Section 4.1, Figure 4). They further report that compute-optimal loss is better described by a power law plus a constant than by a pure power law (Section 4.2), that open-loop distance metrics improve with compute (Section 5.1), that closed-loop failure count decreases with pretraining compute (Section 5.2), that inference-time sampling can make smaller models competitive up to a crossover point (Section 6), and that training on other agents' trajectories transfers to the ego-agent (Section 7). The paper is careful in several places to label the open-loop and closed-loop trends as preliminary, noting that more compute orders of magnitude are needed to establish power-law behavior.

Significance. If the central scaling exponents are reliable, this is one of the first large-scale demonstrations that compute-optimal scaling in an autonomous-driving prediction and planning task differs quantitatively from language modeling, with substantially smaller optimal models and a stronger preference for data. The paper also contributes a controlled comparison between open-loop and closed-loop scaling within a single architecture and loss, which is valuable for a field that has mostly seen confounded cross-model comparisons. Credit should be given for the explicit caveats in Sections 5.1 and 9 about the limited compute range, the reporting of fit-error propagation formulas in Appendix B, and the unusually large proprietary dataset. The main risk is that the headline exponents are extracted from a narrow, curvature-dominated compute range using a two-stage fitting procedure whose error bars do not include model-form or grid-selection uncertainty; this concern is central to the paper's primary quantitative claim and needs to be addressed before the exponents can be treated as predictive.

major comments (3)
  1. [Section 4.1, Figure 4, Appendix B] The quoted exponents N_opt ∝ C^0.63±0.08 and D_opt ∝ C^0.44±0.06 rest on a two-stage fit: a parabola in log x within each of seven iso-FLOP bands, followed by a power-law fit through the seven minima. Appendix B propagates only the parabola-fit covariance into the displayed error bands, but it does not account for model-form uncertainty or for the coarse model grid: model size is varied by changing layer count at only two width-to-depth ratios, giving roughly a dozen configurations per band. Because Section 4.2 shows that the loss is already approaching an irreducible floor over this same range, the parabola minima in the low-compute bands are weakly constrained, and the second-stage power law depends heavily on the least informative points. I ask the authors to demonstrate stability of the exponents by, for example, fitting the full loss surface directly to Eq. (4), by excluding or down-weighting the low-compute bands, or by bootstrap resampling over the 84 runs; otherwise the ±0.08 and ±0.06 uncertainties are likely optimistic.
  2. [Section 4.2, Figure 5] The paper states that adding a constant to the power-law fit, L(C) = aC^b + L∞, explains the compute-optimal loss data significantly better than a pure power law. This is not merely a remark about the loss curve; it directly bears on the exponent extraction in Section 4.1, because the same loss values determine the parabola minima used to estimate N_opt(C) and D_opt(C). If the loss is close to an irreducible floor, the low-compute bands may not lie on the same pure power-law surface assumed by the two-stage procedure. I request a sensitivity analysis that fits the parabola with an explicit floor, or that re-extracts the exponents after removing the lowest-compute bands and reports whether N_opt ∝ C^0.63 and D_opt ∝ C^0.44 remain within the stated uncertainties.
  3. [Section 5.2, Figure 10] The closed-loop claim — that the number of failures η decreases as a power law of pretraining compute — is presented with a fitted curve but without reporting the fit parameters, their uncertainties, the number of compute budgets used, the number of scenarios in the closed-loop validation set, or confidence intervals on η. Since η is a low-count failure metric and each model's progress-bias hyperparameter α is tuned individually to match assertiveness, the plot alone does not establish that the trend is statistically distinguishable from noise or that the α-calibration is not absorbing model differences. I ask that the closed-loop analysis report the quantitative fit constants and error bars, and ideally a bootstrap or confidence-interval calculation over scenarios.
minor comments (6)
  1. [Abstract and Section 3, Table 1] The abstract says a '500 thousand hours' driving dataset, while Table 1 reports 447 thousand hours; please make the rounding explicit or use a single number consistently.
  2. [Section 5.1, Eq. (5)] In the minADE definition, the notation ||y_t^k − ŷ_t||_2 is used for what should be a displacement error averaged over time; consider clarifying that the norm is taken over spatial coordinates.
  3. [Section 5.2, Eq. (7)] The progress-bias parameter α is introduced without stating its units or range; since α is calibrated per model, a brief description of the calibration procedure and how assertiveness is measured would help reproducibility.
  4. [Section 7, Figure 15] The '10 observed miles equivalent to 2–3 demonstrated miles' claim would be more informative with confidence intervals from the iso-loss fits, and with an explicit reminder that the data are collected on the same AV platform, as the text already notes.
  5. [Appendix B] The appendix correctly notes that first-order error propagation is biased for nonlinear fits, but it does not quantify how large this bias is for the reported 3σ bands; a short numeric comparison would be useful.
  6. [Figures 9 and 13] The qualitative trajectory figures would benefit from captions stating which model sizes and which sample counts are shown, and whether the same scenarios are used across panels.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the scaling exponents are empirical fits to the paper's own training runs and are not forced by construction; self-citations provide only architecture, dataset, and metric definitions.

full rationale

All load-bearing claims are derived from the paper's own 84 training runs via standard iso-FLOP analysis (Section 4.1, Figures 2-4), not from an equation that is equivalent to the target result by construction. The functional form L(N,D)=E+A/N^alpha+B/D^beta (Eq. 4) is an externally motivated assumption taken from Hoffmann et al. (2022) and Ghorbani et al. (2021), and the exponents N_opt proportional to C^0.63 and D_opt proportional to C^0.44 are estimated by first fitting per-band parabolas and then a power law through the minima; nothing in that procedure forces the reported exponents. The paper explicitly cautions that power-law fits for open-loop metrics are provisional (Section 5.1, Figure 7), and Section 4.2 reports that a power-law-plus-constant fits the loss better, which is an honest model-selection statement rather than a circular step. Self-citations to MotionLM (Seff et al., 2023), Wayformer (Nayakanti et al., 2022), and WOMD (Ettinger et al., 2021) supply the architecture, scene-encoder design, and dataset/metric interface; they do not supply the scaling-law result, so they are not load-bearing. The skeptic's concern about the narrow compute range and optimistic error bars is a robustness and statistical-inference issue, not circular reasoning: no fitted parameter is renamed as an independent prediction, and no equation reduces to its own inputs by definition.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claims rest on fitted power-law forms, an assumed FLOPs model, and an internal simulator. The ledger is therefore dominated by fitting constants and domain assumptions rather than free-standing physical laws. No new entities such as forces, particles, or conserved quantities are introduced.

free parameters (7)
  • Loss-scaling constants E, A, B, alpha, beta (Eq. 4) = Not reported numerically
    Fitted to the 84 training runs; they define the claimed L(N,D) power law and determine all downstream compute-optimal statements.
  • IsoFLOP parabola parameters a, x_opt, L_opt per compute band = Per-band fit values not tabulated
    Used to extract N_opt and D_opt for each of the seven compute budgets.
  • Power-law coefficients for N_opt(C) and D_opt(C) = N_opt ∝ C^(0.63±0.08), D_opt ∝ C^(0.44±0.06)
    These exponents are the paper's central quantitative result, fit to the parabola minima.
  • L(C) = aC^b + L_inf fit constants = Not reported numerically
    Used to support the claim that loss scaling follows a power law plus an irreducible constant.
  • Closed-loop progress-bias alpha (Eq. 7) = Tuned per model on validation set
    Calibrated to equalize assertiveness across models so closed-loop failure counts are comparable.
  • Closed-loop failure-count fit constants = Not reported numerically
    Underlie the claim that closed-loop failures decrease as a power law of pretraining compute.
  • Iso-loss fit constants for cross-agent data equivalency = Result: 10 observed miles ≈ 2-3 demonstrated miles
    Derived from power-law fits to AV loss versus traveled miles; depends on those fit constants.
assumptions (5)
  • domain assumption Training loss obeys the functional form L(N,D) = E + A/N^alpha + B/D^beta (Eq. 4).
    Adopted from language-model scaling literature without derivation; it fixes the functional form being fitted and cannot be falsified by the fits themselves.
  • domain assumption The Appendix A FLOPs model accurately ranks training cost across model sizes.
    IsoFLOP bands and compute-optimal exponents depend on this cost model, which ignores embedding and normalization layers and assumes a specific attention/feed-forward cost.
  • domain assumption Validation cross-entropy is the correct scalar objective for driving quality.
    The paper tests correlation with open-loop and closed-loop metrics but does not prove that loss reduction is the causal mechanism.
  • domain assumption The closed-loop simulator with logged playback and imitation-learned agents is a faithful proxy for real-world driving safety.
    The closed-loop scaling claim is only about this internal simulator, not real AV deployment.
  • domain assumption Compute-optimal exponents fitted from 900K to 118M parameter models extrapolate to larger compute budgets.
    The paper extrapolates N_opt ∝ C^0.63 and D_opt ∝ C^0.44 beyond the fitted range to discuss optimal scaling, which is an untested assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling Laws of Motion Forecasting and Planning -- Technical Report." pith.science (2026). https://pith.science/paper/GKLJYD2F

@misc{pith2026250608228,
  author       = {Pith},
  title        = {Pith review of: Scaling Laws of Motion Forecasting and Planning -- Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GKLJYD2F}},
  note         = {Machine review of arXiv:2506.08228}
}
read the original abstract

We study the empirical scaling laws of a family of encoder-decoder autoregressive transformer models on the task of joint motion forecasting and planning in the autonomous driving domain. Using a 500 thousand hours driving dataset, we demonstrate that, similar to language modeling, model performance improves as a power-law function of the total compute budget, and we observe a strong correlation between model training loss and model evaluation metrics. Most interestingly, closed-loop metrics also improve with scaling, which has important implications for the suitability of open-loop metrics for model development and hill climbing. We also study the optimal scaling of the number of transformer parameters and the training data size for a training compute-optimal model. We find that as the training compute budget grows, optimal scaling requires increasing the model size 1.5x as fast as the dataset size. We also study inference-time compute scaling, where we observe that sampling and clustering the output of smaller models makes them competitive with larger models, up to a crossover point beyond which a larger models becomes more inference-compute efficient. Overall, our experimental results demonstrate that optimizing the training and inference-time scaling properties of motion forecasting and planning models is a key lever for improving their performance to address a wide variety of driving scenarios. Finally, we briefly study the utility of training on general logged driving data of other agents to improve the performance of the ego-agent, an important research area to address the scarcity of robotics data for large capacity models training.

Figures

Figures reproduced from arXiv: 2506.08228 by the authors.

Figure 1
Figure 1. The model architecture is a pair of encoder/decoder Transformer networks. This model takes multimodal [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Training cross-entropy loss for all experiments as a function of compute. The losses are presented without [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Final losses as a function of the total number of non-embedding parameters (left), and the total number of [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Training-compute optimal number of model parameters ( [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Training compute-optimal loss as a function of total compute budget. The [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Joint predictions of multiple agents in the scene improves performance for all agents. The slope (exponent) [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Open-loop distance metrics, minADE and wADE, of compute-optimal models as a function of the total [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Comparison of the model number of parameters optimized for Eval cross-entropy versus minADE. For smaller [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Qualitative differences between models of increasing size. Blue/green - predicted trajectories, Red - Ground [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Performance in closed-loop, as indicated by the number of failures [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Distance metrics for 3 models of increasing capacity. Each model has a range of inference compute for [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Coverage metrics for 3 models of increasing capacity. Similar to the distance metrics, each model has a [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Qualitative differences between clustered trajectories from 16 and 1024 rollouts. Blue/green - predicted [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Compute-optimal models trained on the 8-agent prediction task, excluding the AV, and zero-shot evaluated [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Compute-optimal models trained on the 8-agent prediction task, excluding the AV, and zero-shot evaluated [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When to Plan: Learning to Select Between Reactive Control and Deliberative Planning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    An RL-trained meta-policy that uses ensemble uncertainty to choose between a cheap reactive policy and costly planning reaches goals faster than fixed baselines and adapts as the reactive policy improves.

  2. Vernata: Self-Supervised Learning of LiDAR Point Representations

    cs.CV 2026-08 conditional novelty 4.0 of 10

    Vernata extends Sonata with sparse-view augmentation, a Sinkhorn-Knopp memory bank, and high-resolution 2D-to-3D distillation, improving LiDAR semantic segmentation mIoU by 5.9 to 7.3 points over baselines.

Reference graph

Works this paper leans on

31 extracted references · 5 canonical work pages · cited by 2 Pith papers

  1. [1]

    Bouzidi, C

    M.-K. Bouzidi, C. Schlauch, N. Scheuerer, Y . Yao, N. Klein, D. Göhring, and J. Reichardt. Closing the loop: Motion prediction models beyond open-loop benchmarks. arXiv preprint arXiv:2505.05638,

    arXiv

  2. [2]

    ,θn), that depends on estimated parameters ˆθi, and one also has access to their estimated covariance σi,j =cov( ˆθi, ˆθ j), how does one estimate the errors in f(x;, ˆθ0,

    +m∗(28Dd2 +4dD 2 +4Ed 2 +4dDE) Number of parameters:(12n+16m)∗d 2 B Fits Error Propagation The key question in error propagation is that if one has a function f(x;θ 0, . . . ,θn), that depends on estimated parameters ˆθi, and one also has access to their estimated covariance σi,j =cov( ˆθi, ˆθ j), how does one estimate the errors in f(x;, ˆθ0, . . . ,ˆθn)...

    work page 2006

  3. [4]

    URL https://arxiv.org/abs/2410.06158. J. Cheng, Y . Chen, X. Mei, B. Yang, B. Li, and M. Liu. Rethinking imitation-based planners for autonomous driving. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14123–14130. IEEE,

    arXiv 2024

  4. [8]

    URLhttps://arxiv.org/abs/2407.21783. Z. Hao, D. W. Romero, T.-Y . Lin, and M.-Y . Liu. Meshtron: High-fidelity, artist-like 3d mesh generation at scale,

    arXiv

  5. [9]

    URLhttps://arxiv.org/abs/2412.09548. T. Henighan, J. Kaplan, M. Katz, M. Chen, C. Hesse, J. Jackson, H. Jun, T. B. Brown, P. Dhariwal, S. Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701,

    arXiv 2010

  6. [11]

    URLhttps://arxiv.org/abs/2102.01293. D. Hernandez, T. Brown, T. Conerly, N. DasSarma, D. Drain, S. El-Showk, N. Elhage, Z. Hatfield-Dodds, T. Henighan, T. Hume, S. Johnston, B. Mann, C. Olah, C. Olsson, D. Amodei, N. Joseph, J. Kaplan, and S. McCandlish. Scaling laws and interpretability of learning from repeated data,

    arXiv

  7. [12]

    URLhttps://arxiv.org/abs/2205.10487. J. Hestness, S. Narang, N. Ardalani, G. Diamos, H. Jun, H. Kianinejad, M. M. A. Patwary, Y . Yang, and Y . Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409,

    arXiv

  8. [14]

    A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080,

    arXiv

Show all 31 references
  1. [16]

    URLhttps://arxiv.org/abs/2403.13331. J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361,

    2001 arXiv

  2. [17]

    URLhttps://arxiv.org/abs/2406.09246. S. Kudugunta, A. Kusupati, T. Dettmers, K. Chen, I. Dhillon, Y . Tsvetkov, H. Hajishirzi, S. Kakade, A. Farhadi, P. Jain, et al. Matformer: Nested transformer for elastic inference. arXiv preprint arXiv:2310.07707,

    arXiv

  3. [18]

    Z. Li, S. Wang, S. Lan, Z. Yu, Z. Wu, and J. M. Alvarez. Hydra-next: Robust closed-loop driving with open-loop training. arXiv preprint arXiv:2503.12030,

    arXiv

  4. [20]

    doi: 10.1109/ICRA40945.2020.9197340. N. Muennighoff, A. M. Rush, B. Barak, T. L. Scao, A. Piktus, N. Tazi, S. Pyysalo, T. Wolf, and C. Raffel. Scaling data-constrained language models,

    2020

  5. [21]

    URLhttps://arxiv.org/abs/2305.16264. N. Nayakanti, R. Al-Rfou, A. Zhou, K. Goel, K. S. Refaat, and B. Sapp. Wayformer: Motion forecasting via simple & efficient attention networks. ArXiv, abs/2207.05844,

    arXiv

  6. [22]

    URLhttps://arxiv.org/abs/2106.08417. K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. Fast: Efficient action tokenization for vision-language-action models,

    arXiv

  7. [23]

    URLhttps://arxiv.org/abs/2501.09747. N. Sardana and J. Frankle. Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. arXiv preprint arXiv:2401.00448,

    arXiv

  8. [24]

    URLhttps://arxiv.org/abs/2306.17770. C. Snell, J. Lee, K. Xu, and A. Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314,

    arXiv

  9. [25]

    URLhttps://arxiv.org/abs/2312.13139. D. Yang, J. Tian, X. Tan, R. Huang, S. Liu, X. Chang, J. Shi, S. Zhao, J. Bian, Z. Zhao, X. Wu, and H. Meng. Uniaudio: An audio foundation model toward universal audio generation,

    arXiv

  10. [26]

    URL https://arxiv.org/abs/ 2310.00704. B. Yao, A. Ganesh, Z. Li, and A. Petiushko. At nuro, we conduct an ai- first approach by using ml everywhere,

    arXiv

  11. [27]

    Accessed: 2025-03-12

    URL https://medium.com/nuro/ at-nuro-we-conduct-an-ai-first-approach-by-using-ml-everywhere-5faf4657fff3 . Accessed: 2025-03-12. S. Ye, J. Jang, B. Jeon, S. Joo, J. Yang, B. Peng, A. Mandlekar, R. Tan, Y .-W. Chao, B. Y . Lin, L. Liden, K. Lee, J. Gao, L. Zettlemoyer, D. Fox, ...

    2025

  12. [28]

    URL https://arxiv.org/abs/ 2410.11758. C. Yu, X. Ma, J. Ren, H. Zhao, and S. Yi. Spatio-temporal graph transformer networks for pedestrian trajectory prediction,

    arXiv

  13. [29]

    URLhttps://arxiv.org/abs/2005.08514. Y . Yuan, X. Weng, Y . Ou, and K. M. Kitani. Agentformer: Agent-aware transformers for socio-temporal multi- agent forecasting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9813–9823, October

    2005 arXiv

  14. [30]

    Zheng, Z

    Y . Zheng, Z. Xia, Q. Zhang, T. Zhang, B. Lu, X. Huo, C. Han, Y . Li, M. Yu, B. Jin, et al. Preliminary investigation into data scaling laws for imitation learning-based end-to-end autonomous driving. arXiv preprint arXiv:2412.02689,

  15. [31]

    Appendices A FLOPs and Parameters Computation When computing flops and number of parameters, we include einsums in attention and feed-forward layers

    URLhttps://arxiv.org/abs/2405.17372. Appendices A FLOPs and Parameters Computation When computing flops and number of parameters, we include einsums in attention and feed-forward layers. We ignore embedding and normalization layers. When computing flops, each multiply-add coun...

    arXiv

  16. [2009]

    URLhttps://doi.org/10.1137/070710111

    doi: 10.1137/070710111. URLhttps://doi.org/10.1137/070710111. D. Dauner, M. Hallgarten, A. Geiger, and K. Chitta. Parting with misconceptions about learning-based vehicle motion planning, 2023a. URLhttps://arxiv.org/abs/2306.07962. D. Dauner, M. Hallgarten, A. Geiger, and K. C...

    arXiv

  17. [2017]

    Hoffmann, S

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556,

    arXiv

  18. [2020]

    URLhttps://arxiv.org/abs/2005.14165. S. Casas, A. Sadat, and R. Urtasun. Mp3: A unified model to map, perceive, predict and plan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14403–14412,

    2005 arXiv

  19. [2021]

    Ghorbani, O

    B. Ghorbani, O. Firat, M. Freitag, A. Bapna, M. Krikun, X. Garcia, C. Chelba, and C. Cherry. Scaling laws for neural machine translation. arXiv preprint arXiv:2109.07740,

    arXiv

  20. [2022]

    URLhttps://arxiv.org/abs/2104.00563. A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, A. Zhang, A....

    arXiv

  21. [2023]

    Huang, E

    X. Huang, E. M. Wolff, P. Vernaza, T. Phan-Minh, H. Chen, D. S. Hayden, M. Edmonds, B. Pierce, X. Chen, P. E. Jacob, et al. Drivegpt: Scaling autoregressive behavior models for driving. arXiv preprint arXiv:2412.14415,

    arXiv

  22. [2024]

    URLhttps://arxiv.org/abs/2407.21787. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Ch...

    arXiv

  23. [2025]

    Mercat, T

    J. Mercat, T. Gilles, N. El Zoghby, G. Sandou, D. Beauvois, and G. P. Gil. Multi-head attention for multi-modal joint vehicle motion forecasting. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 9638–9644,

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.