REVIEW 3 major objections 5 minor 1 cited by
ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Driving
T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read ARTEMIS claims a state-of-the-art 87.0 PDMS on the NAVSIM navtest split by generating trajectory waypoints one at a time and routing each planning step to specialized experts.
desk verdict A capable end-to-end driving paper with a real but fixable evaluation gap: no ablation for the current control-command input, which could be a causal shortcut. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the autoregressive planning module with a mixture-of-experts block, i.e., a Transformer encoder that consumes the current planning query, the ego state, and the bird's-eye-view feature, and produces the next planning query; the MoE block contains one shared expert and five domain-specific experts with a two-layer router that selects the top two experts per step. The autoregressive loop imposes a factorization of the trajectory distribution into per-waypoint conditional probabilities, so each waypoint conditions on previously generated points, which is what captures temporal dependencies. The router is endogenous because expert selection depends on the scene and the trajectory so far, not on an external command label; this is the mechanism the paper claims prevents trajectory degradation when commands are ambiguous or wrong. The batch-reallocation strategy sorts samples by activated expert, applies each expert to contiguous blocks, and restores the order, which is what makes the sparse MoE trainable at large batch sizes. A two-stage refinement, kinematic optimization followed by cross-attention with agent and ego features, cleans up sampled waypoints before the final trajectory is scored.
What would settle it
Mask out or randomize the control-command components of the ego state at test time and rerun the NAVSIM evaluation; if PDMS falls well below the reported 87.0, the model was exploiting leaked command information, and a second check would be training a probe that predicts the future expert trajectory from the ego-state vector alone and measuring its accuracy.
Extended reading notes
Core claim
The central discovery claimed is that autoregressive waypoint generation combined with end-to-end learned expert routing yields state-of-the-art open-loop planning on NAVSIM. At each of eight half-second steps, the model predicts a distribution over the next waypoint, samples from it, and feeds the updated query sequence back into a Transformer that reads a bird's-eye-view scene representation. A router then activates two of five domain specialists plus one shared expert for that step, so different maneuvers such as turning versus going straight are handled by different sub-networks without any explicit maneuver label. The paper's experiments show that removing the autoregressive loop, removing the mixture-of-experts, or removing the final kinematic-and-attention refinement each lowers PDMS by at least 2.3 points, and that the learned router beats a driving-command-based expert selector by 3.5 PDMS points. On extended metrics the largest gains are in lane keeping (96.5) and time-to-collision (97.4), which the paper attributes to the temporal coherence of sequential generation and the router's ability to follow the correct expert in ambiguous scenes.
Load-bearing premise
The result depends on the assumption that the current steering or throttle command fed into the model as part of the ego state does not secretly encode the future path the model is supposed to predict; if it does, the high scores come from reading the command rather than understanding the scene.
Editorial extensions
If this is right
- End-to-end planners that emit all waypoints at once leave performance on the table: the paper's ablation attributes about 3.0 PDMS points to the autoregressive loop alone.
- Learned routing can replace hand-specified maneuver guidance in planning, which should make the system more robust when navigation commands and actual expert behavior disagree.
- The batch-reallocation speedup, reported as 26.2 times at batch size 256, removes a practical obstacle to training large MoE planners on limited GPU resources.
- Five to six experts appear to be the practical sweet spot for this dataset size, since the paper's ablation shows ten experts reduce PDMS by 1.5 points.
- The gains in lane keeping and time-to-collision indicate that this architecture is a stronger starting point for safety-critical driving metrics than one-shot baselines.
Reading between the lines
- One testable consequence not explored in the paper is that the control command inside the ego state could leak the future expert trajectory; zeroing out the command channel at test time and watching PDMS drop would settle whether the reported 87.0 reflects scene understanding rather than shortcut reading.
- The visualized expert specialization suggests each expert learns a maneuver primitive, so the same routing structure could be reused for interpretability, such as auditing which expert is responsible for a risky decision.
- The autoregressive-MoE recipe is not ego-specific and should transfer to multi-agent trajectory prediction, since the factorization and routing do not depend on the ego vehicle's role.
- Because NAVSIM is an open-loop benchmark, the closed-loop value of the router is speculative; under feedback, the benefit of endogenous routing could be larger or smaller than the scores here suggest.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents ARTEMIS, an end-to-end trajectory planning framework for autonomous driving that generates waypoints autoregressively and uses a Mixture-of-Experts (MoE) module with an 'intrinsic routing' mechanism to select experts based on scene queries. The perception pipeline follows Transfuser with camera and LiDAR inputs; the planner outputs a 4-second trajectory on the NAVSIM benchmark. The paper reports 87.0 PDMS and 83.1 EPDMS with a ResNet-34 backbone, claims state-of-the-art results on extended metrics, and proposes a 'batch reallocation' strategy to speed up MoE training. Ablations show that removing the autoregressive module, the MoE module, or the refinement module each degrades PDMS, and that intrinsic routing outperforms command-based expert assignment.
Significance. If the reported results are reproducible, ARTEMIS would be a strong end-to-end planner on NAVSIM: the extended-metric scores (especially LK 96.5 and TTC 97.4) substantially exceed prior ResNet-34 baselines, and the paper's component-wise ablations provide a clear picture of which design choices matter. The comparison between intrinsic routing and command-based assignment is a well-designed experiment that directly motivates the MoE routing. The paper also makes a practical contribution by addressing MoE training throughput. However, the empirical claim currently rests on a single run with no released code or weights, and the paper does not resolve a plausible causal-shortcut concern about the use of the current control command as an input. The value of the work will depend on whether those issues are addressed.
major comments (3)
- [Section II-B-2; Section III-E] The current ego state s0 is defined to include 'control commands' (Section II-B-2), but no experiment in Section III-E removes this command from the input. The paper explicitly motivates excluding historical ego states to avoid 'causal confusion' (citing [26], [27]), yet it never tests the analogous shortcut through the current command. In NAVSIM's open-loop PDMS/EPDMS evaluation, a trajectory prior conditioned on the command (e.g., go straight vs. turn) plus current kinematics could plausibly produce high scores without meaningful scene perception. The striking lane-keeping jump in Table II (LK 96.5 vs. 67.2 for the next best model) is consistent with the model learning a lane-following shortcut rather than perception-grounded planning. Because the central claim is that the autoregressive MoE architecture is responsible for the SOTA result, this control needs to be added: please ablate the command from s0, and ideally include a sensor-free ego-status-only baseline to bound how much score is attainable without perception.
- [Section III-D; Section III-E; Abstract] All quantitative tables (Tables I, II, III, IV, V, VI, VII) appear to report a single training run with no standard deviation, no number of seeds, and no released code or checkpoints. The margin over the closest ResNet-34 baseline on PDMS is 0.4 points (87.0 vs. 86.6 for Hydra-MDP++ in Table I), and the extended-metric margin is 2.5 EPDMS; without variance estimates, the SOTA claim is not robust. The abstract states 'Code will be available' but the repository is not linked and no code/checkpoints are provided. I ask the authors to report the mean and standard deviation over at least three seeds, and to release the trained checkpoints or otherwise specify a concrete procedure for independent verification.
- [Table VIII] The batch-reallocation speedup results in Table VIII are internally implausible: without reallocation, the reported training throughput decreases as batch size increases (2.31 samples/s at batch 64, 1.95 at 128, 1.61 at 256), which contradicts standard scaling behavior and suggests either a flawed baseline implementation or an inefficient sparse-computation path in the no-reallocation condition. Since the batch-reallocation strategy is one of the paper's three stated contributions, this is a load-bearing claim. Please re-measure the comparison, describe how the no-reallocation baseline handles top-k sparse expert computation, and report wall-clock time per training step in addition to samples per second.
minor comments (5)
- [Abstract; Section III-D] The abstract states that ARTEMIS 'demonstrates state-of-the-art performance on multiple metrics,' but on the original PDMS metric in Table I, ARTEMIS (87.0) is not state-of-the-art; DiffusionDrive reports 88.1. The sentence should be qualified to specify that the SOTA claim is with respect to the extended metrics (EPDMS) or to specific metrics such as EP, TTC, and LK.
- [Section I, contribution (1)] The claim of being the 'first investigation to incorporate the Mixture-of-Experts (MoE) into end-to-end autonomous driving' is in tension with reference [22], which also applies MoE to planning. The paper distinguishes [22] as using 'structured data representations as input,' but the wording should be sharpened to 'first sensor-based end-to-end planner with MoE' or similar to avoid confusion.
- [Section II-B-2, Eq. (2)] The notation E_shared and E_private is used both as sets of experts and as indices in the summation; this is slightly confusing. Please introduce separate symbols for the expert sets and the individual expert functions.
- [Section III-B] The EPDMS metric is introduced only by reference to Hydra-MDP++ [16]; the individual metric weights and the false-alarm penalty process are not described. A brief formal definition or a pointer to the exact equations in [16] would make the evaluation section self-contained.
- [Throughout] There are minor typographical issues: 'NA VSIM' spacing appears inconsistently ('NA VSIM' vs. 'NAVSIM'), and Section III-B contains 'the real word' instead of 'the real world.' Please proofread.
Circularity Check
No significant circularity: ARTEMIS's performance claims are external benchmark results, and the ablations are self-contained comparisons; the only self-citation is non-load-bearing.
full rationale
The paper's central claims are empirical scores on the NAVSIM navtest benchmark (87.0 PDMS and 83.1 EPDMS), evaluated with the official PDMS/EPDMS metrics. No fitted constant is relabeled as a prediction: all reported numbers come from trained model variants compared against published baselines, and the ablations (Tables III-VIII) compare the full model against variant models without any of the three main components. The only self-citation, reference [27], is a survey by overlapping authors cited alongside the external study [26] to motivate avoiding historical ego states; this citation is not load-bearing because the architecture is not justified by that survey and the empirical results are benchmarked independently. The concern that including the current control command in the ego state s0 could provide a causal shortcut is a potential correctness or benchmark-gaming risk, not a circularity: PDMS is not a mathematical transform of the command, and the model must still predict full trajectories scored against expert behavior. No equation in the paper reduces to its own input by construction, no fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. The derivation chain is therefore self-contained and the reported performance stands as an external empirical claim rather than a circular one.
Assumptions & free parameters
free parameters (5)
- Number of domain-specific experts E_private =
5
- Top-k active experts =
2
- Trajectory loss weight lambda_traj =
15
- NLL loss weight lambda_NLL =
0.2
- Number of cascaded refinement layers =
2
assumptions (4)
- domain assumption PDMS and EPDMS metrics on the NAVSIM benchmark are valid proxies for real-world driving performance.
- domain assumption The current control command supplied in the ego state s0 does not leak the future expert trajectory in the open-loop benchmark.
- standard math The autoregressive factorization p(Y|S) = product_t p(y_t | y_<t, S) is a valid and beneficial model for trajectory planning.
- domain assumption The trajectory refinement module enforces kinematic and smoothness constraints via its learnable weights.
Cite this review
Pith. "Pith review of ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Driving." pith.science (2026). https://pith.science/paper/HLVFIAZY
@misc{pith2026250419580,
author = {Pith},
title = {Pith review of: ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Driving},
year = {2026},
howpublished = {\url{https://pith.science/paper/HLVFIAZY}},
note = {Machine review of arXiv:2504.19580}
}
read the original abstract
This paper presents ARTEMIS, an end-to-end autonomous driving framework that combines autoregressive trajectory planning with Mixture-of-Experts (MoE). Traditional modular methods suffer from error propagation, while existing end-to-end models typically employ static one-shot inference paradigms that inadequately capture the dynamic changes of the environment. ARTEMIS takes a different method by generating trajectory waypoints sequentially, preserves critical temporal dependencies while dynamically routing scene-specific queries to specialized expert networks. It effectively relieves trajectory quality degradation issues encountered when guidance information is ambiguous, and overcomes the inherent representational limitations of singular network architectures when processing diverse driving scenarios. Additionally, we use a lightweight batch reallocation strategy that significantly improves the training speed of the Mixture-of-Experts model. Through experiments on the NAVSIM dataset, ARTEMIS exhibits superior competitive performance, achieving 87.0 PDMS and 83.1 EPDMS with ResNet-34 backbone, demonstrates state-of-the-art performance on multiple metrics.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving
DIVER uses RL-guided diffusion to produce diverse feasible trajectories from one ground-truth path, addressing mode collapse in imitation learning for autonomous driving.
Reference graph
Works this paper leans on
-
[26]
Is ego status all you need for open-loop end-to-end autonomous driving?
Z. Li, Z. Yu, S. Lan, J. Li, J. Kautz, T. Lu, and J. M. Alvarez, “Is ego status all you need for open-loop end-to-end autonomous driving?” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 864–14 873
work page 2024
-
[27]
End-to-end autonomous driving: Advancements and challenges,
D. Chu, R. Wang, J. Wang, Q. Hua, L. Lu, and C. Wu, “End-to-end autonomous driving: Advancements and challenges,” China Journal of Highway and Transport , vol. 37, no. 10, pp. 209–232, 2024
work page 2024
-
[22]
Generalizing motion planners with mixture of experts for autonomous driving,
Q. Sun, H. Wang, J. Zhan, F. Nie, X. Wen, L. Xu, K. Zhan, P. Jia, X. Lang, and H. Zhao, “Generalizing motion planners with mixture of experts for autonomous driving,” arXiv preprint arXiv:2410.15774 , 2024
arXiv 2024
-
[1]
Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers,
Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Yu, and J. Dai, “Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024
2024
-
[2]
Multi-agent tra- jectory prediction with difficulty-guided feature enhancement network,
G. Xin, D. Chu, L. Lu, Z. Deng, Y . Lu, and X. Wu, “Multi-agent tra- jectory prediction with difficulty-guided feature enhancement network,” IEEE Robotics and Automation Letters , 2025
work page 2025
-
[3]
Pluto: Pushing the limit of imita- tion learning-based planning for autonomous driving,
J. Cheng, Y . Chen, and Q. Chen, “Pluto: Pushing the limit of imita- tion learning-based planning for autonomous driving,” arXiv preprint arXiv:2404.14327, 2024
arXiv 2024
-
[4]
Planning-oriented autonomous driving,
Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, et al. , “Planning-oriented autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 17 853–17 862
work page 2023
-
[5]
Vad: Vectorized scene representation for efficient autonomous driving,
B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 8340–8350
2023
Show all 31 references
-
[6]
Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline,
P. Wu, X. Jia, L. Chen, J. Yan, H. Li, and Y . Qiao, “Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline,” Advances in Neural Information Processing Systems , vol. 35, pp. 6119–6132, 2022
2022
-
[7]
Multimodal motion prediction with stacked transformers,
Y . Liu, J. Zhang, L. Fang, Q. Jiang, and B. Zhou, “Multimodal motion prediction with stacked transformers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 7577– 7586
2021
-
[8]
Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting,
Y . Yuan, X. Weng, Y . Ou, and K. M. Kitani, “Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9813–9823
2021
-
[9]
R2p2: A reparameterized pushforward policy for diverse, precise generative path forecasting,
N. Rhinehart, K. M. Kitani, and P. Vernaza, “R2p2: A reparameterized pushforward policy for diverse, precise generative path forecasting,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 772–788
2018
-
[10]
Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers,
Y . Chen, Y . Wang, and Z. Zhang, “Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers,” arXiv preprint arXiv:2412.18607 , 2024
2024 arXiv
-
[11]
Drivingworld: Constructingworld model for autonomous driving via video gpt,
X. Hu, W. Yin, M. Jia, J. Deng, X. Guo, Q. Zhang, X. Long, and P. Tan, “Drivingworld: Constructingworld model for autonomous driving via video gpt,” arXiv preprint arXiv:2412.19505 , 2024
2024 arXiv
-
[12]
Drama: An efficient end-to-end motion planner for autonomous driving with mamba,
C. Yuan, Z. Zhang, J. Sun, S. Sun, Z. Huang, C. D. W. Lee, D. Li, Y . Han, A. Wong, K. P. Tee, et al. , “Drama: An efficient end-to-end motion planner for autonomous driving with mamba,” arXiv preprint arXiv:2408.03601, 2024
2024 arXiv
-
[13]
Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,
K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger, “Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,” IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 11, pp. 12 878–12 895, 2022
2022
-
[14]
Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,
S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang, “Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,” arXiv preprint arXiv:2402.13243 , 2024
2024 arXiv
-
[15]
Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation,
Z. Li, K. Li, S. Wang, S. Lan, Z. Yu, Y . Ji, Z. Li, Z. Zhu, J. Kautz, Z. Wu, et al. , “Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation,” arXiv preprint arXiv:2406.06978 , 2024
2024 arXiv
-
[16]
Hydra-mdp++: Advancing end-to-end driving via expert- guided hydra-distillation,
K. Li, Z. Li, S. Lan, Y . Xie, Z. Zhang, J. Liu, Z. Wu, Z. Yu, and J. M. Alvarez, “Hydra-mdp++: Advancing end-to-end driving via expert- guided hydra-distillation,” arXiv preprint arXiv:2503.12820 , 2025
2025 arXiv
-
[17]
Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving,
B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y . Zhang, Q. Zhang, et al., “Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving,” arXiv preprint arXiv:2411.15139 , 2024
2024 arXiv
-
[18]
Adaptive mixtures of local experts,
R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,” Neural computation, vol. 3, no. 1, pp. 79–87, 1991
1991
-
[19]
Deepseekmoe: Towards ultimate expert spe- cialization in mixture-of-experts language models,
D. Dai, C. Deng, C. Zhao, R. Xu, H. Gao, D. Chen, J. Li, W. Zeng, X. Yu, Y . Wu, et al. , “Deepseekmoe: Towards ultimate expert spe- cialization in mixture-of-experts language models,” arXiv preprint arXiv:2401.06066, 2024
2024 arXiv
-
[20]
Gpt-4 technical report,
J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023
2023 arXiv
-
[21]
Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,
W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,” Journal of Machine Learning Research , vol. 23, no. 120, pp. 1–39, 2022
2022
-
[23]
nuplan: A closed-loop ml- based planning benchmark for autonomous vehicles,
H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari, “nuplan: A closed-loop ml- based planning benchmark for autonomous vehicles,” arXiv preprint arXiv:2106.11810, 2021
2021 arXiv
-
[24]
Planning with diffu- sion for flexible behavior synthesis,
M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Planning with diffu- sion for flexible behavior synthesis,” arXiv preprint arXiv:2205.09991 , 2022
2022 arXiv
-
[25]
Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking,
D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, et al. , “Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking,” Ad- vances in Neural Information Processing Systems , vol. 37, pp. 28 706– ...
2024
-
[28]
Rethinking the open-loop evaluation of end-to- end autonomous driving in nuscenes,
J.-T. Zhai, Z. Feng, J. Du, Y . Mao, J.-J. Liu, Z. Tan, Y . Zhang, X. Ye, and J. Wang, “Rethinking the open-loop evaluation of end-to- end autonomous driving in nuscenes,” arXiv preprint arXiv:2305.10430, 2023
2023 arXiv
-
[29]
Para- drive: Parallelized architecture for real-time autonomous driving,
X. Weng, B. Ivanovic, Y . Wang, Y . Wang, and M. Pavone, “Para- drive: Parallelized architecture for real-time autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 15 449–15 458
2024
-
[30]
Goalflow: Goal-driven flow matching for multimodal trajec- tories generation in end-to-end autonomous driving,
Z. Xing, X. Zhang, Y . Hu, B. Jiang, T. He, Q. Zhang, X. Long, and W. Yin, “Goalflow: Goal-driven flow matching for multimodal trajec- tories generation in end-to-end autonomous driving,” arXiv preprint arXiv:2503.05689, 2025
2025
-
[31]
Openscene: The largest up-to-date 3d occu- pancy prediction benchmark in autonomous driving,
OpenScene Contributors, “Openscene: The largest up-to-date 3d occu- pancy prediction benchmark in autonomous driving,” 2023
2023
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.