{"id":"3063f32b-7b3d-4aea-baab-6cada7a07c14","arxiv_id":"2504.19580","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An autoregressive mixture-of-experts planner for end-to-end driving achieves 87.0 PDMS and 83.1 EPDMS on NAVSIM, with ablations supporting each component.","lead":"ARTEMIS is an end-to-end autonomous driving planner that generates trajectory waypoints one at a time and routes each step through a mixture of specialized expert networks. On the NAVSIM benchmark it reports 87.0 PDMS and 83.1 EPDMS with a ResNet-34 backbone, competitive with or above several recent end-to-end planners.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Current control command in ego state may be a causal shortcut; no ablation drops it despite the paper's own causal-confusion discussion.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern: the current control command in s0 is an ego-status feature that correlates strongly with the expert's future trajectory, and the paper's causal-confusion discussion does not address it. The concern is concrete and testable: Section III-E reports ablations over AME, MoE, and TR, but no ablation removing the command. The qualitative claim in Fig. 6(a) that ARTEMIS can ignore erroneous commands is only one anecdote, not a systematic control. I do not see an internal inconsistency that would justify rejecting the paper outright; the reported numbers could be genuine if the command is not the dominant factor. However, the absence of code, checkpoints, multi-seed variance, and this specific command ablation leaves the central empirical claim unverified. The reader's CONDITIONAL verdict remains appropriate, with the added condition that the authors supply the command-removal experiment and an ego-status-only baseline.","tokens_in":11608,"tokens_out":7740,"duration_ms":86925,"concrete_test":"Retrain ARTEMIS on navtrain with the control-command channels of s0 zeroed (or replaced by constants) and re-evaluate on navtest; additionally train a lightweight MLP baseline that receives only s0 (command, velocity, acceleration) and no sensor features and outputs the 8-waypoint trajectory, evaluating the same metrics. If the command-ablated ARTEMIS still achieves roughly 87.0 PDMS, the command is not the source; if the sensor-free baseline approaches 87.0 PDMS / 83.1 EPDMS, the headline numbers are dominated by ego-status shortcut rather than the proposed perception/planning architecture.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section II-B-2 defines the current ego state s0 ∈ R8 as including 'control commands, 2D velocity, and acceleration,' feeding the same command the expert driver used to produce the NAVSIM trajectory. The paper explicitly motivates excluding historical ego states to avoid 'causal confusion' (citing [26], [27]), but it never tests the analogous shortcut through the current command. NAVSIM's open-loop PDMS/EPDMS metrics score a planned trajectory against a non-reactive rollout; a trajectory prior conditioned on command (e.g., go straight vs. turn) plus current kinematics is highly predictive of the expert trajectory in simple navtest scenes, so high PDMS can be achieved without scene understanding. The evidence in Section III-E contains no ablation with s0 deprived of the command, and no ego-status-only baseline; the extended-metric jump in LK (96.5 vs 67.2 for the runner-up, Table II) is exactly the signature of a lane-following shortcut rather than perception-grounded planning. Since all baselines also receive ego state, the relative ranking may survive, but the paper's central claim that the MoE/autoregressive architecture is responsible for the SOTA score is not established without controlling this input.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ARTEMIS, an end-to-end trajectory planning framework for autonomous driving that generates waypoints autoregressively and uses a Mixture-of-Experts (MoE) module with an 'intrinsic routing' mechanism to select experts based on scene queries. The perception pipeline follows Transfuser with camera and LiDAR inputs; the planner outputs a 4-second trajectory on the NAVSIM benchmark. The paper reports 87.0 PDMS and 83.1 EPDMS with a ResNet-34 backbone, claims state-of-the-art results on extended metrics, and proposes a 'batch reallocation' strategy to speed up MoE training. Ablations show that removing the autoregressive module, the MoE module, or the refinement module each degrades PDMS, and that intrinsic routing outperforms command-based expert assignment.","tokens_in":11861,"tokens_out":5161,"duration_ms":54451,"significance":"If the reported results are reproducible, ARTEMIS would be a strong end-to-end planner on NAVSIM: the extended-metric scores (especially LK 96.5 and TTC 97.4) substantially exceed prior ResNet-34 baselines, and the paper's component-wise ablations provide a clear picture of which design choices matter. The comparison between intrinsic routing and command-based assignment is a well-designed experiment that directly motivates the MoE routing. The paper also makes a practical contribution by addressing MoE training throughput. However, the empirical claim currently rests on a single run with no released code or weights, and the paper does not resolve a plausible causal-shortcut concern about the use of the current control command as an input. The value of the work will depend on whether those issues are addressed.","major_comments":[{"comment":"The current ego state s0 is defined to include 'control commands' (Section II-B-2), but no experiment in Section III-E removes this command from the input. The paper explicitly motivates excluding historical ego states to avoid 'causal confusion' (citing [26], [27]), yet it never tests the analogous shortcut through the current command. In NAVSIM's open-loop PDMS/EPDMS evaluation, a trajectory prior conditioned on the command (e.g., go straight vs. turn) plus current kinematics could plausibly produce high scores without meaningful scene perception. The striking lane-keeping jump in Table II (LK 96.5 vs. 67.2 for the next best model) is consistent with the model learning a lane-following shortcut rather than perception-grounded planning. Because the central claim is that the autoregressive MoE architecture is responsible for the SOTA result, this control needs to be added: please ablate the command from s0, and ideally include a sensor-free ego-status-only baseline to bound how much score is attainable without perception.","section":"Section II-B-2; Section III-E"},{"comment":"All quantitative tables (Tables I, II, III, IV, V, VI, VII) appear to report a single training run with no standard deviation, no number of seeds, and no released code or checkpoints. The margin over the closest ResNet-34 baseline on PDMS is 0.4 points (87.0 vs. 86.6 for Hydra-MDP++ in Table I), and the extended-metric margin is 2.5 EPDMS; without variance estimates, the SOTA claim is not robust. The abstract states 'Code will be available' but the repository is not linked and no code/checkpoints are provided. I ask the authors to report the mean and standard deviation over at least three seeds, and to release the trained checkpoints or otherwise specify a concrete procedure for independent verification.","section":"Section III-D; Section III-E; Abstract"},{"comment":"The batch-reallocation speedup results in Table VIII are internally implausible: without reallocation, the reported training throughput decreases as batch size increases (2.31 samples/s at batch 64, 1.95 at 128, 1.61 at 256), which contradicts standard scaling behavior and suggests either a flawed baseline implementation or an inefficient sparse-computation path in the no-reallocation condition. Since the batch-reallocation strategy is one of the paper's three stated contributions, this is a load-bearing claim. Please re-measure the comparison, describe how the no-reallocation baseline handles top-k sparse expert computation, and report wall-clock time per training step in addition to samples per second.","section":"Table VIII"}],"minor_comments":[{"comment":"The abstract states that ARTEMIS 'demonstrates state-of-the-art performance on multiple metrics,' but on the original PDMS metric in Table I, ARTEMIS (87.0) is not state-of-the-art; DiffusionDrive reports 88.1. The sentence should be qualified to specify that the SOTA claim is with respect to the extended metrics (EPDMS) or to specific metrics such as EP, TTC, and LK.","section":"Abstract; Section III-D"},{"comment":"The claim of being the 'first investigation to incorporate the Mixture-of-Experts (MoE) into end-to-end autonomous driving' is in tension with reference [22], which also applies MoE to planning. The paper distinguishes [22] as using 'structured data representations as input,' but the wording should be sharpened to 'first sensor-based end-to-end planner with MoE' or similar to avoid confusion.","section":"Section I, contribution (1)"},{"comment":"The notation E_shared and E_private is used both as sets of experts and as indices in the summation; this is slightly confusing. Please introduce separate symbols for the expert sets and the individual expert functions.","section":"Section II-B-2, Eq. (2)"},{"comment":"The EPDMS metric is introduced only by reference to Hydra-MDP++ [16]; the individual metric weights and the false-alarm penalty process are not described. A brief formal definition or a pointer to the exact equations in [16] would make the evaluation section self-contained.","section":"Section III-B"},{"comment":"There are minor typographical issues: 'NA VSIM' spacing appears inconsistently ('NA VSIM' vs. 'NAVSIM'), and Section III-B contains 'the real word' instead of 'the real world.' Please proofread.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The skeleton concern from the stress-test is valid: the control-command input is a plausible shortcut and the paper's own causal-confusion motivation makes its absence from the ablations conspicuous. I would not reject on this basis because it is addressable with additional experiments. I also note that the paper's self-assessment as 'first' MoE end-to-end planner is carefully hedged against [22], but the framing should be checked. The lack of code and single-run results are the standard weaknesses of this submission type; if the authors release the code and add the command-removal ablation, I would be willing to see a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nARTEMIS is a serious engineering effort: autoregressive waypoint generation plus a shared/domain-specialized MoE with intrinsic routing for raw sensor end-to-end driving. The NAVSIM numbers are competitive (87.0 PDMS, 83.1 EPDMS), and the ablations in Tables III-VII actually support each design choice. The batch reallocation trick is a genuine engineering contribution, and I believe the speedup numbers are plausible. The paper deserves a serious referee.\n\nThe soft spot is exactly where the stress-test points. The authors motivate excluding historical ego states to avoid causal confusion, citing [26], [27], but they keep the current control command in the ego state. On an open-loop benchmark like NAVSIM, the expert trajectory is strongly correlated with the command (follow lane, turn left/right), so conditioning on command plus velocity/acceleration gives the model a large head start. The jump in LK (96.5 vs 67.2 for Hydra-MDP++) is a warning sign: lane keeping is the easiest metric to game this way. The paper has no ablation that removes the command from the ego-state input, so the central claim that the MoE/autoregressive architecture is responsible for the SOTA margin is not fully established. This is fixable: run the same model with and without the command, or at least compare against a baseline that excludes it.\n\nOther issues are more minor. Single-run numbers with no error bars, code not yet released, and the 'first end-to-end MoE' claim is a bit too broad given [22] (they qualify it, but still). Table VIII's without-reallocation throughput is suspiciously low, but that's a detail.\n\nIf I were the editor, I would send this to peer review. The architecture is interesting, the empirical work is mostly careful, and the shortcut concern is testable. The authors should be asked to add the command-input ablation, release code and checkpoints, and tone down the SOTA claim until the shortcut is ruled out. My guess is the relative ranking across methods survives, but the absolute story needs the control.","headline":"A capable end-to-end driving paper with a real but fixable evaluation gap: no ablation for the current control-command input, which could be a causal shortcut.","tokens_in":12429,"tokens_out":2424,"would_cite":false,"duration_ms":25426,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ARTEMIS claims a state-of-the-art 87.0 PDMS on the NAVSIM navtest split by generating trajectory waypoints one at a time and routing each planning step to specialized experts.","keywords":["autonomous driving","end-to-end planning","autoregressive trajectory prediction","mixture of experts","NAVSIM","PDMS","bird's-eye-view representation","expert routing"],"falsifier":"Mask out or randomize the control-command components of the ego state at test time and rerun the NAVSIM evaluation; if PDMS falls well below the reported 87.0, the model was exploiting leaked command information, and a second check would be training a probe that predicts the future expert trajectory from the ego-state vector alone and measuring its accuracy.","tokens_in":11431,"feed_emoji":"🚗","tokens_out":6337,"duration_ms":64380,"temperature":0.7,"pith_summary":"This paper contends that end-to-end driving planning improves when the trajectory is built step by step and each step is handled by a mixture-of-experts network that routes the scene to a few specialized sub-networks. The proposed system, ARTEMIS, reports 87.0 Predictive Driving Model Score (PDMS) and 83.1 Extended PDMS on the NAVSIM navtest split with a ResNet-34 backbone, beating prior end-to-end planners on the same benchmark. The design matters because one-shot planners that emit the whole path at once cannot adapt to evolving scenes, and single-network planners cannot represent the several reasonable maneuvers a scene allows. ARTEMIS also replaces explicit driving-command guidance with a learned router, which the paper argues avoids quality loss when commands disagree with actual expert trajectories. A batch-reallocation trick lets MoE training run at useful speeds, reported as a large throughput gain.","feed_headline":"Autoregressive MoE planner hits 87 PDMS on NAVSIM","feed_subtitle":"Step-by-step waypoint generation with trained expert routing beats one-shot end-to-end baselines on the navtest benchmark.","key_machinery":"The load-bearing object is the autoregressive planning module with a mixture-of-experts block, i.e., a Transformer encoder that consumes the current planning query, the ego state, and the bird's-eye-view feature, and produces the next planning query; the MoE block contains one shared expert and five domain-specific experts with a two-layer router that selects the top two experts per step. The autoregressive loop imposes a factorization of the trajectory distribution into per-waypoint conditional probabilities, so each waypoint conditions on previously generated points, which is what captures temporal dependencies. The router is endogenous because expert selection depends on the scene and the trajectory so far, not on an external command label; this is the mechanism the paper claims prevents trajectory degradation when commands are ambiguous or wrong. The batch-reallocation strategy sorts samples by activated expert, applies each expert to contiguous blocks, and restores the order, which is what makes the sparse MoE trainable at large batch sizes. A two-stage refinement, kinematic optimization followed by cross-attention with agent and ego features, cleans up sampled waypoints before the final trajectory is scored.","core_discovery":"The central discovery claimed is that autoregressive waypoint generation combined with end-to-end learned expert routing yields state-of-the-art open-loop planning on NAVSIM. At each of eight half-second steps, the model predicts a distribution over the next waypoint, samples from it, and feeds the updated query sequence back into a Transformer that reads a bird's-eye-view scene representation. A router then activates two of five domain specialists plus one shared expert for that step, so different maneuvers such as turning versus going straight are handled by different sub-networks without any explicit maneuver label. The paper's experiments show that removing the autoregressive loop, removing the mixture-of-experts, or removing the final kinematic-and-attention refinement each lowers PDMS by at least 2.3 points, and that the learned router beats a driving-command-based expert selector by 3.5 PDMS points. On extended metrics the largest gains are in lane keeping (96.5) and time-to-collision (97.4), which the paper attributes to the temporal coherence of sequential generation and the router's ability to follow the correct expert in ambiguous scenes.","pith_inferences":["One testable consequence not explored in the paper is that the control command inside the ego state could leak the future expert trajectory; zeroing out the command channel at test time and watching PDMS drop would settle whether the reported 87.0 reflects scene understanding rather than shortcut reading.","The visualized expert specialization suggests each expert learns a maneuver primitive, so the same routing structure could be reused for interpretability, such as auditing which expert is responsible for a risky decision.","The autoregressive-MoE recipe is not ego-specific and should transfer to multi-agent trajectory prediction, since the factorization and routing do not depend on the ego vehicle's role.","Because NAVSIM is an open-loop benchmark, the closed-loop value of the router is speculative; under feedback, the benefit of endogenous routing could be larger or smaller than the scores here suggest."],"forward_implications":["End-to-end planners that emit all waypoints at once leave performance on the table: the paper's ablation attributes about 3.0 PDMS points to the autoregressive loop alone.","Learned routing can replace hand-specified maneuver guidance in planning, which should make the system more robust when navigation commands and actual expert behavior disagree.","The batch-reallocation speedup, reported as 26.2 times at batch size 256, removes a practical obstacle to training large MoE planners on limited GPU resources.","Five to six experts appear to be the practical sweet spot for this dataset size, since the paper's ablation shows ten experts reduce PDMS by 1.5 points.","The gains in lane keeping and time-to-collision indicate that this architecture is a stronger starting point for safety-critical driving metrics than one-shot baselines."],"supporting_citations":[{"why":"Supplies the NAVSIM dataset, navtest split, and the PDMS/EPDMS metrics on which all headline scores are measured.","marker":"[25]"},{"why":"Provides the perception backbone and sensor-fusion design that ARTEMIS adopts, and serves as the main baseline for ablation and qualitative comparison.","marker":"[13]"},{"why":"Supplies the shared-and-private expert design with sparse gating that ARTEMIS adapts for trajectory planning.","marker":"[19]"},{"why":"Motivates the decision to exclude historical ego states to avoid causal confusion, the premise behind the paper's input design.","marker":"[26]"},{"why":"Defines the extended benchmark metrics and provides the strongest prior baseline that ARTEMIS claims to outperform on EPDMS.","marker":"[16]"},{"why":"Represents the diffusion-based one-shot planning paradigm that ARTEMIS compares against as the main competing approach.","marker":"[17]"},{"why":"Provides another leading end-to-end multimodal planning baseline whose official scores are used for direct comparison.","marker":"[15]"}],"fun_headline_variants":["Autoregressive MoE: 87 PDMS on NAVSIM","Sequential waypoint prediction with expert routing hits 87 PDMS","ARTEMIS: MoE router boosts planner to 87 PDMS on NAVSIM","Temporal waypoints + learned experts: 87 PDMS on NAVSIM","Step-wise planning with MoE achieves 87 PDMS SOTA on NAVSIM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on the assumption that the current steering or throttle command fed into the model as part of the ego state does not secretly encode the future path the model is supposed to predict; if it does, the high scores come from reading the command rather than understanding the scene.","fun_headline_variants_meta":{"raw":{"variants":["Autoregressive MoE: 87 PDMS on NAVSIM","Sequential waypoint prediction with expert routing hits 87 PDMS","ARTEMIS: MoE router boosts planner to 87 PDMS on NAVSIM","Temporal waypoints + learned experts: 87 PDMS on NAVSIM","Step-wise planning with MoE achieves 87 PDMS SOTA on NAVSIM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001414,"raw_usage":{"total_tokens":5713,"prompt_tokens":948,"completion_tokens":4765,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":4665}},"tokens_in":564,"tokens_out":4765,"duration_ms":32427,"temperature":1.0,"reasoning_tokens":4665,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:49:18.816631+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Mask out or randomize the control-command components of the ego state at test time and rerun the NAVSIM evaluation; if PDMS falls well below the reported 87.0, the model was exploiting leaked command information, and a second check would be training a probe that predicts the future expert trajectory from the ego-state vector alone and measuring its accuracy.","supporting_citations":[{"cited_title":"Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking,","cited_arxiv_id":null,"evidence_quote":"Supplies the NAVSIM dataset, navtest split, and the PDMS/EPDMS metrics on which all headline scores are measured."},{"cited_title":"Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,","cited_arxiv_id":null,"evidence_quote":"Provides the perception backbone and sensor-fusion design that ARTEMIS adopts, and serves as the main baseline for ablation and qualitative comparison."},{"cited_title":"Is ego status all you need for open-loop end-to-end autonomous driving?","cited_arxiv_id":null,"evidence_quote":"Motivates the decision to exclude historical ego states to avoid causal confusion, the premise behind the paper's input design."}],"review_version":1}