Pith. sign in

REVIEW 5 major objections 5 minor 26 references

HMAD: Advancing E2E Driving with Anchored Offset Proposals and Simulation-Supervised Multi-target Scoring

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read HMAD achieves a 44.5% driving score on the CVPR 2025 private test set by separating diverse trajectory generation from a learned, simulation-supervised multi-criteria scorer.

desk verdict A competent, honest competition entry that integrates published components into a strong NAVSIM score, but the learned scorer's use as an argmax ranker is the least supported link in the chain. read the letter →

arxiv 2505.23129 v1 pith:2VBOQB6E submitted 2025-05-29 cs.CV

classification cs.CV
keywords end-to-endautonomousdrivingbird's-eye-viewrepresentationtrajectoryproposalanchor-offsetdecodingmulti-criteriascoringsimulation-supervisedlearningextendedPDMscorehard-casemining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces HMAD, an end-to-end driving framework for the CVPR 2025 End-to-End Driving Challenge built around a two-part design: generate many diverse candidate trajectories, then let a learned scorer pick the best one. The generator uses bird's-eye-view features from BEVFormer and learnable anchored queries, initialized from a clustered trajectory dictionary and refined by iterative offset decoding, to avoid the mode collapse of single-trajectory imitation. The scorer is trained with simulator ground-truth labels to predict several interpretable metrics — no at-fault collisions, drivable-area compliance, comfort, and the extended PDM score — and the top-ranked trajectory is selected at inference. On the warmup benchmark the full model reaches an extended PDM score of 65.94, and on the CVPR 2025 private test set it reports a 44.5% driving score; ablations show that removing the scorer drops EPDMS by 15.63 points and removing hard-case mining drops it by 6.93 points.

What carries the argument

The load-bearing components are (1) the BEV-aware anchor-offset decoder, which turns a set of clustered trajectory anchors into learnable query embeddings and refines each query through iterative attention over bird's-eye-view features with offset clipping to keep proposals stable, and (2) the simulation-supervised multi-criteria scorer, a cross-attention module that predicts the extended PDM score along with individual collision, drivable-area, and comfort metrics from simulator ground truth; top-1 selection is made on the predicted overall score at inference.

What would settle it

If on a held-out set of adversarial scenarios (occluded junctions, unprotected turns, sharp curves) the trajectory chosen by the learned scorer produces more simulator-measured at-fault collisions or lane departures than the second- or third-ranked candidate, the scorer's ranking is not generalizing and the top-1 selection mechanism fails.

Watch

Extended reading notes

Core claim

The central claim is that decoupling trajectory generation from trajectory selection yields better end-to-end driving performance than jointly regressing a single trajectory or applying fixed heuristic scoring. The trajectory branch produces a diverse candidate set by refining anchored queries — derived from a trajectory dictionary and updated with clipped offsets conditioned on bird's-eye-view features — and the scoring branch, trained against simulator-generated ground truth, predicts multiple interpretable scores (extended PDM, collision, drivable-area compliance, comfort) that determine the final choice. The authors report that this design attains an EPDMS of 65.94 on the warmup benchmark and a 44.5% driving score on the CVPR 2025 private test set, and they attribute the gains specifically to the learned scorer and to hard-case mining of rare scenarios.

Load-bearing premise

The learned scorer's rankings, trained on simulator-provided scores, stay correct when the model encounters situations the simulator never generated or scored differently.

Editorial extensions

If this is right

  • Trajectory generation and scoring can be trained with different supervision signals — imitation-style for generation and simulator evaluation for scoring — without collapsing the end-to-end pipeline.
  • Because the scorer outputs interpretable per-criterion scores, the system can provide explanations for why one trajectory was chosen over others.
  • The same anchored-offset proposal mechanism could be reused in other multimodal motion-prediction tasks that need stable, diverse candidate sets.
  • Hard-case mining of rare scenarios (unprotected turns, occluded junctions) transfers into broad safety gains, suggesting data distribution is a first-order factor in closed-loop planning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The large ablation gap from removing the scorer (15.63 EPDMS points) suggests that candidate selection, not candidate generation, is the dominant bottleneck on this benchmark; the authors do not push this comparison further.
  • Because the scorer is trained exclusively on simulator labels, its ranking inherits simulator fidelity; a real-world deployment would need recalibration, which the paper does not address.
  • The post-processing filter uses an external 2D detector in image space, so the claimed end-to-end property is partial; a natural extension is to fold this geometric grounding into the learned scorer.
  • The anchor dictionary is fixed and K-means-derived; updating anchors during training or conditioning them on scene context could extend coverage to maneuvers the dictionary does not contain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes HMAD, an end-to-end autonomous driving planner for the CVPR 2025 End-to-End Driving Challenge. HMAD combines a BEVFormer perception backbone, an anchored trajectory decoder that refines offets from a trajectory dictionary, and a simulation-supervised multi-criteria scorer that predicts safety, compliance, comfort, and overall driving quality (EPDMS). At inference, the scorer's predicted overall quality score is used to select the top-ranked trajectory, after a YOLOPv2-based 2D post-processing filter. The paper reports a 44.5% driving score on the CVPR 2025 private test set and a 65.94 EPDMS on the warmup benchmark, with ablations showing that removing the scorer or hard-case mining reduces EPDMS. The central claim is that decoupling trajectory generation from learned multi-criteria scoring improves end-to-end driving.

Significance. If the reported results hold, HMAD provides a meaningful data point for the value of learned, simulation-supervised trajectory scoring over heuristic selection in end-to-end driving, and the 44.5% private-test driving score is a strong competition result. The paper also demonstrates a practical combination of BEVFormer, anchor-offset decoding, and multi-task scoring that is directly relevant to the autonomous driving community. However, the significance is conditional on the scorer's ranking behavior, the statistical reliability of the ablations, and the reproducibility of the training pipeline, none of which are currently established: the results are single-run, on a private benchmark, without code, variance estimates, or ranking diagnostics.

major comments (5)
  1. [§2.3 and Table 1] The scorer is trained as a per-trajectory regressor predicting metrics such as EPDMS, but it is deployed as an argmax ranker over candidate trajectories. The paper does not specify the training loss, any ranking objective, or any calibration analysis, rank correlation, or oracle-versus-selected comparison. Table 1 ablates the scorer entirely, so the 65.94 vs. 50.31 comparison cannot distinguish whether learned scoring ranks candidates well from whether some learned per-trajectory signal helps. Because the top-1 selection mechanism is the central claimed contribution, this is a load-bearing gap; the paper should add ranking diagnostics or pair/triplet training for the scorer.
  2. [§3.3 and Table 1] All ablations are single runs on the warmup benchmark, with no standard deviations, no multiple seeds, and no held-out selection procedure. The decoder-depth comparison in Table 1 is used to choose 2 layers for the full model, meaning the depth is selected on the same benchmark used for reporting, which risks overfitting to the benchmark. The paper should report variance across seeds or at least acknowledge that the depth choice is not independently validated.
  3. [§2.2 vs. §3.2] There is a direct inconsistency in the anchor definition: §2.2 describes the anchor trajectory dictionary as constructed by unsupervised clustering (e.g., K-means), while §3.2 states that the model follows DiffusionDrive and initializes 20 trajectory queries from its predefined anchor set. This affects reproducibility and the claim of a learnable anchored query mechanism; the paper must clarify which source the anchors come from and whether they are fixed or learned during training.
  4. [§2.3 and §3.2] The simulation-supervised scorer is the paper's key innovation, yet its training procedure is severely underspecified: the text does not state the loss function, the relative weighting of the multiple metric predictions, the simulator version, the label normalization, or the network architecture beyond 'cross-attention with BEV features.' Without these details, the method cannot be reproduced or compared fairly against Hydra-MDP-style baselines.
  5. [§2.4 and §3.3] The final system includes a YOLOPv2-based 2D post-processing filter that removes trajectories based on lane-line and obstacle detections, followed by a fallback mechanism. This external, non-differentiable filter is a substantial component of the reported full-model performance, but it is not ablated separately and its quantitative contribution is unknown. This weakens the 'end-to-end' characterization and the attribution of gains to the learned scorer.
minor comments (5)
  1. [Equation (5)] The text says EPDMS includes four multiplicative penalty terms, but the list contains five items: NC, DAC, DDC, TLC, and false-positive filtering; please clarify whether false-positive filtering is a separate term or part of one of the four.
  2. [§2.2] The K-means clustering for the anchor dictionary is not specified: what representation of trajectories is clustered, how many clusters are used, and what distance metric is applied?
  3. [§3.2] The text says 'nav-train split' without defining the split size or relationship to the NA VSIM benchmark; please provide dataset statistics.
  4. [Figure 1] The architecture figure is difficult to read at the submitted resolution, especially the labels inside the decoder and scorer blocks; a higher-resolution figure with a clear legend would improve reproducibility.
  5. [Throughout] The benchmark name is written inconsistently as 'NA VSIM' and 'NAVSIM'; please unify the spelling.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the reported result is an external benchmark score, and no claimed derivation reduces to its inputs.

full rationale

The paper's derivation chain is empirical rather than self-referential. Trajectory candidates are produced by BEVFormer features, learnable anchored queries initialized from a K-means trajectory dictionary, and iterative offset decoding (Eqs. 1-4). The scorer (Section 2.3) is a learned regressor trained with simulator-generated ground-truth labels for EPDMS, collision, drivable-area, and comfort metrics; at inference it ranks candidates by its predicted overall quality. The headline EPDMS of 65.94 and the CVPR 2025 private-test driving score are computed by the external NA VSIM benchmark simulator from the selected trajectory, not by the scorer, so the evaluation is not the fitted input renamed as a prediction. The same simulator family (NA VSIM) provides both training labels and evaluation, but this is standard benchmark practice and does not make the central claim equivalent to its inputs. The scorer ablation (-15.63 EPDMS) is an external contrast, and the backbone/anchor/scoring components are drawn from independent prior work (BEVFormer, DiffusionDrive, Hydra-MDP), not from load-bearing self-citations. The absence of a ranking-calibration study is a robustness concern, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several domain assumptions and a few hand-chosen hyperparameters. No entirely new entities are introduced.

free parameters (3)
  • Offset clipping bound δmax
    Predicted offsets are clipped to a predefined range to stabilize training; the bound is chosen by hand and affects the maximum per-layer trajectory change.
  • Number of trajectory anchors N = 20
    Initialized from DiffusionDrive's predefined anchor set; the number of candidate trajectories directly affects proposal diversity.
  • Decoder layers L = 2
    Selected by ablation on the warmup benchmark; the paper reports that 4 layers hurt EPDMS, so this is a tuned hyperparameter.
assumptions (4)
  • domain assumption BEVFormer provides a sufficiently accurate BEV representation for downstream planning.
    The method relies on BEVFormer to encode multi-camera inputs into a queryable BEV feature map; no analysis of BEV error propagation is provided.
  • domain assumption The NAVSIM simulator's EPDMS is a valid measure of driving quality for supervision and evaluation.
    The scorer is trained on simulator-produced scores and performance is measured by the same metric family; the validity of EPDMS is taken as given.
  • domain assumption The DiffusionDrive anchor set covers the diverse maneuvers needed for the NAVSIM scenarios.
    The trajectory dictionary is inherited from DiffusionDrive without verification that it covers NAVSIM edge cases.
  • domain assumption YOLOPv2's lane and obstacle detection is reliable enough for safety filtering.
    The post-processing filters trajectories based on YOLOPv2 outputs; its false-positive and false-negative rates are not characterized.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HMAD: Advancing E2E Driving with Anchored Offset Proposals and Simulation-Supervised Multi-target Scoring." pith.science (2026). https://pith.science/paper/2VBOQB6E

@misc{pith2026250523129,
  author       = {Pith},
  title        = {Pith review of: HMAD: Advancing E2E Driving with Anchored Offset Proposals and Simulation-Supervised Multi-target Scoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2VBOQB6E}},
  note         = {Machine review of arXiv:2505.23129}
}
read the original abstract

End-to-end autonomous driving faces persistent challenges in both generating diverse, rule-compliant trajectories and robustly selecting the optimal path from these options via learned, multi-faceted evaluation. To address these challenges, we introduce HMAD, a framework integrating a distinctive Bird's-Eye-View (BEV) based trajectory proposal mechanism with learned multi-criteria scoring. HMAD leverages BEVFormer and employs learnable anchored queries, initialized from a trajectory dictionary and refined via iterative offset decoding (inspired by DiffusionDrive), to produce numerous diverse and stable candidate trajectories. A key innovation, our simulation-supervised scorer module, then evaluates these proposals against critical metrics including no at-fault collisions, drivable area compliance, comfortableness, and overall driving quality (i.e., extended PDM score). Demonstrating its efficacy, HMAD achieves a 44.5% driving score on the CVPR 2025 private test set. This work highlights the benefits of effectively decoupling robust trajectory generation from comprehensive, safety-aware learned scoring for advanced autonomous driving.

Figures

Figures reproduced from arXiv: 2505.23129 by the authors.

Figure 1
Figure 1. The Overall Architecture of HMAD. at-fault collisions, drivable area compliance, and driving comfort) from simulator ground-truth, enabling learned, context-dependent trade-offs for nuanced selection within an end-to-end framework. Furthermore, recognizing that many planning failures are concentrated in long-tail scenarios—such as unpro￾tected turns, occluded junctions, sharp curves, and lane departures—we integrate… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 18 canonical work pages

  1. [1]

    nuplan: A closed-loop ml-based plan- ning benchmark for autonomous vehicles

    Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based plan- ning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021. 4

  2. [2]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13243,

  3. [3]

    Lopez, and Adrien Gaidon

    Felipe Codevilla, Eder Santana, Antonio M. Lopez, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019. 1

  4. [4]

    Openscene: The largest up-to- date 3d occupancy prediction benchmark in autonomous driving

    OpenScene Contributors. Openscene: The largest up-to- date 3d occupancy prediction benchmark in autonomous driving. https://github.com/OpenDriveLab/ OpenScene, 2023. 3, 4

  5. [5]

    Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking

    Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, et al. Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems, 37:28706–28719, 2024. 4

  6. [6]

    Hint-ad: Holistically aligned interpretability in end-to-end autonomous driving

    Kairui Ding, Boyuan Chen, Yuchen Su, Huan-ang Gao, Bu Jin, Chonghao Sima, Wuqiang Zhang, Xiaohui Li, Paul Barsch, Hongyang Li, and Hao Zhao. Hint-ad: Holistically aligned interpretability in end-to-end autonomous driving. Conference on Robot Learning (CoRL), 2024. 1

  7. [7]

    Yuchao Feng, Wei Hua, and Yuxiang Sun. Nle-dm: Natural- language explanations for decision making of autonomous driving based on semantic scene understanding.IEEE Trans- actions on Intelligent Transportation Systems, 2023. 1

  8. [8]

    From semi-supervised to omni-supervised room layout esti- mation using point clouds

    Huan-ang Gao, Beiwen Tian, Pengfei Li, Xiaoxue Chen, Hao Zhao, Guyue Zhou, Yurong Chen, and Hongbin Zha. From semi-supervised to omni-supervised room layout esti- mation using point clouds. In 2023 IEEE International Con- ference on Robotics and Automation (ICRA) , pages 2803–

Show all 26 references
  1. [9]

    Dqs3d: Densely-matched quantization- aware semi-supervised 3d detection

    Huan-ang Gao, Beiwen Tian, Pengfei Li, Hao Zhao, and Guyue Zhou. Dqs3d: Densely-matched quantization- aware semi-supervised 3d detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 21905–21915, 2023. 1

  2. [10]

    Yolopv2: Better, faster, stronger for panoptic driving perception

    Cheng Han, Qichao Zhao, Shuyi Zhang, Yinzi Chen, Zhen- lin Zhang, and Jinwei Yuan. Yolopv2: Better, faster, stronger for panoptic driving perception. arXiv preprint arXiv:2208.11434, 2022. 3

  3. [11]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2

  4. [12]

    Planning-oriented autonomous driv- ing

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning-oriented autonomous driv- ing. In Proceedings of the IEEE/CVF Conference on Com...

  5. [13]

    Planning-oriented autonomous driv- ing

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning-oriented autonomous driv- ing. In Proceedings of the IEEE/CVF Conference on Com...

  6. [14]

    Vad: Vectorized scene representation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. ICCV, 2023. 1

  7. [15]

    P-mapnet: Far-seeing map generator enhanced by both sdmap and hdmap priors

    Zhou Jiang, Zhenxin Zhu, Pengfei Li, Huan-ang Gao, Tianyuan Yuan, Yongliang Shi, Hang Zhao, and Hao Zhao. P-mapnet: Far-seeing map generator enhanced by both sdmap and hdmap priors. arXiv preprint arXiv:2403.10521,

  8. [16]

    Learning to drive in a day

    Alex Kendall, Jeffrey Hawke, David Janz, Przemyslaw Mazur, Daniele Reda, John-Mark Allen, Vinh-Dieu Lam, Alex Bewley, and Amar Shah. Learning to drive in a day. In 2019 international conference on robotics and automation (ICRA), 2019

  9. [17]

    Training-free model merging for multi- target domain adaptation

    Wenyi Li, Huan-ang Gao, Mingju Gao, Beiwen Tian, Rong Zhi, and Hao Zhao. Training-free model merging for multi- target domain adaptation. arXiv preprint arXiv:2407.13771,

  10. [18]

    Hydra-mdp: End-to-end multimodal planning with multi- target hydra-distillation

    Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, et al. Hydra-mdp: End-to-end multimodal planning with multi- target hydra-distillation. arXiv preprint arXiv:2406.06978 ,

  11. [19]

    Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Qiao Yu, and Jifeng Dai. Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2024. 1, 2

  12. [20]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving

    Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. arXiv preprint arXiv:2411.15139, 2024. 1, 2, 4

  13. [21]

    Multi- modal fusion transformer for end-to-end autonomous driv- ing

    Aditya Prakash, Kashyap Chitta, and Andreas Geiger. Multi- modal fusion transformer for end-to-end autonomous driv- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 7077– 7087, 2021. 1

  14. [22]

    Centaur: Robust end-to-end autonomous driving with test-time training

    Chonghao Sima, Kashyap Chitta, Zhiding Yu, Shiyi Lan, Ping Luo, Andreas Geiger, Hongyang Li, and Jose M Al- varez. Centaur: Robust end-to-end autonomous driving with test-time training. arXiv preprint arXiv:2503.11650, 2025. 1

  15. [23]

    Unsupervised road anomaly de- tection with language anchors

    Beiwen Tian, Mingdao Liu, Huan-ang Gao, Pengfei Li, Hao Zhao, and Guyue Zhou. Unsupervised road anomaly de- tection with language anchors. In 2023 IEEE international conference on robotics and automation (ICRA), pages 7778–

  16. [24]

    Autonomous ve- hicle motion planning via recurrent spline optimization

    Wenda Xu, Qian Wang, and John M Dolan. Autonomous ve- hicle motion planning via recurrent spline optimization. In 2021 IEEE International Conference on Robotics and Au- tomation (ICRA), 2021. 1

  17. [25]

    End-to-end urban driving by imitat- ing a reinforcement learning coach

    Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, and Luc Van Gool. End-to-end urban driving by imitat- ing a reinforcement learning coach. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15222–15232, 2021. 1

  18. [26]

    Steps: Joint self-supervised nighttime image enhancement and depth estimation

    Yupeng Zheng, Chengliang Zhong, Pengfei Li, Huan-ang Gao, Yuhang Zheng, Bu Jin, Ling Wang, Hao Zhao, Guyue Zhou, Qichao Zhang, et al. Steps: Joint self-supervised nighttime image enhancement and depth estimation. In 2023 IEEE International Conference on Robotics and Automation...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.