MIND-Stack: Modular, Interpretable, End-to-End Differentiability for Autonomous Navigation

Felix Jahncke; Johannes Betz

REVIEW 3 major objections 5 minor 44 references

MIND-Stack: Modular, Interpretable, End-to-End Differentiability for Autonomous Navigation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that training a localization network on the downstream control loss of a differentiable Stanley controller reduces path-following error by up to 77.43% in simulation and 50.77% on a real vehicle.

desk verdict Solid engineering paper with an addressable flaw: the headline end-to-end training benefit is not isolated from extra training time, but the architecture and real-world demo are worth refereeing. read the letter →

arxiv 2505.21734 v1 pith:Z5ABMBO5 submitted 2025-05-27 cs.RO cs.LG

Felix Jahncke , Johannes Betz This is my paper

classification cs.ROcs.LG

keywords autonomousnavigationend-to-enddifferentiabilitymodularsoftwarestacklocalizationStanleycontrollercross-trackerrorsim-to-realtransferembeddedsystems

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MIND-Stack is a modular autonomous-navigation stack that keeps a conventional rule-based controller—a Stanley steering law—but makes the whole pipeline from LiDAR to steering angle differentiable. The paper tries to establish that the upstream localization network can be trained on the downstream control loss (a squared cross-track error), so that the network's job is not merely to estimate the vehicle's pose but to produce pose estimates that reduce path-following error. In simulation across six track segments, this reduces the mean absolute cross-track error by 9.99% to 77.43%, with an average reduction of 45.61%, and the improvement transfers to a real vehicle with up to a 50.77% reduction. The authors also show that training the localization network and the controller's hand-tuned gains simultaneously outperforms training either module alone. If the result holds, it would mean that modular, interpretable autonomy stacks do not have to give up end-to-end learning.

What carries the argument

The central mechanism is the differentiable Stanley controller, a steering law $\delta = k_h e_{\mathrm{head}} + \arctan(k_e e_{\mathrm{cross}}/v)$ that maps a pose estimate to a steering angle and is differentiable with respect to that pose. To make the control loss flow into the network, the stack propagates the network's pose estimate two timesteps (0.01 s each) through a kinematic bicycle model, so the cross-track error $e_{\mathrm{cross}}$ depends on the steering command and therefore on the pose. A parallel unaltered localization module, not the trained network, provides the pose used to compute that cross-track error, which preserves interpretability of the network's output. The training objective is $L_{\mathrm{total}} = \alpha e_{\mathrm{cross}}^2 + \beta|\theta_t - 2\theta_{t-1} + \theta_{t-2}| + \gamma L_{\mathrm{loc}}$, mixing the downstream error, a heading-smoothness term, and the original localization loss.

What would settle it

Measure the unaltered parallel localization module's own mean absolute cross-track error in each scenario; if it is comparable to or larger than the reported 9.99-77.43% reductions, the control loss may be fitting to that module's bias rather than to true path-tracking. A direct way to test this is to retrain in simulation using ground-truth pose to compute the cross-track error and see whether the gains persist.

Watch

Extended reading notes

Core claim

The paper's central claim is that end-to-end differentiability through a modular stack lets an upstream localization module be optimized for the downstream control objective, not just for its own supervised loss. Concretely, the localization CNN is first trained on a pose-reconstruction loss, then fine-tuned on a total loss that combines a squared cross-track error (computed by propagating the trained network's pose two timesteps through a kinematic bicycle model), a smoothness penalty on heading, and the original localization loss. Gradient flow is made possible because the Stanley controller's steering formula is differentiable with respect to its pose input. In six simulated scenarios the validation loss, mean absolute cross-track error per timestep relative to ground truth, falls by between 9.99% and 77.43%; in two real-world scenarios it falls by up to 50.77%. The paper interprets this as evidence that a localization module can be trained to serve the controller, and that combined training of the controller's gains and the network yields a further 12% validation-loss reduction over the second-best setup.

Load-bearing premise

The unaltered parallel localization module's pose is assumed accurate enough to serve as the reference for the cross-track error in the training loss, but the paper never quantifies that module's own error or systematic bias.

Editorial extensions

If this is right

A localization network trained on the control loss can cut mean absolute cross-track error by up to 77.43% in simulation and 50.77% on a real vehicle, without changing the controller.
The same differentiable-stack design can be extended to perception, prediction, and planning modules so each is optimized for the final driving objective rather than only its local loss.
Simultaneous training of a neural module and a hand-crafted controller's parameters is feasible and yields lower path error than training either module alone.
The pipeline runs in about 0.019 s per inference on an embedded GPU board, faster than the 40 Hz LiDAR input, so it is deployable on low-power hardware.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A likely mechanism behind the reported reductions is that control-loss training adjusts the pose estimate to compensate for the fixed controller's steady-state errors, so the network's output becomes less a pure sensor estimate and more a controller-aware correction; one could test this by comparing the trained network's pose output with the untrained module's output and with ground truth.
The general recipe suggested here is that any differentiable downstream cost can fine-tune upstream modules, so substituting a planning or MPC cost for the cross-track error could shape different driving behaviors, such as aggressiveness or comfort.
Because the cross-track error in training is computed relative to the parallel unaltered localization module, the reported improvements could partly be the network learning to reproduce that module's bias rather than learning better geometry; a direct measurement of that module's own error would bound the effect.
The scenario spread (9.99% minimum, 77.43% maximum) suggests the benefit is environment-dependent, so future work could characterize when control-loss training helps most—for example, on tracks where the baseline localization is weak relative to the controller.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read MIND-Stack. The core idea is sensible: string a CNN localizer, a differentiable Stanley controller, and a kinematic bicycle model into one graph, then train the localizer on a downstream control loss. The individual pieces are known, but the specific combination—sensor-to-actuator differentiability with a hand-crafted control law and real F1Tenth deployment—is not something I've seen in the prior work they cite. The paper deserves credit for shipping open-source code and for showing consistent validation-loss improvements across six simulation scenarios plus a real-world sim-to-real transfer. The simultaneous training of the localizer and Stanley gains (Table V) is a nice demonstration of the architecture's flexibility.

The main weakness is the missing training-duration control. The "before" state is a network trained for 50 epochs on Lloc only; the "after" state is that same network plus 30 laps of additional gradient updates on the control loss. There is no baseline that gets the same 30 additional laps of updates on Lloc alone. Any extra training, even on the original objective, could tighten the pose estimates and lower the path-tracking error. So the headline claim—that end-to-end differentiability specifically trains the localizer to reduce downstream control error—is not yet isolated. The same issue affects Table V: the four setups are not shown to have matched update counts. Adding that control is straightforward and would make the central result much cleaner.

A second, smaller soft spot: the training loss's cross-track error is computed using a parallel "unaltered" localization module, and the paper never quantifies that module's own error or bias. If that module has a systematic offset, the loss could be training against the wrong error signal. This is minor, but worth addressing.

Also, the "first fully differentiable stack" phrasing is a bit strong; the paper does enough to distinguish itself from DiffStack, DAN, and PyPose, but that claim should be softened to "to our knowledge" and perhaps with more precise bounds on prior art. The SOTA comparison runs on only one scenario, so the "outperforms SOTA" claim rests on a thin base, though the baseline already being strong is a point in their favor.

Overall, this is a solid engineering paper with an addressable flaw. It would benefit from a matched-budget ablation and a quantification of the parallel localizer's accuracy. I'd send it to referees; it's the kind of work where a careful revision could make the central claim stick.

Referee Report

3 major / 5 minor

Summary. The paper introduces MIND-Stack, a modular and fully differentiable autonomous-navigation stack that couples a CNN-based LiDAR localization module with a hand-crafted Stanley lateral controller. The localization network is first trained supervised on a localization loss, then fine-tuned on a combined control-oriented loss that includes cross-track error, an orientation-jerk penalty, and (in one scenario) the localization loss. Experiments in six simulation scenarios report 9.99%–77.43% reductions in mean absolute cross-track error after optimization, a comparison against several state-of-the-art controllers in one scenario, transfer of a simulation-trained model to a real F1Tenth vehicle with reduced errors, and a combined training setting in which both the localization network and the Stanley parameters are optimized jointly. The authors argue that this demonstrates a novel capability: an upstream localization module can reduce downstream control error through end-to-end differentiability while retaining modularity and interpretability.

Significance. If the central claim is sustained, MIND-Stack is a useful and timely contribution to hybrid modular autonomous driving: it spans from raw LiDAR input to steering output, is fully differentiable, keeps human-interpretable intermediate states, and is validated on an embedded real vehicle. The open-source code, the consistent per-timestep evaluation metrics, and the real-world deployment are explicit strengths that support reproducibility and practical relevance. However, the key causal claim—that end-to-end differentiability specifically enables the localization module to reduce downstream control error—is currently supported mainly by a before/after comparison that does not control for the amount of additional training the localization network receives. The absence of a matched training-duration baseline, together with the unquantified bias of the reference localization module used in the control-loss computation, leaves the central mechanism less certain than the abstract suggests. These are fixable with additional experiments, so the result is promising but not yet conclusive.

major comments (3)

[§V-A, Table II; §V-D, Table V] The main before/after comparison lacks a matched training-budget control. 'Before Optimization' is a localization network trained for 50 epochs on Lloc, while 'After Optimization' is the same network after an additional 30 laps of training on the control loss. There is no baseline that trains the localization network for the same 30 additional laps on Lloc alone. Any additional gradient updates, even if supervised by the original localization loss, could improve pose estimates and hence reduce the ground-truth cross-track error reported as validation loss. The same issue appears in the 'Localization Only' row of Table V and in the real-world transfer comparison of Table IV, where the after-optimization model has simply received more training than the before-optimization model. Please add a control that trains the localization network for the same number of laps/updates on Lloc only and report its validation loss, so the specific contribution of the downstream control loss is isolated.
[§III-C] The cross-track error term in Eq. (3) is computed from a pose propagated from an 'unaltered localization module' running in parallel with the trained network. The paper states that this module provides 'an accurate pose estimate,' but it never quantifies the module's own error or systematic bias relative to ground truth in the simulation or real-world scenarios. If the reference pose has a systematic offset, the control loss is optimized against a biased error signal, and the observed reductions in validation loss could be a side effect of the network compensating for that bias rather than evidence that it improves true path-following. Please report the unaltered module's localization error/bias in each scenario and, ideally, ablate the cross-track error computation by using the ground-truth pose as the reference.
[§V-D, Table I] In most scenarios γ is absent (Table I), so the end-to-end loss does not constrain the localization output to the ground-truth pose. The paper claims interpretability through intermediate pose representations, but it never measures how far the optimized localization output deviates from the unaltered module or from ground truth. If the network learns a pose that is far from the true pose while producing low control error, the 'human interpretable state representation' is no longer an accurate vehicle state, weakening a stated contribution. Please report the localization error of the optimized network and explicitly discuss the trade-off between control loss and pose interpretability.

minor comments (5)

[Eq. (1)] The orientation error term in Eq. (1) uses the raw difference (θ̂ − θ) without wrapping to the interval [−π, π]; this can produce discontinuous gradients near ±π and may distort the localization loss in scenarios where the heading crosses that boundary.
[Table I] The '–' entries for γ in Table I are not explained in the text; please state explicitly that those scenarios set γ = 0 (or otherwise omit the localization term from Ltotal).
[§V-C] The real-world experiments operate at a 40 Hz control rate while the simulation uses 100 Hz, but the paper does not state whether the simulation-trained model and the kinematic bicycle model in the loss were trained at 100 Hz and how the transfer to 40 Hz affects the two-step propagation time step; please clarify.
[Table III] The state-of-the-art comparison is reported for a single scenario without a statistical significance test; adding a paired test across the 30 evaluation laps would strengthen the claim that MIND-Stack outperforms the listed controllers.
[§I, Related Work] There are minor language issues, including 'in-, output' in the Related Work section and 'at competitive speeds and lap times' in Section V-B; a careful proofread would improve clarity.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the central claim rests on measured before/after validation losses, not on an equation that reduces to its own inputs; the paper's self-citations are minor and non-load-bearing.

full rationale

No load-bearing step reduces to its own inputs. MIND-Stack's central claim is empirical: a localization CNN trained for 30 laps on L_total = α e_cross^2 + β |θ_t − 2θ_{t−1} + θ_{t−2}| + γ L_loc (Eq. 3) reduces mean absolute cross-track error measured against ground truth (Tables II and IV). The training objective is not identical to the validation metric: the e_cross term in Eq. 3 is computed from an unaltered parallel localization module's pose propagated through a kinematic bicycle model, while the validation CTE is computed from ground-truth pose. Reducing one therefore does not by construction reduce the other; the reported improvements are measured outcomes of optimization, not fitted parameters renamed as predictions. The paper's self-citations — [38] Zang et al. for noise-injected previous-pose training and [41] Evans et al. for F1TENTH benchmarks — are not load-bearing: the first justifies a data-augmentation detail and the second sources external baselines, neither supplying the central differentiability result. The main caveats are experimental rather than circular: the 'After Optimization' model receives extra gradient updates (30 additional training laps) compared to 'Before Optimization', so training duration is not matched in the central ablation, and the real-world ground-truth source for CTE is not explicitly specified. These caveats affect whether the specific mechanism is cleanly isolated, but they do not make any derived quantity equivalent to an input by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the fidelity of the synthetic LiDAR data, the unaltered module's pose as a ground-truth proxy, and the adequacy of the kinematic bicycle model for the gradient path. These are domain assumptions that are neither externally benchmarked nor ablated in the paper.

free parameters (5)

Loss weights alpha, beta, gamma = Scenario-dependent: alpha 1.0 to 5.5, beta 0 to 1.0, gamma 0 to 0.005 (Table I)
Selected via manual grid search per scenario; changes the training objective and can inflate improvements.
Stanley gains ke, kh = ke=1.8, kh=1.3 (Section III-B)
Hand-tuned; in Section V-D they are trained instead, with learning rate 1e-3.
LiDAR noise standard deviation = 0.25
Derived by deploying models with different values on the real vehicle (Section III-A).
Kinematic bicycle model horizon = 2 timesteps (0.02 s)
Chosen for gradient flow; not systematically varied.
Learning rates per scenario = 3e-9 to 9e-8 (Table I)
Manual grid search per scenario.

assumptions (4)

domain assumption The synthetic LiDAR scans generated by raytracing with Gaussian noise are representative enough of real LiDAR for the trained network to transfer to the real world.
Used in Section III-A and IV-B; central to sim-to-real claims.
domain assumption The unaltered localization module's pose estimate is accurate enough to serve as the base for computing the cross-track training signal.
Section III-C uses it as the 'accurate pose estimate' without reporting its error.
domain assumption The kinematic bicycle model accurately predicts vehicle motion over two 0.01 s steps for the purpose of gradient computation.
Section III-C propagates the pose with this model; mismatch with true dynamics could mislead gradients.
standard math Backpropagation through the Stanley controller and the bicycle model yields usable gradients, with no vanishing or exploding gradient issues.
Assumed by the end-to-end training procedure in Section III-C.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MIND-Stack: Modular, Interpretable, End-to-End Differentiability for Autonomous Navigation." pith.science (2026). https://pith.science/paper/Z5ABMBO5

@misc{pith2026250521734,
  author       = {Pith},
  title        = {Pith review of: MIND-Stack: Modular, Interpretable, End-to-End Differentiability for Autonomous Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z5ABMBO5}},
  note         = {Machine review of arXiv:2505.21734}
}

read the original abstract

Developing robust, efficient navigation algorithms is challenging. Rule-based methods offer interpretability and modularity but struggle with learning from large datasets, while end-to-end neural networks excel in learning but lack transparency and modularity. In this paper, we present MIND-Stack, a modular software stack consisting of a localization network and a Stanley Controller with intermediate human interpretable state representations and end-to-end differentiability. Our approach enables the upstream localization module to reduce the downstream control error, extending its role beyond state estimation. Unlike existing research on differentiable algorithms that either lack modules of the autonomous stack to span from sensor input to actuator output or real-world implementation, MIND-Stack offers both capabilities. We conduct experiments that demonstrate the ability of the localization module to reduce the downstream control loss through its end-to-end differentiability while offering better performance than state-of-the-art algorithms. We showcase sim-to-real capabilities by deploying the algorithm on a real-world embedded autonomous platform with limited computation power and demonstrate simultaneous training of both the localization and controller towards one goal. While MIND-Stack shows good results, we discuss the incorporation of additional modules from the autonomous navigation pipeline in the future, promising even greater stability and performance in the next iterations of the framework.

Figures

Figures reproduced from arXiv: 2505.21734 by the authors.

**Figure 2.** By leveraging a modular architecture with end-to-end differentiability, MIND-Stack enables the upstream localization to improve the downstream control loss (left). MIND-Stack optimizes losses (middle) while being lightweight and efficient, as verified on an autonomous platform (right), where the vehicle learns to optimize its driving policy and trajectory. 3) Adaptability and real-world performance: We demonstrate t… view at source ↗

**Figure 3.** Six scenarios used to train and evaluate MIND-Stack, presenting different challenges: From left to right Scenario 1 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Visualization of the new driven trajectory ( [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 43 canonical work pages

[1]

Perception, planning, control, and coordination for autonomous vehicles,

S. D. Pendleton, H. Andersen, X. Du, X. Shen, M. Meghjani, Y . H. Eng, D. Rus, and M. H. Ang, “Perception, planning, control, and coordination for autonomous vehicles,” Machines, vol. 5, no. 1, 2017

2017
[2]

A decision-theoretic approach to planning, perception, and control,

K. Basye, T. Dean, J. Kirman, and M. Lejter, “A decision-theoretic approach to planning, perception, and control,” IEEE Expert , vol. 7, no. 4, pp. 58–65, 1992

work page 1992
[3]

A Survey of Deep Learning Techniques for Autonomous Driving,

S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A Survey of Deep Learning Techniques for Autonomous Driving,” Journal of Field Robotics, vol. 37, no. 3, pp. 362–386, Apr. 2020

work page 2020
[4]

Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey,

P. S. Chib and P. Singh, “Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey,” IEEE Trans- actions on Intelligent V ehicles, vol. 9, no. 1, pp. 103–118, Jan. 2024

work page 2024
[5]

Learning to drive in a day,

A. Kendall, J. Hawke, D. Janz, P. Mazur, D. Reda, J.-M. Allen, V .-D. Lam, A. Bewley, and A. Shah, “Learning to drive in a day,” in 2019 international conference on robotics and automation (ICRA) . IEEE, 2019, pp. 8248–8254

work page 2019
[6]

Diffstack: A differentiable and modular control stack for autonomous vehicles,

P. Karkus, B. Ivanovic, S. Mannor, and M. Pavone, “Diffstack: A differentiable and modular control stack for autonomous vehicles,” in 6th Annual Conference on Robot Learning , 2022

work page 2022
[7]

You Only Look Once: Unified, Real-Time Object Detection,

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV , USA: IEEE, Jun. 2016, pp. 779–788

work page 2016
[8]

Center-based 3d object detec- tion and tracking,

T. Yin, X. Zhou, and P. Krahenbuhl, “Center-based 3d object detec- tion and tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2021, pp. 11 784–11 793

work page 2021

Show all 44 references

[9]

Deepfusion: Lidar- camera deep fusion for multi-modal 3d object detection,

Y . Li, A. W. Yu, T. Meng, B. Caine, J. Ngiam, D. Peng, J. Shen, Y . Lu, D. Zhou, Q. V . Le, A. Yuille, and M. Tan, “Deepfusion: Lidar- camera deep fusion for multi-modal 3d object detection,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022,...

2022
[10]

Motionnet: Joint perception and motion prediction for autonomous driving based on bird’s eye view maps,

P. Wu, S. Chen, and D. N. Metaxas, “Motionnet: Joint perception and motion prediction for autonomous driving based on bird’s eye view maps,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020, pp. 11 382–11 392

2020
[11]

Deep Learning-Based Vehicle Behavior Prediction for Au- tonomous Driving Applications: A Review,

S. Mozaffari, O. Y . Al-Jarrah, M. Dianati, P. Jennings, and A. Mouza- kitis, “Deep Learning-Based Vehicle Behavior Prediction for Au- tonomous Driving Applications: A Review,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 33–47, Jan. 2022

2022
[12]

One thousand and one hours: Self-driving motion prediction dataset,

J. Houston, G. Zuidhof, L. Bergamini, Y . Ye, L. Chen, A. Jain, S. Omari, V . Iglovikov, and P. Ondruska, “One thousand and one hours: Self-driving motion prediction dataset,” in Proceedings of the 2020 Conference on Robot Learning , ser. Proceedings of Machine Learning Resear...

2020
[13]

Deep learning based motion planning for autonomous vehicle using spatiotemporal lstm network,

Z. Bai, B. Cai, W. ShangGuan, and L. Chai, “Deep learning based motion planning for autonomous vehicle using spatiotemporal lstm network,” in 2018 Chinese Automation Congress (CAC) , 2018, pp. 1610–1614

2018
[14]

Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Driving,

M. Klimke, B. V ¨olz, and M. Buchholz, “Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Driving,” arXiv, 2023

2023
[15]

Real-Time Neural MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms,

T. Salzmann, E. Kaufmann, J. Arrizabalaga, M. Pavone, D. Scara- muzza, and M. Ryll, “Real-Time Neural MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms,” IEEE Robotics and Automation Letters , vol. 8, no. 4, pp. 2397–2404, Apr. 2023

2023
[16]

Parallel Learning-Based Steering Control for Autonomous Driving,

F. Tian, Z. Li, F.-Y . Wang, and L. Li, “Parallel Learning-Based Steering Control for Autonomous Driving,” IEEE Transactions on Intelligent V ehicles, vol. 8, no. 1, pp. 379–389, Jan. 2023

2023
[17]

Event-Triggered Model Pre- dictive Control With Deep Reinforcement Learning for Autonomous Driving,

F. Dang, D. Chen, J. Chen, and Z. Li, “Event-Triggered Model Pre- dictive Control With Deep Reinforcement Learning for Autonomous Driving,” IEEE Transactions on Intelligent V ehicles, vol. 9, no. 1, pp. 459–468, Jan. 2024

2024
[18]

Pontryagin differentiable programming: An end-to-end learning and control framework,

W. Jin, Z. Wang, Z. Yang, and S. Mou, “Pontryagin differentiable programming: An end-to-end learning and control framework,” in Advances in Neural Information Processing Systems , vol. 33. Curran Associates, Inc., 2020, pp. 7979–7992

2020
[19]

Carl-Lead: Lidar-based End- to-End Autonomous Driving with Contrastive Deep Reinforcement Learning,

P. Cai, S. Wang, H. Wang, and M. Liu, “Carl-Lead: Lidar-based End- to-End Autonomous Driving with Contrastive Deep Reinforcement Learning,” arXiv, Sep. 2021

2021
[20]

Learning from all vehicles,

D. Chen and P. Kr ¨ahenb¨uhl, “Learning from all vehicles,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2022, pp. 17 222–17 231

2022
[21]

Multimodal end-to-end autonomous driving,

Y . Xiao, F. Codevilla, A. Gurram, O. Urfalioglu, and A. M. L ´opez, “Multimodal end-to-end autonomous driving,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 1, pp. 537–547, 2022

2022
[22]

Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,

K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger, “Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,” IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, vol. 45, no. 11, pp. 12 878–12 895, nov 2023

2023
[23]

End-to-end model-free reinforcement learning for urban driving using implicit affordances,

M. Toromanoff, E. Wirbel, and F. Moutarde, “End-to-end model-free reinforcement learning for urban driving using implicit affordances,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020, pp. 7151–7160

2020
[24]

Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning,

J. Chen, S. E. Li, and M. Tomizuka, “Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 6, pp. 5068–5078, 2022

2022
[25]

FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing,

K. Stachowicz, D. Shah, A. Bhorkar, I. Kostrikov, and S. Levine, “FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing,” in 7th Annual Conference on Robot Learning, Aug. 2023

2023
[26]

High-speed au- tonomous racing using trajectory-aided deep reinforcement learning,

B. D. Evans, H. A. Engelbrecht, and H. W. Jordaan, “High-speed au- tonomous racing using trajectory-aided deep reinforcement learning,” IEEE Robotics and Automation Letters , vol. 8, no. 9, pp. 5353–5359, 2023

2023
[27]

Explanations in Autonomous Driving: A Survey,

D. Omeiza, H. Webb, M. Jirotka, and L. Kunze, “Explanations in Autonomous Driving: A Survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 8, pp. 10 142–10 162, Aug. 2022

2022
[28]

Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions,

S. Atakishiyev, M. Salameh, H. Yao, and R. Goebel, “Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions,” arXiv, Apr. 2024

2024
[29]

Interpretable learning for self-driving cars by visualizing causal attention,

J. Kim and J. Canny, “Interpretable learning for self-driving cars by visualizing causal attention,” in 2017 IEEE International Conference on Computer Vision (ICCV) , 2017, pp. 2961–2969

2017
[30]

Deep model reassembly,

X. Yang, D. Zhou, S. Liu, J. Ye, and X. Wang, “Deep model reassembly,” in Advances in Neural Information Processing Systems , vol. 35, 2022, pp. 25 739–25 753

2022
[31]

Differentiable algorithm networks for composable robot learn- ing,

P. Karkus, X. Ma, D. Hsu, L. Kaelbling, W. S. Lee, and T. Lozano- Perez, “Differentiable algorithm networks for composable robot learn- ing,” in Proceedings of Robotics: Science and Systems , Freiburgim- Breisgau, Germany, June 2019

2019
[32]

Pypose: A library for robot learning with physics-based optimization,

C. Wang, D. Gao, K. Xu, J. Geng, Y . Hu, Y . Qiu, B. Li, F. Yang, B. Moon, A. Pandey, Aryan, J. Xu, T. Wu, H. He, D. Huang, Z. Ren, S. Zhao, T. Fu, P. Reddy, X. Lin, W. Wang, J. Shi, R. Talak, K. Cao, Y . Du, H. Wang, H. Yu, S. Wang, S. Chen, A. Kashyap, R. Bandaru, K. Dantu, ...

2023
[33]

Pypose v0.6: The imperative programming interface for robotics,

Z. Zhan, X. Li, Q. Li, H. He, A. Pandey, H. Xiao, Y . Xu, X. Chen, K. Xu, K. Cao, Z. Zhao, Z. Wang, H. Xu, Z. Fang, Y . Chen, W. Wang, X. Fang, Y . Du, T. Wu, X. Lin, Y . Qiu, F. Yang, J. Shi, S. Su, Y . Lu, T. Fu, K. Dantu, J. Wu, L. Xie, M. Hutter, L. Carlone, S. Scherer, D....

2023
[34]

Autonomous Automobile Trajectory Tracking for Off-Road Driving: Controller Design, Experimental Validation and Racing,

G. M. Hoffmann, C. J. Tomlin, M. Montemerlo, and S. Thrun, “Autonomous Automobile Trajectory Tracking for Off-Road Driving: Controller Design, Experimental Validation and Racing,” in 2007 American Control Conference , Jul. 2007, pp. 2296–2301

2007
[35]

Minimum curvature trajectory planning and control for an autonomous race car,

A. Heilmeier, A. Wischnewski, L. Hermansdorfer, J. Betz, M. Lienkamp, and B. Lohmann, “Minimum curvature trajectory planning and control for an autonomous race car,” V ehicle System Dynamics, vol. 58, no. 10, pp. 1497–1527, Oct. 2020

2020
[36]

SLAM Toolbox: SLAM for the dynamic world,

S. Macenski and I. Jambrecic, “SLAM Toolbox: SLAM for the dynamic world,” Journal of Open Source Software , vol. 6, no. 61, p. 2783, May 2021

2021
[37]

Autonomous vehicles on the edge: A survey on autonomous vehicle racing,

J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V . Krovi, and R. Mangharam, “Autonomous vehicles on the edge: A survey on autonomous vehicle racing,” IEEE Open J. Intell. Transp. Syst. , vol. 3, pp. 458–488, 2022

2022
[38]

Local INN: Implicit map representation and localization with invertible neural networks,

Z. Zang, H. Zheng, J. Betz, and R. Mangharam, “Local INN: Implicit map representation and localization with invertible neural networks,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 11 742–11 748

2023
[39]

CommonRoad: Com- posable benchmarks for motion planning on roads,

M. Althoff, M. Koschi, and S. Manzinger, “CommonRoad: Com- posable benchmarks for motion planning on roads,” in 2017 IEEE Intelligent V ehicles Symposium (IV). Los Angeles, CA, USA: IEEE, Jun. 2017, pp. 719–726

2017
[40]

F1TENTH: An Open-source Evaluation Environment for Continuous Control and Reinforcement Learning,

M. O’Kelly, H. Zheng, D. Karthik, and R. Mangharam, “F1TENTH: An Open-source Evaluation Environment for Continuous Control and Reinforcement Learning,” in Proceedings of the NeurIPS 2019 Competition and Demonstration Track . PMLR, Aug. 2020, pp. 77– 89

2019
[41]

Unifying F1TENTH Autonomous Racing: Survey, Methods and Benchmarks,

B. D. Evans, R. Trumpp, M. Caccamo, F. Jahncke, J. Betz, H. W. Jordaan, and H. A. Engelbrecht, “Unifying F1TENTH Autonomous Racing: Survey, Methods and Benchmarks,” arXiv, Apr. 2024

2024
[42]

A novel obstacle avoidance algorithm: “Follow the Gap Method

V . Sezer and M. Gokasan, “A novel obstacle avoidance algorithm: “Follow the Gap Method”,” Robotics and Autonomous Systems , vol. 60, no. 9, pp. 1123–1134, Sep. 2012

2012
[43]

Implementation of the pure pursuit path tracking algorithm,

R. C. Coulter, “Implementation of the pure pursuit path tracking algorithm,” Pittsburgh, PA, Tech. Rep. CMU-RI-TR-92-01, Jan. 1992

1992
[44]

Cddt: Fast approximate 2d ray casting for accelerated localization,

C. H. Walsh and S. Karaman, “Cddt: Fast approximate 2d ray casting for accelerated localization,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) , 2018, pp. 3677–3684

2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Perception, planning, control, and coordination for autonomous vehicles,

S. D. Pendleton, H. Andersen, X. Du, X. Shen, M. Meghjani, Y . H. Eng, D. Rus, and M. H. Ang, “Perception, planning, control, and coordination for autonomous vehicles,” Machines, vol. 5, no. 1, 2017

2017

[2] [2]

A decision-theoretic approach to planning, perception, and control,

K. Basye, T. Dean, J. Kirman, and M. Lejter, “A decision-theoretic approach to planning, perception, and control,” IEEE Expert , vol. 7, no. 4, pp. 58–65, 1992

work page 1992

[3] [3]

A Survey of Deep Learning Techniques for Autonomous Driving,

S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A Survey of Deep Learning Techniques for Autonomous Driving,” Journal of Field Robotics, vol. 37, no. 3, pp. 362–386, Apr. 2020

work page 2020

[4] [4]

Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey,

P. S. Chib and P. Singh, “Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey,” IEEE Trans- actions on Intelligent V ehicles, vol. 9, no. 1, pp. 103–118, Jan. 2024

work page 2024

[5] [5]

Learning to drive in a day,

A. Kendall, J. Hawke, D. Janz, P. Mazur, D. Reda, J.-M. Allen, V .-D. Lam, A. Bewley, and A. Shah, “Learning to drive in a day,” in 2019 international conference on robotics and automation (ICRA) . IEEE, 2019, pp. 8248–8254

work page 2019

[6] [6]

Diffstack: A differentiable and modular control stack for autonomous vehicles,

P. Karkus, B. Ivanovic, S. Mannor, and M. Pavone, “Diffstack: A differentiable and modular control stack for autonomous vehicles,” in 6th Annual Conference on Robot Learning , 2022

work page 2022

[7] [7]

You Only Look Once: Unified, Real-Time Object Detection,

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV , USA: IEEE, Jun. 2016, pp. 779–788

work page 2016

[8] [8]

Center-based 3d object detec- tion and tracking,

T. Yin, X. Zhou, and P. Krahenbuhl, “Center-based 3d object detec- tion and tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2021, pp. 11 784–11 793

work page 2021