Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Non-differentiable Reward Optimization for Diffusion-based Autonomous Motion Planning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Diffusion motion planners can be trained to directly optimize collision and goal-success metrics by using them as RL rewards, with dynamic thresholds shaping a dense signal.

desk verdict A legitimate DDPO-for-planning application with a useful dynamic thresholding trick; the headline numbers are plausible but the missing variance and threshold specification make the central claim under-supported. read the letter →

arxiv 2507.12977 v1 pith:PEJZI354 submitted 2025-07-17 cs.RO

Giwon Lee , Daehee Park , Jaewoo Jeong , Kuk-Jin Yoon This is my paper
classification cs.RO
keywords diffusionmodelsmotionplanningreinforcementlearningnon-differentiablerewardscollisionavoidancedynamicthresholdingtrajectorypredictionautonomousnavigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that diffusion-based motion planners, normally trained by denoising toward ground-truth trajectories, can be fine-tuned with reinforcement learning to directly optimize non-differentiable planning metrics such as collision rate, goal success, and discomfort rate. The authors show that the central obstacle, sparse rewards from threshold-based metrics, can be handled by a dynamic thresholding procedure that keeps the batch mean reward near half its maximum, producing a dense and stable training signal. On CrowdNav and ETH-UCY, the fine-tuned planner reduces CrowdNav collision rate from 0.0628 to 0.0150 and raises success rate from 0.3540 to 0.4931, outperforming differentiable baselines and gradient-guidance alternatives. The claim is that reward-driven training on the deployed metrics themselves, not surrogate losses, is what delivers the gains.

What carries the argument

The mechanism is a multi-step MDP view of reverse diffusion plus a dynamic thresholding reward shaper. Each reverse denoising step from noise to trajectory is a state-action transition whose log-probability is weighted by an advantage computed from the non-differentiable reward of the final trajectory; the threshold parameter in each metric (collision distance, goal distance, jerk) is adapted per iteration so that the batch mean reward approaches B/2, turning a mostly-zero signal into a signal that distinguishes good from bad samples. All gradient information for denoising remains differentiable; the reward itself never needs a gradient.

What would settle it

Train the same backbone with the dynamic thresholding algorithm under several initial thresholds and adaptation schedules, then evaluate at the fixed CrowdNav thresholds; if collision and success rates fluctuate with the initial hyperparameters or fall to backbone level when the initial threshold is far from the evaluation threshold, the claimed benefit of dynamic thresholding would be refuted. A simpler version would run the Table IV ablation with a grid of deviation and adaptation-rate values and check whether the 0.0146 collision rate persists.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a diffusion planner's denoising process can be treated as sequential decisions and updated by policy gradient with a reward that directly evaluates the generated trajectory by collision rate and success rate. Because these rewards are threshold-based and often sparse, the paper introduces dynamic thresholding: at each iteration, the threshold defining a collision or a success is raised or lowered so that the batch mean reward stays close to B/2, the midpoint of the batch; reward then weights the likelihood of each denoising step through an importance-sampled policy gradient with a baseline normalized by the reward standard deviation. The measured consequence is large gains on both safety and effectiveness metrics relative to the backbone and to methods that use differentiable collision costs, with the non-differentiable discrete rewards outperforming their differentiable counterparts in the ablations.

Load-bearing premise

The method assumes that keeping the batch mean reward near half the maximum, via the dynamic threshold updates, produces a stable and informative training signal that transfers to the fixed thresholds used at evaluation; the algorithm's hyperparameters and sensitivity are not reported.

Editorial extensions

If this is right

  • A planner fine-tuned this way can be steered toward whichever metric is rewarded; optimizing success rate alone pushes collision rate down as well, indicating that the metrics are coupled.
  • Dynamic thresholding benefits even differentiable rewards like collision cost, but gives the largest improvement on discrete non-differentiable rewards.
  • The method can incorporate arbitrary numbers and types of rewards (safety, effectiveness, comfort) weighted by importance factors, so deployment-specific objectives can be added without rearchitecting.
  • Because the forecaster is kept frozen, the reported gains come from planner training alone, suggesting the planning module can be improved independently of prediction quality.
  • The final model improves ADE as well, so safety gains do not come at the cost of fidelity to ground-truth trajectories.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If dynamic thresholding works by keeping the batch reward near B/2, then training thresholds differ from the fixed evaluation thresholds; a natural testable extension is to anneal thresholds from adaptive values to the deployment thresholds during training and measure whether the gains persist.
  • The sparse-reward argument suggests the same scheme could apply to any threshold-based planning metric (comfort, rule violations, time-to-collision) or to other generative policies, since the method does not rely on domain-specific differentiable structure.
  • The ablation showing that performance drops as thresholds become less sensitive implies that threshold choice may matter as much as the RL update itself; sweeping initial thresholds and adaptation rates would clarify how much of the gain comes from dynamic adjustment versus the reward definition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes a reinforcement learning-based fine-tuning scheme for diffusion-based motion planners, built on DDPO, to optimize non-differentiable rewards derived from collision rate, success rate, and discomfort rate. To handle sparse rewards, it introduces a dynamic thresholding procedure (Algorithm 1) that adjusts the threshold used in each binary reward so that the batch mean reward stays near B/2. Experiments on CrowdNav and ETH-UCY report lower collision rate, higher success rate, and lower ADE compared to DTPP, DIPP, GameTheoretic, the backbone, and gradient-guidance baselines. Ablations compare single- versus multi-reward training, differentiable versus non-differentiable rewards, and training with versus without dynamic thresholding.

Significance. If the empirical claims hold, the paper makes a useful contribution: it demonstrates a practical way to directly optimize non-differentiable planning metrics in diffusion planners and shows consistent improvements on safety and effectiveness metrics over surrogate differentiable losses. The method is grounded in the standard DDPO gradient (Eq. 7), and the multi-reward formulation is flexible. The authors also provide a direct comparison against gradient guidance and an ablation isolating dynamic thresholding, which are informative. However, the strength of the evidence is limited by missing uncertainty estimates, under-specified hyperparameters of the key new algorithm, and an incomplete description of the baseline comparison protocol, so the significance is conditional on additional validation.

major comments (4)
  1. [Algorithm 1, Sections III-E and IV-D] Algorithm 1 introduces the paper's central dynamic thresholding mechanism, but none of its controlling hyperparameters are specified: the initial threshold epsilon_init for each reward (the fixed evaluation thresholds in Section IV-B are not explicitly identified with epsilon_init), the deviation delta, the adaptation rate schedule alpha(j), and the maximum iterations J are all absent, and no threshold trajectories or final learned thresholds are reported. Because the algorithm deliberately adjusts epsilon away from the evaluation thresholds as the policy changes, the reported improvements (e.g., collision rate 0.0150 in Table I) may depend on the specific adaptation path rather than on a robust property of the learned planner. A sensitivity analysis over these hyperparameters, at least one training curve of epsilon, and the exact values used for the reported runs are necessary to support the central claim.
  2. [Tables I-IV] All quantitative results in Tables I-IV are single point estimates: no standard deviations, confidence intervals, or numbers of seeds are reported. This is particularly problematic for the small margins (e.g., ADE 0.1984 vs 0.1991 in Table I, success rate 0.4931 vs 0.4987 in Table II) and for the small collision-rate values, where stochasticity can change rankings. The dynamic-thresholding ablation in Table IV is also a single configuration. The state-of-the-art claim requires mean and variance over at least several independent runs, ideally with paired comparisons.
  3. [Section V-B and Table IV] Section V-B claims that non-differentiable rewards generally outperform differentiable rewards, but Table IV shows the opposite when dynamic thresholding is not used: collision cost achieves 0.0216 collision rate without DT, while collision rate achieves 0.0232. The superiority of the non-differentiable reward appears only in the w/ DT row. This interaction between reward type and dynamic thresholding should be acknowledged, because it complicates the interpretation of Table III and the paper-level claim that non-differentiable rewards are the source of the gains.
  4. [Section IV-C and Section IV-D] The comparison protocol for the main baselines is under-specified. Section IV-C states that all baseline models are conventionally trained with gradients from L2 with GT trajectory and collision cost, but it does not say whether DTPP, DIPP, and GameTheoretic use their original architectures, whether they are retrained under the same frozen-forecaster condition, or whether the numbers come from prior papers. Without a clear description of backbone, training schedule, and evaluation protocol, the state-of-the-art claim in Table I cannot be independently assessed.
minor comments (6)
  1. [Section III-C, Eq. (5)] The objective sums R(y_k, F) over all diffusion steps, but the reward is defined only for the final trajectory and Eqs. (6)-(7) use R(hat y, F). This is inconsistent; either remove the sum and use R(y_0) or define intermediate rewards.
  2. [Section IV-B] The sentence giving thresholds, '0.6 m, 0.2 m for CrowdNav and 0.2 m, 0.5 m for ETH-UCY', does not state which threshold corresponds to collision rate and which to success rate; please label them explicitly.
  3. [Section V-A and Tables II-III] The discomfort rate is used as a reward and metric, but its definition (the jerk threshold and how jerk is computed) is never given; this should be added to Section IV-B.
  4. [Section IV-D] The reward-based training details omit standard RL and optimization hyperparameters such as learning rate, number of fine-tuning epochs, number of diffusion steps K, and the alpha(j) schedule used for dynamic thresholding; including these is needed for reproducibility.
  5. [Table III caption] The caption misspells CrowdNav as 'CrowNav'; please correct it.
  6. [General] The paper does not mention code or model release; given the number of unspecified training details, a public implementation would substantially help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training reward is the optimization target by design, but no prediction is derived from a fitted input and all load-bearing machinery is external or self-contained.

full rationale

The paper's central claim is that RL fine-tuning with dynamic thresholding lets a diffusion planner optimize non-differentiable safety/effectiveness metrics. The overlap between the training reward (collision rate, success rate in Sec. III-D) and the headline metrics (Sec. IV-B) is explicit and is the intended optimization target, not a hidden fitted input passed off as a prediction; test-set evaluation still requires generalization. The dynamic thresholding algorithm (Algorithm 1) adjusts the reward threshold to keep batch-mean reward near B/2, so the training reward is not identical to the fixed evaluation thresholds (0.6/0.2 m for CrowdNav, 0.2/0.5 m for ETH-UCY), further decoupling the optimization from the reported metric. Independent support exists: ADE, which is not in the CrowdNav reward (only collision, success, and discomfort are), also improves (Table I), and the dynamic-thresholding ablation (Table IV) compares both collision-rate and collision-cost rewards with and without DT. Self-citations ([4], [22], [26]) appear only as background on trajectory prediction and transferability and are not load-bearing; the core RL method cites external DDPO [19], and the baselines are external ([15], [45], [47], [48]). The reader's concern about unspecified threshold hyperparameters and the lack of sensitivity analysis is a robustness and correctness matter, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central contribution is a training procedure, not a derivation, so the main free parameters are the reward weighting and the dynamic thresholding hyperparameters. The DDPO formulation is taken as given; the only new ingredient is the threshold reshaping heuristic.

free parameters (3)
  • reward weights lambda_m = CrowdNav: 4,5,1 for collision, success, discomfort; ETH-UCY: 3,7
    In Section III-D, the authors state these are 'empirically determined'; no sensitivity analysis is provided.
  • dynamic threshold hyperparameters (alpha(j), J, delta) = not reported
    Algorithm 1 requires initial threshold, deviation, adaptation rate, and max iterations, but the paper gives no values or tuning procedure.
  • reward target mean B/2 = B/2 by design
    The algorithm forces the batch reward mean toward half the batch size; this design choice is not justified theoretically.
assumptions (4)
  • standard math DDPM training objective is a valid ELBO for the diffusion model
    Inherited from Ho et al. [20]; used as backbone initialization.
  • domain assumption Treating the denoising process as an MDP with reward at the final step (DDPO formulation)
    Borrowed from Black et al. [19]; the paper applies it directly with no modification.
  • standard math The reward is non-differentiable but its expected value can be optimized via policy gradient with importance sampling (Eq. 7)
    Policy gradient theorem; the paper cites [41].
  • ad hoc to paper A batch mean reward near B/2 is a good target for learning
    Central heuristic of Algorithm 1; no theoretical or empirical justification beyond the reported numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Non-differentiable Reward Optimization for Diffusion-based Autonomous Motion Planning." pith.science (2026). https://pith.science/paper/PEJZI354

@misc{pith2026250712977,
  author       = {Pith},
  title        = {Pith review of: Non-differentiable Reward Optimization for Diffusion-based Autonomous Motion Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PEJZI354}},
  note         = {Machine review of arXiv:2507.12977}
}
read the original abstract

Safe and effective motion planning is crucial for autonomous robots. Diffusion models excel at capturing complex agent interactions, a fundamental aspect of decision-making in dynamic environments. Recent studies have successfully applied diffusion models to motion planning, demonstrating their competence in handling complex scenarios and accurately predicting multi-modal future trajectories. Despite their effectiveness, diffusion models have limitations in training objectives, as they approximate data distributions rather than explicitly capturing the underlying decision-making dynamics. However, the crux of motion planning lies in non-differentiable downstream objectives, such as safety (collision avoidance) and effectiveness (goal-reaching), which conventional learning algorithms cannot directly optimize. In this paper, we propose a reinforcement learning-based training scheme for diffusion motion planning models, enabling them to effectively learn non-differentiable objectives that explicitly measure safety and effectiveness. Specifically, we introduce a reward-weighted dynamic thresholding algorithm to shape a dense reward signal, facilitating more effective training and outperforming models trained with differentiable objectives. State-of-the-art performance on pedestrian datasets (CrowdNav, ETH-UCY) compared to various baselines demonstrates the versatility of our approach for safe and effective motion planning.

Figures

Figures reproduced from arXiv: 2507.12977 by the authors.

Figure 1
Figure 1. Conventional methods cannot train diffusion models on non [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed method for optimizing non-differentiable rewards in diffusion-based motion planning for dynamic multi [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results on the CrowdNav and ETH-UCY, where human (orange) represent surrounding agents and the robot (blue, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Improvement of planning results over training epochs on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparison of optimization performances with different [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Model

    cs.CV 2025-07 conditional novelty 6.0 of 10

    GALTraj improves long-tail trajectory prediction by actively mining high-error samples and augmenting them with controllable diffusion-generated variations during training.

Reference graph

Works this paper leans on

49 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [15]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems, vol. 34, pp. 8780–8794, 2021

    2021

  2. [1]

    St-p3: End- to-end vision-based autonomous driving via spatial-temporal feature learning,

    S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao, “St-p3: End- to-end vision-based autonomous driving via spatial-temporal feature learning,” in European Conference on Computer Vision . Springer, 2022, pp. 533–549

    work page 2022

  3. [2]

    Per- ceive, predict, and plan: Safe motion planning through interpretable semantic representations,

    A. Sadat, S. Casas, M. Ren, X. Wu, P. Dhawan, and R. Urtasun, “Per- ceive, predict, and plan: Safe motion planning through interpretable semantic representations,” in Computer Vision–ECCV 2020: 16th Eu- ropean Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII 16. Springer, 2020, pp. 414–430

    work page 2020

  4. [3]

    Dsdnet: Deep structured self-driving network,

    W. Zeng, S. Wang, R. Liao, Y . Chen, B. Yang, and R. Urtasun, “Dsdnet: Deep structured self-driving network,” in Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16 . Springer, 2020, pp. 156–172

    work page 2020

  5. [4]

    Multi-modal knowl- edge distillation-based human trajectory forecasting,

    J. Jeong, S. Lee, D. Park, G. Lee, and K.-J. Yoon, “Multi-modal knowl- edge distillation-based human trajectory forecasting,” in Proceedings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 24 222–24 233

    work page 2025

  6. [5]

    Fast-replanning motion control for non-holonomic vehicles with aborting a*,

    M. Missura, A. Roychoudhury, and M. Bennewitz, “Fast-replanning motion control for non-holonomic vehicles with aborting a*,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 10 267–10 274

    work page 2022

  7. [6]

    Informed rrt*: Optimal sampling-based path planning focused via direct sampling of an admissible ellipsoidal heuristic,

    J. D. Gammell, S. S. Srinivasa, and T. D. Barfoot, “Informed rrt*: Optimal sampling-based path planning focused via direct sampling of an admissible ellipsoidal heuristic,” in 2014 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2014, pp. 2997– 3004

    work page 2014

  8. [7]

    Path planning using neural a* search,

    R. Yonetani, T. Taniai, M. Barekatain, M. Nishimura, and A. Kanezaki, “Path planning using neural a* search,” in International conference on machine learning. PMLR, 2021, pp. 12 029–12 039

    work page 2021

Show all 49 references
  1. [8]

    Sampling-based algorithms for optimal motion planning,

    S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” The international journal of robotics research , vol. 30, no. 7, pp. 846–894, 2011

    2011

  2. [9]

    Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety,

    J. Chen, B. Yuan, and M. Tomizuka, “Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety,” in 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2019, pp. 2884–2890

    2019

  3. [10]

    Safe reinforcement learning with stability guarantee for motion planning of autonomous vehicles,

    L. Zhang, R. Zhang, T. Wu, R. Weng, M. Han, and Y . Zhao, “Safe reinforcement learning with stability guarantee for motion planning of autonomous vehicles,” IEEE transactions on neural networks and learning systems, vol. 32, no. 12, pp. 5435–5444, 2021

    2021

  4. [11]

    Parting with misconceptions about learning-based vehicle motion planning,

    D. Dauner, M. Hallgarten, A. Geiger, and K. Chitta, “Parting with misconceptions about learning-based vehicle motion planning,” in Proceedings of The 7th Conference on Robot Learning, ser. Proceedings of Machine Learning Research, J. Tan, M. Toussaint, and K. Darvish, Eds., vo...

    2023

  5. [12]

    Dif- ferentiable constrained imitation learning for robot motion planning and control,

    C. Diehl, J. Adamek, M. Kr ¨uger, F. Hoffmann, and T. Bertram, “Dif- ferentiable constrained imitation learning for robot motion planning and control,” arXiv preprint arXiv:2210.11796 , 2022

    2022 arXiv

  6. [13]

    Diffusion-es: Gradient-free planning with diffusion for autonomous and instruction-guided driving,

    B. Yang, H. Su, N. Gkanatsios, T.-W. Ke, A. Jain, J. Schneider, and K. Fragkiadaki, “Diffusion-es: Gradient-free planning with diffusion for autonomous and instruction-guided driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024,...

    2024

  7. [14]

    Motiondiffuser: Controllable multi-agent motion prediction using diffusion,

    C. Jiang, A. Cornman, C. Park, B. Sapp, Y . Zhou, D. Anguelov, et al. , “Motiondiffuser: Controllable multi-agent motion prediction using diffusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 9644–9653

    2023

  8. [16]

    Language-guided traffic simulation via scene-level diffu- sion,

    Z. Zhong, D. Rempe, Y . Chen, B. Ivanovic, Y . Cao, D. Xu, M. Pavone, and B. Ray, “Language-guided traffic simulation via scene-level diffu- sion,” in Conference on Robot Learning. PMLR, 2023, pp. 144–177

    2023

  9. [17]

    Safe-sim: Safety-critical closed-loop traffic simulation with diffusion- controllable adversaries,

    W.-J. Chang, F. Pittaluga, M. Tomizuka, W. Zhan, and M. Chandraker, “Safe-sim: Safety-critical closed-loop traffic simulation with diffusion- controllable adversaries,” in European Conference on Computer Vi- sion. Springer, 2025, pp. 242–258

    2025

  10. [18]

    Fine-tuning language models with reward learning on policy,

    H. Lang, F. Huang, and Y . Li, “Fine-tuning language models with reward learning on policy,” arXiv preprint arXiv:2403.19279 , 2024

    2024 arXiv

  11. [19]

    Training diffusion models with reinforcement learning,

    K. Black, M. Janner, Y . Du, I. Kostrikov, and S. Levine, “Training diffusion models with reinforcement learning,” in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=YCWjhGrJFD

    2024

  12. [20]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840–6851, 2020

    2020

  13. [21]

    Improved denoising diffusion prob- abilistic models,

    A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion prob- abilistic models,” in International conference on machine learning . PMLR, 2021, pp. 8162–8171

    2021

  14. [22]

    Improving transferability for cross- domain trajectory prediction via neural stochastic differential equa- tion,

    D. Park, J. Jeong, and K.-J. Yoon, “Improving transferability for cross- domain trajectory prediction via neural stochastic differential equa- tion,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 9, 2024, pp. 10 145–10 154

    2024

  15. [23]

    Maximum likelihood training of score-based diffusion models,

    Y . Song, C. Durkan, I. Murray, and S. Ermon, “Maximum likelihood training of score-based diffusion models,” Advances in neural infor- mation processing systems , vol. 34, pp. 1415–1428, 2021

    2021

  16. [24]

    Data-driven diffusion models for enhancing safety in autonomous vehicle traffic simulations,

    J. Lu, S. Azam, G. Alcan, and V . Kyrki, “Data-driven diffusion models for enhancing safety in autonomous vehicle traffic simulations,” arXiv preprint arXiv:2410.04809, 2024

    2024 arXiv

  17. [25]

    Plan- ning with diffusion for flexible behavior synthesis,

    M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Plan- ning with diffusion for flexible behavior synthesis,” arXiv preprint arXiv:2205.09991, 2022

    2022 arXiv

  18. [26]

    Leveraging future relationship reasoning for vehicle trajectory prediction,

    D. Park, H. Ryu, Y . Yang, J. Cho, J. Kim, and K.-J. Yoon, “Leveraging future relationship reasoning for vehicle trajectory prediction,” in The Eleventh International Conference on Learning Representations, 2023

    2023

  19. [27]

    R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduc- tion. MIT press, 2018

    2018

  20. [28]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. , “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  21. [29]

    M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, 2014

    2014

  22. [30]

    A markovian decision process,

    R. Bellman, “A markovian decision process,” Journal of Mathematics and Mechanics, vol. 6, no. 5, pp. 679–684, 1957

    1957

  23. [31]

    Deterministic policy gradient algorithms,

    D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Ried- miller, “Deterministic policy gradient algorithms,” International Con- ference on Machine Learning , pp. 387–395, 2014

    2014

  24. [32]

    Comprehensive reactive safety: No need for a trajectory if you have a strategy,

    F. Da, “Comprehensive reactive safety: No need for a trajectory if you have a strategy,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2022, pp. 2903–2910

    2022

  25. [33]

    Autonomous driving motion planning with constrained iterative lqr,

    J. Chen, W. Zhan, and M. Tomizuka, “Autonomous driving motion planning with constrained iterative lqr,” IEEE Transactions on Intelli- gent Vehicles, vol. 4, no. 2, pp. 244–254, 2019

    2019

  26. [34]

    Leader: Learning attention over driving behaviors for planning under uncertainty,

    M. H. Danesh, P. Cai, and D. Hsu, “Leader: Learning attention over driving behaviors for planning under uncertainty,” in Conference on robot learning. PMLR, 2023, pp. 199–211

    2023

  27. [35]

    Kb-tree: Learnable and continuous monte- carlo tree search for autonomous driving planning,

    L. Lei, R. Luo, R. Zheng, J. Wang, J. Zhang, C. Qiu, L. Ma, L. Jin, P. Zhang, and J. Chen, “Kb-tree: Learnable and continuous monte- carlo tree search for autonomous driving planning,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2...

    2021

  28. [36]

    Driving maneuvers prediction based autonomous driving control by deep monte carlo tree search,

    J. Chen, C. Zhang, J. Luo, J. Xie, and Y . Wan, “Driving maneuvers prediction based autonomous driving control by deep monte carlo tree search,” IEEE transactions on vehicular technology, vol. 69, no. 7, pp. 7146–7158, 2020

    2020

  29. [37]

    Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforce- ment learning,

    C. Chen, Y . Liu, S. Kreiss, and A. Alahi, “Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforce- ment learning,” in 2019 international conference on robotics and automation (ICRA). IEEE, 2019, pp. 6015–6022

    2019

  30. [38]

    Rethinking closed-loop training for autonomous driving,

    C. Zhang, R. Guo, W. Zeng, Y . Xiong, B. Dai, R. Hu, M. Ren, and R. Urtasun, “Rethinking closed-loop training for autonomous driving,” in European Conference on Computer Vision . Springer, 2022, pp. 264–282

    2022

  31. [39]

    Umbrella: Uncertainty-aware model-based offline reinforcement learning leveraging planning,

    C. Diehl, T. Sievernich, M. Kr ¨uger, F. Hoffmann, and T. Bertram, “Umbrella: Uncertainty-aware model-based offline reinforcement learning leveraging planning,” arXiv preprint arXiv:2111.11097, 2021

    2021 arXiv

  32. [40]

    Model-based reinforcement learning for atari,

    L. Kaiser, M. Babaeizadeh, P. Milos, B. Osinski, R. H. Camp- bell, K. Czechowski, D. Erhan, C. Finn, P. Kozakowski, S. Levine, et al., “Model-based reinforcement learning for atari,” arXiv preprint arXiv:1903.00374, 2019

    1903 arXiv

  33. [41]

    Approximately optimal approximate reinforcement learning,

    S. Kakade and J. Langford, “Approximately optimal approximate reinforcement learning,” in Proceedings of the nineteenth international conference on machine learning , 2002, pp. 267–274

    2002

  34. [42]

    High- dimensional continuous control using generalized advantage estima- tion,

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estima- tion,” arXiv preprint arXiv:1506.02438 , 2015

    2015 arXiv

  35. [43]

    You’ll never walk alone: Modeling social behavior for multi-target tracking,

    S. Pellegrini, A. Ess, K. Schindler, and L. Van Gool, “You’ll never walk alone: Modeling social behavior for multi-target tracking,” in 2009 IEEE 12th international conference on computer vision . IEEE, 2009, pp. 261–268

    2009

  36. [44]

    Crowds by example,

    A. Lerner, Y . Chrysanthou, and D. Lischinski, “Crowds by example,” in Computer graphics forum , vol. 26, no. 3. Wiley Online Library, 2007, pp. 655–664

    2007

  37. [45]

    A game-theoretic framework for joint forecasting and planning,

    K. Kedia, P. Dan, and S. Choudhury, “A game-theoretic framework for joint forecasting and planning,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 6773–6778

    2023

  38. [46]

    Human trajectory prediction via neural social physics,

    J. Yue, D. Manocha, and H. Wang, “Human trajectory prediction via neural social physics,” in European conference on computer vision . Springer, 2022, pp. 376–394

    2022

  39. [47]

    Dtpp: Differentiable joint conditional prediction and cost evaluation for tree policy planning in autonomous driving,

    Z. Huang, P. Karkus, B. Ivanovic, Y . Chen, M. Pavone, and C. Lv, “Dtpp: Differentiable joint conditional prediction and cost evaluation for tree policy planning in autonomous driving,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6806–6812

    2024

  40. [48]

    Differentiable integrated motion prediction and planning with learnable cost function for autonomous driving,

    Z. Huang, H. Liu, J. Wu, and C. Lv, “Differentiable integrated motion prediction and planning with learnable cost function for autonomous driving,” IEEE transactions on neural networks and learning systems , 2023

    2023

  41. [49]

    Stochastic trajectory prediction via motion indeterminacy diffusion,

    T. Gu, G. Chen, J. Li, C. Lin, Y . Rao, J. Zhou, and J. Lu, “Stochastic trajectory prediction via motion indeterminacy diffusion,” in Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 17 113–17 122

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.