Pith. sign in

REVIEW 3 major objections 6 minor 53 references

RealDrive: Retrieval-Augmented Driving with Diffusion Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Retrieved driving demos cut collision rate by 40 percent

desk verdict A genuinely new interpolation mechanism for retrieval-augmented diffusion planning, with a solid control design, but the core mechanism is under-validated and several key details are missing. read the letter →

arxiv 2505.24808 v1 pith:2I5FUKI5 submitted 2025-05-30 cs.RO cs.AI

Wenhao Ding , Sushant Veer , Yuxiao Chen , Yulong Cao , Chaowei Xiao , Marco Pavone This is my paper
classification cs.ROcs.AI
keywords retrieval-augmentedgenerationdiffusionmodelautonomousdrivingmotionplanningtrajectorydiversitylong-tailscenariosscenarioretrievalembedding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RealDrive claims that a learning-based driving planner becomes safer and more diverse when, at inference time, it retrieves expert trajectories similar to the current scene from a database and feeds them into a diffusion denoising process. The paper argues that interpolating both observations and actions between the retrieved example and the current scene, rather than pasting in a retrieved action directly, lets the planner adapt the expert behavior to the situation at hand. It also reports that a retrieval model trained with planning objectives finds more useful examples than a generic scene-similarity model. On the Waymo Open Motion Dataset the retrieval-augmented planner reduces average collision rate by roughly 40 percent relative to the same diffusion planner without retrieval. These results matter because data-driven planners are known to struggle with rare, safety-critical scenarios that are under-represented in their training data.

What carries the argument

The load-bearing mechanism is the Retrieval Interpolation Module (RIM), which couples the retrieved demonstration with the current scene throughout the diffusion denoising process. RIM computes an interpolation coefficient $\lambda$ that follows a sigmoid scheduler over denoising steps; at each step it blends the outputs of the multi-head attention layers for the current and retrieved keys and values, and blends the MLP-mapped noisy actions, before feeding the result as the query into the attention. This lets the retrieved expert behavior act as a soft prior that the denoiser gradually adapts to the current scene, rather than a hard replacement. A second component, a task-specific embedding model, maps scenes to vectors so that Euclidean nearest-neighbor search returns plan-relevant examples.

What would settle it

Run RealDrive on a held-out suite of query scenarios deliberately matched with retrieved scenes that have very different agent counts or lane topologies (for example, a 5-agent intersection retrieval for a 20-agent highway query), and compare collision rates against the standard non-retrieval planner on the same queries; if the interpolated latent prior systematically raises collisions or displaces trajectories off-road on such mismatched pairs, the claim that unaligned latent interpolation adapts retrieved knowledge fails.

Watch

Extended reading notes

Core claim

The central claim is that retrieval-augmented generation, applied to a diffusion-based trajectory planner for autonomous driving, improves open-loop planning accuracy, safety, and diversity compared to a diffusion planner trained and used without retrieval. The proposed method, RealDrive, retrieves the top-K most similar expert demonstrations to the current observation, then injects them into the denoising process through a Retrieval Interpolation Module (RIM). RIM interpolates, with a schedule-dependent coefficient $\lambda$, both the transformer attention outputs that represent the retrieved and current observations and the MLP-processed noisy actions, so the retrieved prior is transferred gradually as denoising proceeds. A second claim is that the retrieval model should itself be trained with planning objectives rather than generic scene similarity: the paper reports that a task-specific embedding model improves planning performance over a task-agnostic retriever. The paper reports a 40 percent reduction in collision rate on Waymo and argues that the gain concentrates in the long tail of difficult scenarios.

Load-bearing premise

The paper assumes that interpolating latent embeddings between a retrieved scenario and the current scene in the denoising process remains meaningful even when the two scenes have different numbers of agents, different lane structures, and different map layouts, with no explicit alignment of those elements.

Editorial extensions

If this is right

  • If RealDrive is correct, open-loop planning benchmarks such as Waymo Open Motion can be improved simply by adding a retrieval database to an existing diffusion planner, with a measured 40 percent collision-rate reduction.
  • Joint training with retrieval is required: the paper reports that applying retrieval only at inference time to a model not trained with retrieval does not help, and can hurt.
  • Controllability follows from the choice of retrieved examples: selecting samples with desired characteristics (for example, from a defensive expert) is claimed to transfer those behaviors to the output.
  • Database quality and size are limiting factors: oracle retrieval with ground-truth future trajectories gives the best results, so building larger and higher-quality retrieval corpora is a direct path to further gains.
  • Inference overhead is modest: retrieval adds roughly 0.009 seconds per query in the Waymo setting, keeping the total under a real-time budget.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The interpolation assumption could be probed by ranking retrievals according to scene heterogeneity; if interpolating across scenes with very different agent counts or lane topologies degrades the plan, the method's scope is narrower than the paper claims.
  • The same retrieval-interpolation mechanism could transfer to other conditional diffusion tasks, such as multi-agent motion prediction or simulated traffic behavior generation, where retrieved ground-truth behaviors are available.
  • Because the retrieval database is open to poisoning, the paper's societal-impact note points to a concrete threat model: an adversary injecting crafted examples could steer the planner toward unsafe actions, and filtering or trustworthiness checks would be needed before deployment.
  • The method is demonstrated only in open-loop evaluation; whether the gains persist in closed-loop driving, where the planner's own outputs influence future observations, is untested and could amplify any interpolation errors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. RealDrive proposes a retrieval-augmented diffusion planner for autonomous driving. A task-specific embedding model retrieves top-K similar expert scenarios from the training database, and a Retrieval Interpolation Module (RIM) interpolates the retrieved observations and actions into the denoising process using a sigmoid scheduler whose coefficient depends on the denoising step. Experiments on nuScenes and Waymo Open Motion Dataset report improvements in open-loop planning metrics, including minADE/minFDE and a roughly 40% reduction in average collision rate on Waymo, plus ablations and random-retrieval controls.

Significance. If the claims hold, the paper would be a useful contribution to retrieval-augmented planning: it combines a task-specific retrieval model with diffusion-based trajectory generation, and it explicitly tests the effect of retrieval quality through random-retrieval and oracle-retrieval conditions. The experimental design has notable strengths: same-scene retrieval is excluded to prevent leakage, the random-retrieval control demonstrates sensitivity to retrieval quality, the oracle setting provides an upper bound, and ablations separate the contributions of observation interpolation, action interpolation, and the task-specific embedding. However, the evaluation is entirely open-loop, the training objective of the embedding model is not specified, and the central interpolation mechanism is validated only qualitatively. These gaps currently prevent full verification of the paper's advertised generalization and safety claims.

major comments (3)
  1. [§4.1, Algorithm 1] The training objective for the task-specific embedding model E is never specified. The text says E is trained with a planning-based objective and adopts the same architecture as the denoising model, but no loss function, training data, or training procedure is given. Algorithm 1 takes E as an input and updates only the denoiser parameters θ, so it is unclear whether E is trained jointly with fθ or separately. Because Table 2 (Settings 5/6) shows that retrieval quality is critical — random retrieval degrades minADE6 from 0.164 to 0.498/0.473 on Waymo — this omission blocks reproduction and verification of the core claim that a task-specific retriever is responsible for the gains. Please specify E's training objective, data, and optimization procedure.
  2. [§4.2, Eqs. (5)-(7)] The interpolation coefficient λ in Eq. (5) depends only on the denoising step h, not on any measure of compatibility between the current and retrieved scenes. The authors state that interpolation is performed without explicit alignment of vehicle counts or lane structures. This is a load-bearing assumption: if the retrieved scene differs in agent count, lane topology, or traffic context, the unaligned blend in Eqs. (6) and (7) could corrupt rather than adapt the planned trajectory. The random-retrieval results in Table 2 indicate strong sensitivity to retrieval quality, and the paper provides only qualitative examples (Figures 3, 5, 8-10) and presence/absence ablations (Table 3), not a systematic test of robustness to scene mismatch. I recommend quantitative analysis stratified by retrieval similarity or scene-structure divergence, variation of K, or a comparison with an alignment-aware interpolation baseline.
  3. [Abstract, §5.3, Fig. 4, Conclusion] The abstract and conclusion claim a 40% collision-rate reduction, and the conclusion specifically attributes this to long-tail scenarios. Table 2 on Waymo supports a roughly 40% reduction in avgCR overall (0.0043 vs. 0.0026), but the long-tail claim is not directly quantified: Figure 4 shows distribution shifts, and Appendix B.2 examines only the top-50 highest-minADE samples without reporting collision-rate tail statistics. The final sentence of the paper should either report long-tail-specific metrics (e.g., worst-decile collision rate, rare-scenario subsets) or be revised to state that the reduction is demonstrated on the full validation set.
minor comments (6)
  1. [Eq. (5)] The normalized variable ĥ is introduced as (h-1)/(H-1), but the notation is not defined inline before its first use; please clarify the relationship between ĥ and the diffusion timestep h.
  2. [Eq. (7)] RIM is overloaded: Eq. (6) uses RIM(q, k, kr) with three arguments, while Eq. (7) uses RIM(ã, ãr) with two. Renaming the action interpolation function would avoid ambiguity.
  3. [Tables 1 and 2] The tables report means and standard deviations over five runs but no significance tests or confidence intervals. Given the small differences between some settings, a paired test or bootstrap interval would strengthen the comparisons.
  4. [§5.5] The text states that the retrieval database contains '107 retrieval entries'; this is presumably 10^7 entries given the scale of Waymo, but the notation should be clarified.
  5. [Appendix C.2] The heading contains a typo: 'Dyanmics' should be 'Dynamics'.
  6. [Figure 6] The scheduler sweep appears to select the best (n, m) on the Waymo validation set and then report results on the same set. Please clarify whether a separate validation split was used for hyperparameter selection, or state that the reported numbers involve selection on the evaluation split.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported RAG gains are measured on held-out scenes against independently trained baselines, and no equation-level reduction of the claimed result to its inputs is present.

full rationale

The derivation chain is empirical rather than definitional. RealDrive conditions a diffusion planner on retrieved observations and actions through RIM (Eqs. 5-7), and the output trajectories are obtained by denoising and evaluated on held-out Waymo and nuScenes validation scenes with same-scene retrieval excluded. None of the paper's equations defines the reported performance metrics in terms of the retrieved data by construction. The scheduler (n,m) sweep in Figure 6 is hyperparameter selection, not an identity between a fitted parameter and the reported prediction; the headline gains (e.g., Setting 4 versus Setting 1 in Table 2) compare two independently trained models under the same retrieval database and evaluation protocol. Self-citations to the authors' prior work [37] appear only as a comparison baseline (w/o TSE) and as background, not as a load-bearing premise; no uniqueness theorem or ansatz is imported from the authors' earlier papers. The central claim is therefore self-contained with respect to the external benchmarks and baselines reported.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The free parameter count is minimal: only the two scheduler exponents are tuned for the central mechanism. The load-bearing assumptions are the validity of open-loop metrics, the bicycle model, the Euclidean embedding metric, and the existence of a planning-objective training procedure for the retrieval model, which the paper does not specify. No invented physical entities are introduced.

free parameters (1)
  • Interpolation scheduler exponents (n, m) = (2, 1)
    Parameters of the sigmoid scheduler in Eq. (5), selected by parameter sweep on Waymo (Figure 6). They control the denoising-step-dependent blending coefficient lambda and are central to the method's behavior.
assumptions (5)
  • domain assumption Open-loop metrics (minADE6, minCR, avgCR, TTCE) on 6 sampled trajectories are a valid proxy for planning safety and performance
    All conclusions are drawn from open-loop evaluation in Section 5.1; no closed-loop or on-road validation is provided, so a mismatch between open-loop collision rate and closed-loop safety would invalidate the claimed safety benefit.
  • domain assumption The kinematic bicycle model F (Appendix C.2) accurately represents vehicle motion for loss computation and rollout
    Training loss and trajectory rollouts use F from Eq. (13); errors in this model propagate into all reported ADE, FDE, and collision metrics.
  • standard math The DDPM forward and reverse processes (Eqs. 1-4) are standard and correctly implemented
    The method builds on Ho et al. [11]; no novel mathematical claim is made about diffusion itself.
  • domain assumption Euclidean distance in the embedding space reflects planning-relevant scenario similarity
    Retrieval uses Euclidean distance on embeddings from E (Section 4.1); if this metric does not align with planning relevance, the retrieved priors would not help or could harm, as Settings 5-6 in Table 2 show.
  • ad hoc to paper The task-specific embedding model E can be trained with a planning-based objective as claimed
    Section 4.1 asserts the embedding model is trained with planning objectives but provides no training loss, algorithm, or equation. The central mechanism depends on this unspecified training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RealDrive: Retrieval-Augmented Driving with Diffusion Models." pith.science (2026). https://pith.science/paper/2I5FUKI5

@misc{pith2026250524808,
  author       = {Pith},
  title        = {Pith review of: RealDrive: Retrieval-Augmented Driving with Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2I5FUKI5}},
  note         = {Machine review of arXiv:2505.24808}
}
read the original abstract

Learning-based planners generate natural human-like driving behaviors by learning to reason about nuanced interactions from data, overcoming the rigid behaviors that arise from rule-based planners. Nonetheless, data-driven approaches often struggle with rare, safety-critical scenarios and offer limited controllability over the generated trajectories. To address these challenges, we propose RealDrive, a Retrieval-Augmented Generation (RAG) framework that initializes a diffusion-based planning policy by retrieving the most relevant expert demonstrations from the training dataset. By interpolating between current observations and retrieved examples through a denoising process, our approach enables fine-grained control and safe behavior across diverse scenarios, leveraging the strong prior provided by the retrieved scenario. Another key insight we produce is that a task-relevant retrieval model trained with planning-based objectives results in superior planning performance in our framework compared to a task-agnostic retriever. Experimental results demonstrate improved generalization to long-tail events and enhanced trajectory diversity compared to standard learning-based planners -- we observe a 40% reduction in collision rate on the Waymo Open Motion dataset with RAG.

Figures

Figures reproduced from arXiv: 2505.24808 by the authors.

Figure 1
Figure 1. The ego vehicle (pink) first re￾trieves similar scenarios in the database, then uses the retrieved behaviors to aug￾ment the driving policy. Recent advancements in autonomous driving have increas￾ingly shifted motion planners towards learning-based [1, 2] and end-to-end [3, 4] driving policies that predict planning waypoints directly from vectorized or visual inputs. This trend has largely been fueled by the ease of… view at source ↗
Figure 2
Figure 2. Training and inference pipeline of RealDrive. After sampling the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Examples of retrieved scenarios. The top row shows the query scenarios, and the bottom [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Using RAG reduces the num￾ber of samples with high collision rate and minade6. Using RAG with oracle samples can further reduce the number. We evaluate the influence of retrieval augmentation on planning performance using the nuScenes and Waymo datasets, with results s…
Figure 5
Figure 5. Figure 5: The comparison between the planners w/ RAG and w/o RAG. It is shown that using RAG [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: minADE6 of different in￾terpolation schedulers on Waymo. When the curve is flatter close to h = 0, the performance is better. A critical requirement for driving planners is low-latency in￾ference. However, incorporating retrieval during inference introduces additional …
Figure 7
Figure 7. Figure 7: Time statistics of retrieving under different configurations. The setting we used for the [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: More examples of the retrieval model. B.2 Comparison between w/ RAG and w/o RAG models on long-tail scenarios. We find the top 50 samples according to the minADE of the w/o RAG model and plot them in [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Top 50 samples with largest minADE of w/o RAG model. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: More examples of the comparison between different model outputs. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 19 canonical work pages

  1. [1]

    Pluto: Pushing the limit of imitation learning- based planning for autonomous driving.arXiv preprint arXiv:2404.14327, 2024

    Jie Cheng, Yingbing Chen, and Qifeng Chen. Pluto: Pushing the limit of imitation learning- based planning for autonomous driving.arXiv preprint arXiv:2404.14327, 2024

    arXiv 2024

  2. [2]

    Diffusion-based planning for autonomous driving with flexible guidance.arXiv preprint arXiv:2501.15564, 2025

    Yinan Zheng, Ruiming Liang, Kexin Zheng, Jinliang Zheng, Liyuan Mao, Jianxiong Li, Weihao Gu, Rui Ai, Shengbo Eben Li, Xianyuan Zhan, et al. Diffusion-based planning for autonomous driving with flexible guidance.arXiv preprint arXiv:2501.15564, 2025

    arXiv 2025

  3. [3]

    Transfuser: Imitation with transformer-based sensor fusion for autonomous driving.IEEE transactions on pattern analysis and machine intelligence, 45(11):12878–12895, 2022

    Kashyap Chitta, Aditya Prakash, Bernhard Jaeger, Zehao Yu, Katrin Renz, and Andreas Geiger. Transfuser: Imitation with transformer-based sensor fusion for autonomous driving.IEEE transactions on pattern analysis and machine intelligence, 45(11):12878–12895, 2022

    2022

  4. [4]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17853–17862, 2023

    2023

  5. [5]

    A survey on safety-critical driving scenario generation—a methodological perspective.IEEE Transactions on Intelligent Transportation Systems, 24(7):6971–6988, 2023

    Wenhao Ding, Chejian Xu, Mansur Arief, Haohong Lin, Bo Li, and Ding Zhao. A survey on safety-critical driving scenario generation—a methodological perspective.IEEE Transactions on Intelligent Transportation Systems, 24(7):6971–6988, 2023

    work page 2023

  6. [6]

    Lmdrive: Closed-loop end-to-end driving with large language models

    Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L Waslander, Yu Liu, and Hong- sheng Li. Lmdrive: Closed-loop end-to-end driving with large language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15120–15130, 2024

    2024

  7. [7]

    Drivevlm: The convergence of autonomous driving and large vision-language models.arXiv preprint arXiv:2402.12289, 2024

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models.arXiv preprint arXiv:2402.12289, 2024

    arXiv 2024

  8. [8]

    Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2, 2023

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2, 2023

    arXiv 2023

Show all 53 references
  1. [9]

    Rag-driver: Generalisable driving explanations with retrieval-augmented in-context learning in multi-modal large language model.arXiv preprint arXiv:2402.10828, 2024

    Jianhao Yuan, Shuyang Sun, Daniel Omeiza, Bo Zhao, Paul Newman, Lars Kunze, and Matthew Gadd. Rag-driver: Generalisable driving explanations with retrieval-augmented in-context learning in multi-modal large language model.arXiv preprint arXiv:2402.10828, 2024

    2024

  2. [10]

    Driving-rag: Driving scenarios embedding, search, and rag applications.arXiv preprint arXiv:2504.04419, 2025

    Cheng Chang, Jingwei Ge, Jiazhe Guo, Zelin Guo, Binghong Jiang, and Li Li. Driving-rag: Driving scenarios embedding, search, and rag applications.arXiv preprint arXiv:2504.04419, 2025

    2025 arXiv

  3. [11]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  4. [12]

    A survey of end-to-end driving: Architectures and training methods.IEEE Transactions on Neural Networks and Learning Systems, 33(4):1364–1384, 2020

    Ardi Tampuu, Tambet Matiisen, Maksym Semikin, Dmytro Fishman, and Naveed Muhammad. A survey of end-to-end driving: Architectures and training methods.IEEE Transactions on Neural Networks and Learning Systems, 33(4):1364–1384, 2020

    2020

  5. [13]

    A rule-based behaviour planner for autonomous driving

    Frédéric Bouchard, Sean Sedwards, and Krzysztof Czarnecki. A rule-based behaviour planner for autonomous driving. InInternational Joint Conference on Rules and Reasoning, pages 263–279. Springer, 2022

    2022

  6. [14]

    End to end learning for self-driving cars.arXiv preprint arXiv:1604.07316, 2016

    Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End to end learning for self-driving cars.arXiv preprint arXiv:1604.07316, 2016

    2016 arXiv

  7. [15]

    Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst.arXiv preprint arXiv:1812.03079, 2018

    Mayank Bansal, Alex Krizhevsky, and Abhijit Ogale. Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst.arXiv preprint arXiv:1812.03079, 2018

    2018 arXiv

  8. [16]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 10

    2017

  9. [17]

    Carla: An open urban driving simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. InConference on robot learning, pages 1–16. PMLR, 2017

    2017

  10. [18]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern re...

    2020

  11. [19]

    Vad: Vectorized scene representation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pag...

    2023

  12. [20]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    2024 arXiv

  13. [21]

    Para-drive: Par- allelized architecture for real-time autonomous driving

    Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Par- allelized architecture for real-time autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15449–15458, 2024

    2024

  14. [22]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  15. [23]

    Motiondiffuser: Controllable multi-agent motion prediction using diffusion

    Chiyu Jiang, Andre Cornman, Cheolho Park, Benjamin Sapp, Yin Zhou, Dragomir Anguelov, et al. Motiondiffuser: Controllable multi-agent motion prediction using diffusion. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9644–9653, 2023

    2023

  16. [24]

    Guided conditional diffusion for controllable traffic simulation

    Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Tong Che, Baishakhi Ray, and Marco Pavone. Guided conditional diffusion for controllable traffic simulation. In 2023 IEEE international conference on robotics and automation (ICRA), pages 3560–3566. IEEE, 2023

    2023

  17. [25]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving.arXiv preprint arXiv:2411.15139, 2024

    Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving.arXiv preprint arXiv:2411.15139, 2024

    2024 arXiv

  18. [26]

    Diffusion-es: Gradient-free planning with diffusion for autonomous driving and zero-shot instruction following.arXiv preprint arXiv:2402.06559, 2024

    Brian Yang, Huangyuan Su, Nikolaos Gkanatsios, Tsung-Wei Ke, Ayush Jain, Jeff Schneider, and Katerina Fragkiadaki. Diffusion-es: Gradient-free planning with diffusion for autonomous driving and zero-shot instruction following.arXiv preprint arXiv:2402.06559, 2024

    2024 arXiv

  19. [27]

    Retrieval-augmented reinforcement learning

    Anirudh Goyal, Abram Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adria Puigdomenech Badia, Arthur Guez, Mehdi Mirza, Peter C Humphreys, Ksenia Konyushova, et al. Retrieval-augmented reinforcement learning. InInternational Confer- ence on Machine Learning, pages 7...

    2022

  20. [28]

    Molecule generation with fragment retrieval augmentation.Advances in Neural Information Processing Systems, 37:132463–132490, 2024

    Seul Lee, Karsten Kreis, Srimukh Veccham, Meng Liu, Danny Reidenbach, Saee Paliwal, Arash Vahdat, and Weili Nie. Molecule generation with fragment retrieval augmentation.Advances in Neural Information Processing Systems, 37:132463–132490, 2024

    2024

  21. [29]

    Retrieval augmented dif- fusion model for structure-informed antibody design and optimization.arXiv preprint arXiv:2410.15040, 2024

    Zichen Wang, Yaokun Ji, Jianing Tian, and Shuangjia Zheng. Retrieval augmented dif- fusion model for structure-informed antibody design and optimization.arXiv preprint arXiv:2410.15040, 2024

    2024

  22. [30]

    Ram: Retrieval-based affordance transfer for generalizable zero-shot robotic manipulation.arXiv preprint arXiv:2407.04689, 2024

    Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Congyue Deng, Leonidas Guibas, He Wang, and Yue Wang. Ram: Retrieval-based affordance transfer for generalizable zero-shot robotic manipulation.arXiv preprint arXiv:2407.04689, 2024

    2024 arXiv

  23. [31]

    Strap: Robot sub-trajectory retrieval for augmented policy learning.arXiv preprint arXiv:2412.15182, 2024

    Marius Memmel, Jacob Berg, Bingqing Chen, Abhishek Gupta, and Jonathan Francis. Strap: Robot sub-trajectory retrieval for augmented policy learning.arXiv preprint arXiv:2412.15182, 2024. 11

    2024 arXiv

  24. [32]

    Srsa: Skill retrieval and adaptation for robotic assembly tasks.arXiv preprint arXiv:2503.04538, 2025

    Yijie Guo, Bingjie Tang, Iretiayo Akinola, Dieter Fox, Abhishek Gupta, and Yashraj Narang. Srsa: Skill retrieval and adaptation for robotic assembly tasks.arXiv preprint arXiv:2503.04538, 2025

    2025 arXiv

  25. [33]

    Rac3: Retrieval-augmented corner case comprehension for autonomous driving with vision-language models.arXiv preprint arXiv:2412.11050, 2024

    Yujin Wang, Quanfeng Liu, Jiaqi Fan, Jinlong Hong, Hongqing Chu, Mengjian Tian, Bingzhao Gao, and Hong Chen. Rac3: Retrieval-augmented corner case comprehension for autonomous driving with vision-language models.arXiv preprint arXiv:2412.11050, 2024

    2024

  26. [34]

    Driving with regulation: Interpretable decision-making for autonomous vehicles with retrieval- augmented reasoning via llm.arXiv preprint arXiv:2410.04759, 2024

    Tianhui Cai, Yifan Liu, Zewei Zhou, Haoxuan Ma, Seth Z Zhao, Zhiwen Wu, and Jiaqi Ma. Driving with regulation: Interpretable decision-making for autonomous vehicles with retrieval- augmented reasoning via llm.arXiv preprint arXiv:2410.04759, 2024

  27. [35]

    Ralad: Bridging the real-to-sim domain gap in autonomous driving with retrieval-augmented learning.arXiv preprint arXiv:2501.12296, 2025

    Jiacheng Zuo, Haibo Hu, Zikang Zhou, Yufei Cui, Ziquan Liu, Jianping Wang, Nan Guan, Jin Wang, and Chun Jason Xue. Ralad: Bridging the real-to-sim domain gap in autonomous driving with retrieval-augmented learning.arXiv preprint arXiv:2501.12296, 2025

    2025 arXiv

  28. [36]

    Springer, 2015

    Filippo Santambrogio.Optimal transport for applied mathematicians, volume 87. Springer, 2015

    2015

  29. [37]

    Realgen: Retrieval augmented generation for controllable traffic scenarios

    Wenhao Ding, Yulong Cao, Ding Zhao, Chaowei Xiao, and Marco Pavone. Realgen: Retrieval augmented generation for controllable traffic scenarios. InEuropean Conference on Computer Vision, pages 93–110. Springer, 2024

    2024

  30. [38]

    Retrieval- augmented diffusion models.Advances in Neural Information Processing Systems, 35:15309– 15324, 2022

    Andreas Blattmann, Robin Rombach, Kaan Oktay, Jonas Müller, and Björn Ommer. Retrieval- augmented diffusion models.Advances in Neural Information Processing Systems, 35:15309– 15324, 2022

    2022

  31. [39]

    Rapid: Retrieval augmented training of differentially private diffusion models.arXiv preprint arXiv:2502.12794, 2025

    Tanqiu Jiang, Changjiang Li, Fenglong Ma, and Ting Wang. Rapid: Retrieval augmented training of differentially private diffusion models.arXiv preprint arXiv:2502.12794, 2025

    2025 arXiv

  32. [40]

    Ragdiffusion: Faithful cloth generation via external knowledge assimilation.arXiv preprint arXiv:2411.19528, 2024

    Xianfeng Tan, Yuhan Li, Wenxiang Shang, Yubo Wu, Jian Wang, Xuanhong Chen, Yi Zhang, Ran Lin, and Bingbing Ni. Ragdiffusion: Faithful cloth generation via external knowledge assimilation.arXiv preprint arXiv:2411.19528, 2024

    2024

  33. [41]

    Remodiffuse: Retrieval-augmented motion diffusion model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Remodiffuse: Retrieval-augmented motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 364–373, 2023

    2023

  34. [42]

    Retrieval-augmented diffusion models for time series forecasting.Advances in Neural Information Processing Systems, 37:2766–2786, 2024

    Jingwei Liu, Ling Yang, Hongyan Li, and Shenda Hong. Retrieval-augmented diffusion models for time series forecasting.Advances in Neural Information Processing Systems, 37:2766–2786, 2024

    2024

  35. [43]

    R2-diff: Denoising by diffusion as a refinement of retrieved motion for image-based motion prediction.arXiv preprint arXiv:2306.09483, 2023

    Takeru Oba and Norimichi Ukita. R2-diff: Denoising by diffusion as a refinement of retrieved motion for image-based motion prediction.arXiv preprint arXiv:2306.09483, 2023

    2023 arXiv

  36. [44]

    Sdedit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    2021 arXiv

  37. [45]

    Read: Retrieval-enhanced asymmetric diffusion for motion planning

    Takeru Oba, Matthew Walter, and Norimichi Ukita. Read: Retrieval-enhanced asymmetric diffusion for motion planning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17974–17984, 2024

    2024

  38. [46]

    Ragdp: Retrieve-augmented generative diffusion policy

    Sodtavilan Odonchimed, Tatsuya Matsushima, Simon Holk, Yusuke Iwasawa, and Yutaka Matsuo. Ragdp: Retrieve-augmented generative diffusion policy

  39. [47]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241273668, 2023

    2023

  40. [48]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023. 12

    2023

  41. [49]

    Is conditional generative modeling all you need for decision-making?arXiv preprint arXiv:2211.15657, 2022

    Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making?arXiv preprint arXiv:2211.15657, 2022

    2022 arXiv

  42. [50]

    Billion-scale similarity search with GPUs

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547, 2019

    2019

  43. [51]

    Refdrop: Controllable consistency in image or video generation via reference feature guidance.arXiv preprint arXiv:2405.17661, 2024

    Jiaojiao Fan, Haotian Xue, Qinsheng Zhang, and Yongxin Chen. Refdrop: Controllable consistency in image or video generation via reference feature guidance.arXiv preprint arXiv:2405.17661, 2024

    2024 arXiv

  44. [52]

    Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset

    Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp, Charles R Qi, Yin Zhou, et al. Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset. InProceedings of the IEEE/CVF Internati...

    2021

  45. [53]

    Predictive risk estimation for intelligent adas functions

    Julian Eggert. Predictive risk estimation for intelligent adas functions. In17th International IEEE Conference on Intelligent Transportation Systems (ITSC), pages 711–718. IEEE, 2014. 13 A Potential Societal Impacts While RealDrive improves planning performance and safety by l...

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.