The reviewed record of science sign in
Pith

arxiv: 2605.14832 · v1 · pith:UASJUCFJ · submitted 2026-05-14 · cs.RO · cs.CV

Learning Direct Control Policies with Flow Matching for Autonomous Driving

Marcello Ceresini , Federico Pirazzoli , Andrea Bertogalli , Lorenzo Cipelli , Filippo D'Addeo , Anthony Dell'Eva , Alessandro Paolo Capasso , Alberto Broggi This is my paper

Reviewed by Pith2026-06-30 20:42 UTCgrok-4.3pith:UASJUCFJopen to challenge →

classification cs.RO cs.CV
keywords autonomous drivingflow matchingbird's eye viewcontrol trajectoriesdistribution shiftclosed-loop controlODE integrationtrajectory generation
0
0 comments X

The pith

A flow-matching planner trained only on urban simulator data directly outputs acceleration and curvature controls that maintain stable closed-loop performance on unseen highways and cities.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a flow-matching planner that generates direct control trajectories for autonomous driving. The model is conditioned on a bird's-eye-view raster of the scene and produces acceleration and curvature sequences through a small number of ODE integration steps. Training occurs exclusively on urban scenarios from a 2D traffic simulator with reactive agents representing the city of Parma. Evaluation in closed-loop shows the model completes both in-distribution and out-of-distribution scenarios, including multi-lane highways and unseen urban layouts, while maintaining stable control. The authors attribute reliable generalization to the geometry-centric BEV input and the smooth vector field learned by flow matching.

Core claim

The flow-matching planner directly outputs actionable control trajectories defined by acceleration and curvature profiles. Conditioned on a bird's-eye-view raster of the surrounding scene, it generates control sequences in a small number of ODE integration steps. Trained exclusively on urban scenarios collected from a 2D traffic simulator with reactive agents, the model generalizes reliably to markedly out-of-distribution environments including multi-lane highways and unseen urban scenarios, maintaining stable closed-loop control and successfully completing those scenarios.

What carries the argument

The flow-matching formulation that learns a smooth vector field mapping BEV scene rasters to sequences of acceleration and curvature controls, allowing generation in few ODE integration steps.

If this is right

  • The model maintains stable closed-loop control in environments that differ substantially from the training distribution.
  • Scenarios on multi-lane highways and unseen urban layouts are completed successfully without retraining.
  • Low-latency inference is achieved through a small number of ODE integration steps suitable for real-time re-planning.
  • The BEV representation provides a geometry-centric view that is less sensitive to distributional shifts than other inputs.
  • The learned vector field degrades gracefully under distribution shift rather than producing unstable outputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Geometry-focused inputs like BEV rasters may reduce sensitivity to visual or appearance shifts in other robotic control settings.
  • Few-step ODE sampling could support higher-frequency re-planning loops in vehicles with limited compute.
  • If the simulator's reactive agents capture essential interaction dynamics, the same training pipeline could be applied to additional simulated environments before real-world testing.

Load-bearing premise

The 2D traffic simulator with reactive agents produces training data whose distribution shift properties to multi-lane highways and unseen urban layouts are representative enough for the claimed generalization to hold in closed-loop operation.

What would settle it

A closed-loop test on a multi-lane highway with traffic dynamics outside the 2D simulator's distribution in which the vehicle loses stability or fails to complete the scenario.

Figures

Figures reproduced from arXiv: 2605.14832 by Alberto Broggi, Alessandro Paolo Capasso, Andrea Bertogalli, Anthony Dell'Eva, Federico Pirazzoli, Filippo D'Addeo, Lorenzo Cipelli, Marcello Ceresini.

Figure 1
Figure 1. Figure 1: Satellite view of Parma with scenario crop regions. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Lexus RX 350h, the autonomous vehicle used for real-world testing. The vehicle is [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of the four BEV channels: (a) obstacles, (b) drivable area, (c) ego route, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of ego-agent controls and speed in the training set. Left: acceleration. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of the BEV-conditioned flow-matching architecture. The BEV raster is encoded [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Left: ODE integration time vs. NFE by solver. Right: fraction of total inference time [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
read the original abstract

We present a flow-matching planner for autonomous driving that directly outputs actionable control trajectories defined by acceleration and curvature profiles. The model is conditioned on a bird's-eye-view (BEV) raster of the surrounding scene and generates control sequences in a small number of Ordinary Differential Equations (ODE) integration steps, enabling low-latency inference suitable for real-time closed-loop re-planning. We train exclusively on urban scenarios (real urban city streets, intersections and roundabouts of the city of Parma, Italy) collected from a 2D traffic simulator with reactive agents, and evaluate in closed-loop on both in-distribution and markedly out-of-distribution environments, including multi-lane highways and unseen urban scenarios. Our results show that the model generalizes reliably to these unseen conditions, maintaining stable closed-loop control and successfully completing scenarios that differ substantially from the training distribution. We attribute this to the BEV representation, which provides a geometry-centric view of the scene that is inherently less sensitive to distributional shifts, and to the flow-matching formulation, which learns a smooth vector field that degrades gracefully under distribution shift. We provide video demonstrations of closed-loop behavior at https://marcelloceresini.github.io/DirectControlFlowMatching.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript presents a flow-matching planner for autonomous driving that directly outputs actionable control trajectories defined by acceleration and curvature profiles, conditioned on a bird's-eye-view (BEV) raster of the surrounding scene. The model is trained exclusively on urban scenarios from a 2D traffic simulator with reactive agents (Parma, Italy) and evaluated in closed-loop on both in-distribution and out-of-distribution environments including multi-lane highways and unseen urban layouts. The authors claim reliable generalization to these unseen conditions due to the geometry-centric BEV representation and the smooth vector field learned by flow matching, with low-latency inference via few ODE steps.

Significance. If the generalization results hold under rigorous quantitative evaluation, the approach could provide a useful contribution to real-time planning by combining direct control policies with flow matching for graceful degradation under shift. The focus on BEV inputs and simulation-based closed-loop testing addresses relevant challenges in autonomous driving robustness. However, the absence of supporting metrics substantially reduces the assessed significance of the reported findings.

major comments (2)
  1. Abstract: the claim that 'the model generalizes reliably to these unseen conditions, maintaining stable closed-loop control and successfully completing scenarios that differ substantially from the training distribution' is unsupported by any quantitative metrics (success rates, collision rates, number of scenarios, or failure modes), rendering the central empirical claim unverifiable from the presented material.
  2. Evaluation / closed-loop OOD experiments: the training and test distributions are both generated by the identical 2D reactive-agent simulator; the manuscript provides no analysis or ablation showing that the induced shifts (lacking tire models, latency, sensor noise, and high-speed interaction dynamics) are representative of genuine highway or urban OOD regimes, which is load-bearing for the generalization attribution to BEV + flow matching.
minor comments (1)
  1. Abstract: the video link is referenced but the manuscript does not describe which specific closed-loop behaviors or failure cases are demonstrated in the videos.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback highlighting the need for stronger quantitative support and clearer discussion of the evaluation setup. We address each major comment below and will revise the manuscript to improve verifiability while maintaining an honest account of the simulation-based nature of the work.

read point-by-point responses

  1. Referee: Abstract: the claim that 'the model generalizes reliably to these unseen conditions, maintaining stable closed-loop control and successfully completing scenarios that differ substantially from the training distribution' is unsupported by any quantitative metrics (success rates, collision rates, number of scenarios, or failure modes), rendering the central empirical claim unverifiable from the presented material.

    Authors: We agree that the abstract's generalization claim requires quantitative backing to be verifiable. The current manuscript relies on video demonstrations of closed-loop behavior, but these are insufficient on their own. In the revision we will add a results table with success rates, collision rates, number of scenarios tested, and categorized failure modes for both in-distribution urban and OOD highway/unseen-urban cases, directly supporting the abstract statement. revision: yes

  2. Referee: Evaluation / closed-loop OOD experiments: the training and test distributions are both generated by the identical 2D reactive-agent simulator; the manuscript provides no analysis or ablation showing that the induced shifts (lacking tire models, latency, sensor noise, and high-speed interaction dynamics) are representative of genuine highway or urban OOD regimes, which is load-bearing for the generalization attribution to BEV + flow matching.

    Authors: We concur that all data comes from the same 2D simulator and that the manuscript currently lacks explicit analysis of how the induced shifts relate to real-world OOD regimes. The OOD tests do introduce substantive differences in road topology, lane count, and speed range relative to the Parma urban training set. We will add a limitations subsection discussing the simulator's simplifications (no tire models, latency, or sensor noise) and include any feasible ablations that isolate the role of the BEV representation. Full validation against high-fidelity dynamics lies outside the present scope and is noted as future work. revision: partial

Circularity Check

0 steps flagged

No circularity detected; results are empirical from simulator training and closed-loop testing

full rationale

The paper presents an empirical ML approach: a flow-matching model is trained on BEV rasters from a 2D Parma simulator and evaluated in closed-loop on held-out and OOD scenarios. No derivation chain, equations, or self-citations are shown that reduce any claimed prediction or generalization result to a fitted parameter or input by construction. The generalization claim rests on observed simulation performance rather than any self-referential definition or uniqueness theorem imported from prior author work. This is a standard non-circular empirical finding.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that BEV raster input combined with flow matching enables graceful degradation under distribution shift, as stated in the abstract. No free parameters or invented entities are explicitly introduced in the provided text.

axioms (1)
  • domain assumption Training exclusively on 2D traffic simulator data with reactive agents produces policies that generalize to markedly different environments such as multi-lane highways
    The paper trains only on urban simulator data and claims reliable generalization to OOD scenarios.

pith-pipeline@v0.9.1-grok · 5765 in / 1351 out tokens · 38879 ms · 2026-06-30T20:42:24.072610+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

  1. [1]

    Caesar et al

    H. Caesar et al. Nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. InCVPR ADP3 workshop, 2021

    work page 2021

  2. [2]

    Pseudo-simulation for autonomous driving

    Wei Cao et al. Pseudo-simulation for autonomous driving. InConference on Robot Learning (CoRL), 2025

    work page 2025

  3. [3]

    From Simulation to Real World Maneuver Execution using Deep Reinforcement Learning

    Alessandro Paolo Capasso, Giulio Bacchiani, and Alberto Broggi. From Simulation to Real World Maneuver Execution using Deep Reinforcement Learning. In2020 IEEE Intelligent Vehicles Symposium (IV), pages 1570–1575, 2020

    work page 2020

  4. [4]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi et al. Diffusion policy: Visuomotor policy learning via action diffusion. InRobotics: Science and Systems XIX, 2023

    work page 2023

  5. [5]

    Nvidia rtx a6000 graphics card

    NVIDIA Corporation. Nvidia rtx a6000 graphics card. https://www.nvidia.com/en- us/products/workstations/rtx-a6000/, 2020

    work page 2020

  6. [6]

    Craig Coulter

    R. Craig Coulter. Implementation of the pure pursuit path tracking algorithm. Technical report, Carnegie-Mellon University, The Robotics Institute, 1992

    work page 1992

  7. [7]

    Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks

    Henggang Cui et al. Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks. In2019 International Conference on Robotics and Automation (ICRA), pages 2090–2096, 2019

    work page 2090

  8. [8]

    Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

    Patrick Esser et al. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. InForty-First International Conference on Machine Learning, 2024

    work page 2024

  9. [9]

    Large Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset

    Scott Ettinger et al. Large Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset. In2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9690–9699, 2021

    work page 2021

  10. [10]

    G-PECNet: Towards a Generalizable Pedestrian Trajectory Prediction System

    Aryan Garg and Renu Rameshan. G-PECNet: Towards a Generalizable Pedestrian Trajectory Prediction System. InThe Second Tiny Papers Track at ICLR 2024, 2024

    work page 2024

  11. [11]

    Denoising Diffusion Probabilistic Models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, volume 33, pages 6840–6851, 2020

    work page 2020

  12. [12]

    Planning with Diffusion for Flexible Behavior Synthesis

    Michael Janner et al. Planning with Diffusion for Flexible Behavior Synthesis. InProceedings of the 39th International Conference on Machine Learning, pages 9902–9915. PMLR, 2022

    work page 2022

  13. [13]

    DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving

    Bencheng Liao et al. DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 12037–12047, 2025

    work page 2025

  14. [14]

    Flow Matching for Generative Modeling

    Yaron Lipman et al. Flow Matching for Generative Modeling. InThe Eleventh International Conference on Learning Representations, 2022

    work page 2022

  15. [15]

    Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q. Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow Matching Guide and Code, December 2024. 15

    work page 2024

  16. [16]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. InInternational Conference on Learning Representations, 2018

    work page 2018

  17. [17]

    Springer US, 2012

    Rajesh Rajamani.Vehicle Dynamics and Control. Springer US, 2012

    work page 2012

  18. [18]

    High-Resolution Image Synthesis with Latent Diffusion Models

    Robin Rombach et al. High-Resolution Image Synthesis with Latent Diffusion Models. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2022

    work page 2022

  19. [19]

    U-Net: Convolutional Networks for Biomedical Image Segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. InMedical Image Computing and Computer-Assisted Inter- vention – MICCAI 2015, pages 234–241. Springer International Publishing, 2015

    work page 2015

  20. [20]

    Congested traffic states in empirical observations and microscopic simulations.Phys

    Martin Treiber, Ansgar Hennecke, and Dirk Helbing. Congested traffic states in empirical observations and microscopic simulations.Phys. Rev. E, 62:1805–1824, 2000

    work page 2000

  21. [21]

    FlowDrive: Moderated flow matching with data balancing for trajectory planning, 2025

    Lingguang Wang et al. FlowDrive: Moderated flow matching with data balancing for trajectory planning, 2025

    work page 2025

  22. [22]

    GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Gener- ation in End-to-End Autonomous Driving

    Zebin Xing et al. GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Gener- ation in End-to-End Autonomous Driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1602–1611, 2025

    work page 2025

  23. [23]

    Diffusion-ES: Gradient-Free Planning with Diffusion for Autonomous and Instruction-Guided Driving

    Brian Yang et al. Diffusion-ES: Gradient-Free Planning with Diffusion for Autonomous and Instruction-Guided Driving. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15342–15353, 2024. 16

    work page 2024