{"id":"99b2643b-a80d-460a-a43e-c24c8d3b1a43","arxiv_id":"2504.19077","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A vision-only driving policy can be trained on-policy inside a learned world model simulator and deployed in a real ADAS, without hand-coded steering rules.","lead":"This paper trains a self-driving steering policy entirely from real driving videos, using two simulated worlds built from data instead of hand-written rules. The authors say the resulting policies keep lanes, change lanes, and were deployed in a real driver-assistance product for about two months.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Plan Model's action labels are never validated on the off-distribution states that on-policy training is designed to visit, so the claim of human-like driving without hand-coded rules rests on an ungrounded supervisor.","rationale":"I agree with the reader's weakest assumption. The paper's central contribution is an on-policy training loop whose supervision signal comes entirely from a learned Plan Model. The architecture is plausible, and the closed-loop MetaDrive results are encouraging evidence that the training loop can produce lane-keeping and lane-change behavior. However, the validity of the Plan Model's labels under distribution shift is the load-bearing assumption: if those labels are wrong, the policy is imitating simulator artifacts rather than human driving. The paper explicitly identifies the counterfactual problem for reprojective image generation in Section 3.1, but it never applies the same scrutiny to the counterfactual action labels produced by the Plan Model. The field evaluation does not resolve this because engagement percentage is a usage statistic, not an independent measure of driving competence or safety. I would therefore keep the reader's CONDITIONAL verdict: the paper is a credible systems demonstration, but the central claim that the policy learns human-like driving without hand-coded rules is not fully supported until the Plan Model's ground truth is validated on the off-distribution states that on-policy training deliberately visits. The proposed concrete test would directly probe that assumption with existing rollouts and an independent lane-feasibility check.","tokens_in":10573,"tokens_out":6545,"duration_ms":70155,"concrete_test":"Run the action policy-forced sequential rollouts from Section 4.6.3 on the 1,500 held-out segments, including the ±0.5 m lateral perturbation runs. At each timestep, extract the Plan Model's predicted 10 s trajectory, which is the label a_wp used in Eq. 7, and check it against an independent rule-based lane-feasibility criterion: no solid-lane-line crossings, no off-road excursions, and bounded curvature and lateral acceleration within the vehicle model envelope. If a material fraction, for example more than 5 percent, of predicted trajectories fails this feasibility check on the perturbed states, then the Plan Model is not a valid action ground truth for exactly the out-of-distribution states that on-policy training introduces.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.1 trains the policy to match actions a_wp derived from the Plan Model (Eq. 7), where the Plan Model is a Future Anchored World Model whose trajectories are trained on real human data with the future anchor F drawn from the same recorded segment (Sections 2.5, 2.6). The entire purpose of on-policy training is to expose the policy to states it would not see under the expert distribution; but at exactly those states the Plan Model is being asked to provide ground truth it was never validated on. Section 2.6 calls F a desirable goal state because the Plan Model shows recovery pressure, but desirability is not the same as reachability or safety: F is the original human's future state, not a goal selected for the counterfactual state the policy has reached. The paper acknowledges the counterfactual problem for reprojective simulation (Section 3.1), yet the same issue applies to the action ground truth: a_wp is a counterfactual label for a state no human ever drove. No experiment measures whether Plan Model trajectories are lane-consistent, obstacle-free, or human-approved on the perturbed states used in training, such as the ±0.5 m deviations and lane-change impulses. The MetaDrive unit tests (Table 2) show the final policy passes simple scenarios, but they do not audit the labels that produced it. The field data (Table 3) are engagement percentages without a baseline or safety metric, so they cannot detect a policy that imitates unsafe label artifacts. Thus the central claim that the policy learns safe, human-like driving without hand-coded rules depends on an untested property of the Plan Model.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a vision-only, end-to-end driving policy trained via on-policy interaction in data-driven simulators built from real human driving data. Two simulator variants are presented: a reprojective novel-view synthesis simulator and a learned Diffusion Transformer world model with future anchoring plus a Plan Model that supplies action labels. The policy is trained to imitate those Plan Model actions during simulator rollouts. The paper reports closed-loop MetaDrive unit tests in which on-policy-trained policies outperform an off-policy baseline on lane centering and lane change tasks, together with real-world openpilot ADAS deployment metrics expressed as engagement percentages.","tokens_in":10884,"tokens_out":5847,"duration_ms":59795,"significance":"The paper has a clear and credible core: the closed-loop unit tests in Table 2 (24/24 lane centering for both on-policy methods, 20/20 and 19/20 lane changes) are a clean demonstration that on-policy training in data-driven simulators changes behavior relative to off-policy imitation. The authors also deserve credit for an honest enumeration of reprojective simulation limitations in Section 3.1, for the information-bottleneck regularization idea in Section 5.2, and for reporting scaling trends of LPIPS with model and dataset size in Figure 5. If the Plan Model supervision can be validated on off-distribution states, the approach is a meaningful step toward scaling end-to-end driving from data. However, the real-world claim rests on weak field metrics, and the learned-supervisor loop is not audited; as written, the evidence is not commensurate with the strength of the abstract and conclusion.","major_comments":[{"comment":"The action labels \\hat{a}^{wp} used to train the policy are generated by the Plan Model, which is trained on the same human-driving data that also supplies the future anchors F; the policy is therefore trained to imitate a learned expert that is never validated on the off-distribution states that on-policy rollouts are designed to visit. Section 3.1 identifies the counterfactual problem for reprojected images, but the same counterfactual issue applies to the action labels: at a state displaced by the ±0.5 m noise model or during a lane-change impulse, no human ever drove the trajectory that the Plan Model proposes. No experiment audits whether Plan Model trajectories are lane-consistent, obstacle-free, or human-approved on those perturbed states. Please add such an audit, or an ablation using a filtered or otherwise validated supervisor; the 'no hand-coded driving rules' claim depends on the supervisor being a reliable expert rather than a simulator artifact.","section":"5.1, Eq. (7); 2.6"},{"comment":"The real-world ADAS claim is not supported by the reported metrics. Table 3 gives only engagement percentage (time and distance) for the two policies over roughly 40,000–47,000 trips, with no baseline, no error bars, no disengagement reasons, no safety events, and no comparison against a hand-coded or off-policy baseline on the same cohort. Because openpilot is a level-2 system, engagement percentage is affected by user trust, trip context, and system availability, not only by policy quality. The sentence 'both policies are capable of delivering meaningful driver assistance in real-world conditions' is not derivable from this table. Please report at least a matched baseline and disengagement or safety statistics, or soften the claim to usage acceptance rather than demonstrated capability.","section":"5.4, Table 3"},{"comment":"The lane-change test relies on an externally injected conditioning impulse both during training and at inference to trigger the maneuver. This is a hand-designed trigger, so the paper does not demonstrate that the policy learns the decision of when to change lanes; it demonstrates execution of a lane change once commanded. The abstract and conclusion should be qualified accordingly, and the paper should state explicitly that lane-change initiation remains externally specified.","section":"5.3.1"},{"comment":"The reported evaluation is limited to lateral control in relatively simple MetaDrive scenarios, and the paper states in Section 5.3 that longitudinal metrics are deferred to future work. This is a reasonable scope choice, but the claims about learning 'driving behavior' and 'useful ADAS products' should be tied to this lateral-only scope, especially given the production system in Section 5.3.3 still uses a classical ACC policy for longitudinal control.","section":"5.3, 5.4"}],"minor_comments":[{"comment":"The value of the hyperparameter \\alpha is missing; the text reads 'with a hyperparameter \\alpha = as described in Equation 5.' Please provide the value or a reference to where it is set.","section":"4.2.2, Eq. (5)"},{"comment":"The text says the next-frame prediction result is 'shown in left Figure 6' and the sequential rollout result is 'shown in right Figure 6,' but the caption associates the labels in the opposite way. Please correct the mismatch.","section":"4.6.1, Figure 6"},{"comment":"The claim that the information bottleneck limits the feature extractor to 'roughly 700 bits' would benefit from the explicit capacity calculation for the described Gaussian channel with additive white Gaussian noise; as written, the reader cannot reproduce the number.","section":"5.2"},{"comment":"The scaling plots show single-curve LPIPS values without error bars or multiple seeds; please indicate the variance across seeds or state that only one run was used.","section":"Figure 5"},{"comment":"The field deployment section would be easier to interpret if the cohort size, the definition of an 'engaged' trip, and the collection window were stated in the table caption rather than only in the text.","section":"5.4, Table 3"}],"recommendation":"major_revision","confidential_remarks":"I agree with the conditional assessment: the central training-loop idea is defensible and the simulation results are promising, but the real-world validation in Table 3 is far too thin for the strength of the claims, and the Plan Model supervision needs explicit off-distribution validation. I would support acceptance after the authors add those analyses or proportionally weaken the claims. No obvious novelty-disclosure issue; related prior work is cited."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is integration: a production ADAS steering policy trained on-policy inside a learned world model with future-anchored supervision, then deployed to a few hundred real users. The building blocks are known, but combining them with the Plan Model as action supervisor and showing closed-loop unit tests pass is a real engineering contribution.\n\nThe MetaDrive unit tests are the strongest part. Off-policy training fails (5/24 lane-center, 8/20 lane-change), while both on-policy methods pass (24/24 and 20/20, 19/20). That is concrete evidence the on-policy loop does what it claims. The paper also honestly lists the reprojective simulator's limitations (counterfactual problem, artifacts, shortcut learning) and uses an information bottleneck to address shortcut learning. The architecture is described in enough detail to reproduce, though no code or data are released.\n\nSoft spots, in proportion. First, the real-world claim rests on engagement percentages from a self-selected cohort with no baseline and no safety metrics. That is suggestive, not demonstrative, and the authors' phrasing ('meaningful driver assistance') is appropriately modest. Second, lane changes are triggered by a conditioning impulse added during training and again at inference. That is a hand-coded rule, which sits awkwardly beside the 'no hand-coded driving rules' claim. Third, the Plan Model circularity flagged in the stress-test is real: the policy imitates Plan Model actions, and the Plan Model was trained on the same data with future anchors from the same recordings. At the off-distribution states on-policy training visits, the Plan Model's labels are never audited. I do not think this is load-bearing, because the evaluation is behavioral (does the policy converge to lane center and complete lane changes?) rather than label-matching, and the policy could learn good behavior despite imperfect labels. But the absence of any check on Plan Model trajectory quality for perturbed states is a genuine gap.\n\nThis deserves peer review. It is a serious systems paper with a clear training-loop claim, decent closed-loop evidence, and a deployed real-world system. The field metrics should be treated as suggestive, and the hand-coded lane-change impulse should be acknowledged as a rule. For someone working on world models for control or end-to-end driving, this is worth reading and citing.","headline":"A credible systems demonstration that on-policy training in a learned world model can produce a deployable steering policy, with real but non-fatal weaknesses in the real-world evidence and a mild circularity in the action supervisor.","tokens_in":11499,"tokens_out":2033,"would_cite":true,"duration_ms":20364,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Learned world model trains a vision-only policy that steers a real car with no hand-coded rules.","keywords":["end-to-end driving","world model","on-policy learning","future anchoring","diffusion transformer","driving policy","lane keeping","ADAS"],"falsifier":"Generate a set of counterfactual driving states by running the World Model with forced lateral commands, record the Plan Model's predicted trajectory at each state, and have several human drivers produce reference trajectories from the same states in a fixed-base simulator; if the Plan Model's trajectories deviate from the human reference by more than 0.5 m in lateral position after one second on more than a small fraction of cases, the action labels that supervise the policy are not validated.","tokens_in":10366,"feed_emoji":"🚗","tokens_out":13085,"duration_ms":118224,"temperature":0.7,"pith_summary":"Most self-driving systems are built from hand-coded perception outputs and engineered driving rules. This paper proposes an alternative: train a vision-only driving policy entirely inside data-driven simulators that reproduce real driving scenes, then deploy it in a real car. The authors describe two simulators—one that reprojects camera images from depth maps, and one based on a learned diffusion world model—and use a Future Anchored Plan Model to generate action labels for the policy during on-policy training. In closed-loop tests, the on-policy policies pass nearly all lane-centering and lane-change scenarios, while an off-policy baseline fails most; the same policies then provided steering assistance in real-world driving through an open-source ADAS platform. The paper's claim is that end-to-end learning inside a learned world model is a viable route to production driver assistance without hand-coded rules.","feed_headline":"World-model sim trains a car to drive with no coded rules","feed_subtitle":"End-to-end policies trained inside data-driven simulators pass lane tests and steer an open-source ADAS.","key_machinery":"The central machinery is the Future Anchored World Model operating as a Plan Model: a latent diffusion transformer trained with a rectified flow objective that is conditioned on a future state anchor $F=(f_s, f_e)$ and on a history of images and poses. A Plan Head on the same model predicts a multi-hypothesis 10-second future trajectory, which the Vehicle Model inverts into action labels $\\hat{a}^{\\text{wp}}$ used to supervise the driving policy. The future anchor supplies 'recovery pressure' so that, even when the simulator has drifted out of the training distribution, the predicted trajectory converges to a goal state rather than compounding errors. Conditioning on poses rather than actions lets the World Model ignore the Vehicle Model, so vehicle-dynamics randomization can be applied without retraining the World Model, which is what makes policies trained in simulation transfer to the real vehicle.","core_discovery":"The paper's central claim is that a vision-only driving policy can be trained on-policy in a data-driven simulator and learn meaningful driving behavior without any hand-coded rules. The key mechanism is a Future Anchored World Model that acts as both environment and supervisor: conditioned on a future state anchor, it predicts the next image and a future trajectory that converges to that anchor, and the trajectory is converted into action labels through an invertible Vehicle Model. The policy is trained to imitate those labels while interacting with the simulator, so it experiences and recovers from its own errors rather than only imitating recorded demonstrations. The paper reports that policies trained in this way pass closed-loop lane-keeping and lane-change unit tests and, when deployed in a real-world ADAS, engaged for over a quarter of total driving time. To the authors' knowledge this is the first demonstration of an end-to-end policy trained without handcrafted features being used in a real-world driver assistance system.","pith_inferences":["The same 'future anchoring + learned simulator + on-policy imitation' recipe could apply to other sequential-decision domains where a goal state is available, such as robot manipulation or pedestrian navigation, not just driving.","The paper never ablates the Plan Model against the simpler baseline of using the recorded human action at each simulation state; doing so would isolate how much of the trained policy's success comes from the future-anchored labels rather than from the on-policy interaction itself.","If the Plan Model's labels are valid, then the bottleneck to safer autonomous driving shifts from policy design to the fidelity of the world model and the breadth of the human data, implying that data collection and simulation fidelity, not rule engineering, are the scaling axes.","A direct test of the world-model path would be to train the policy with the reprojective simulator's labels but the world model's images (or vice versa), separating the effect of image realism from the effect of label quality."],"forward_implications":["If the central claim holds, an end-to-end driving policy can be trained for new vehicles and new behaviors simply by collecting more human driving data and scaling the world model, without re-engineering perception outputs or control rules.","The future-anchoring mechanism provides a general supervision signal for counterfactual states, so the policy can recover from errors that never appear in the original demonstrations.","The same training loop should extend to longitudinal control: the paper states that the methods generalize to acceleration and braking policies, which would make the entire driving stack learned end-to-end.","The world-model simulator's image quality and pose accuracy improve with dataset and model size, so policies trained within it should keep improving as the simulator scales, unlike hand-built simulators that require manual scene construction."],"supporting_citations":[{"why":"Provides the distributed actor-learner architecture used to train the driving policy on-policy in the simulator.","marker":"[7]"},{"why":"Introduces the future-anchoring conditioning that gives the Plan Model its recovery pressure toward a goal state.","marker":"[2]"},{"why":"Defines the rectified flow training objective used for the diffusion-based World Model.","marker":"[16]"},{"why":"Supplies the diffusion transformer architecture that the World Model adapts to three-dimensional video patches.","marker":"[19]"},{"why":"Provides the pretrained VAE tokenizer that compresses camera images into latent space for the World Model.","marker":"[23]"},{"why":"Source of the noise-level augmentation that stabilizes sequential sampling against autoregressive drift.","marker":"[29]"},{"why":"Provides the reprojective novel-view synthesis method used for the alternative data-driven simulator.","marker":"[27]"},{"why":"Defines the multi-hypothesis planning loss used to train both the Plan Head and the driving policy's trajectory output.","marker":"[5]"}],"fun_headline_variants":["World model teaches a car to drive with zero coded rules","End-to-end driving policy trained in a data-driven sim, no rules","Car learns to drive from a world model, no hand-coded rules","World-model sim trains a vision-only driver, no rules"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Plan Model's predicted future trajectories are used as ground-truth actions for training, but those predictions are never verified against a real driver or a safety metric in the off-distribution states the simulator can create.","fun_headline_variants_meta":{"raw":{"variants":["World model teaches a car to drive with zero coded rules","End-to-end driving policy trained in a data-driven sim, no rules","Car learns to drive from a world model, no hand-coded rules","World-model sim trains a vision-only driver, no rules"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00044,"raw_usage":{"total_tokens":2180,"prompt_tokens":844,"completion_tokens":1336,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":460,"completion_tokens_details":{"reasoning_tokens":1263}},"tokens_in":460,"tokens_out":1336,"duration_ms":9640,"temperature":1.0,"reasoning_tokens":1263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T06:02:52.882560+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a set of counterfactual driving states by running the World Model with forced lateral commands, record the Plan Model's predicted trajectory at each state, and have several human drivers produce reference trajectories from the same states in a fixed-base simulator; if the Plan Model's trajectories deviate from the human reference by more than 0.5 m in lateral position after one second on more than a small fraction of cases, the action labels that supervise the policy are not validated.","supporting_citations":[{"cited_title":"Impala: Scalable dis- tributed deep-rl with importance weighted actor-learner ar- chitectures","cited_arxiv_id":null,"evidence_quote":"Provides the distributed actor-learner architecture used to train the driving policy on-policy in the simulator."},{"cited_title":"Video pretraining (vpt): Learning to act by watching unlabeled online videos","cited_arxiv_id":null,"evidence_quote":"Introduces the future-anchoring conditioning that gives the Plan Model its recovery pressure toward a goal state."},{"cited_title":"Diffusion models are real-time game engines","cited_arxiv_id":null,"evidence_quote":"Source of the noise-level augmentation that stabilizes sequential sampling against autoregressive drift."},{"cited_title":"View morphing","cited_arxiv_id":null,"evidence_quote":"Provides the reprojective novel-view synthesis method used for the alternative data-driven simulator."},{"cited_title":"Multimodal trajectory predictions for autonomous driving using deep convolutional networks","cited_arxiv_id":null,"evidence_quote":"Defines the multi-hypothesis planning loss used to train both the Plan Head and the driving policy's trajectory output."}],"review_version":1}