{"id":"82cab19a-80c0-406c-a97c-96cfe74a4d61","arxiv_id":"2505.12327","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The authors mix normal and adversarially biased diffusion predictions under expected cost, and report a closed-loop score of 86.6 versus 83.5 for the best baseline in three adversarial driving scenarios.","lead":"Self-driving car planners usually underestimate rare dangerous moves like jaywalking. This paper adds a diffusion model that imagines adversarial moves, then scores plans against both normal and adversarial predictions, yielding robustness without overcautious driving.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison confounds adversarial input distribution with risk measure: CVaR uses only normal predictions, so the reported gain may come from adversarial biasing, not the mixture-expected-cost rule.","rationale":"The reader identified the realism and representativeness of adversarially biased predictions as the weakest assumption. That is a legitimate concern, but the more directly load-bearing issue for the central claim is experimental: the comparison against risk-sensitive baselines varies both the prediction distribution and the risk aggregator simultaneously. Since the abstract and Section IV-D attribute the benefit to computing expected cost under a mixture, a missing control that applies CVaR or worst-case risk to the same adversarially biased samples is a concrete, testable hole. If that control matches MAD, the paper's proposed mechanism is not validated; if MAD still wins, the concern is resolved. This does not require rejecting the paper; it strengthens the need for the conditional verdict the reader already assigned.","tokens_in":18162,"tokens_out":5030,"duration_ms":58717,"concrete_test":"Rerun the CVaR baseline with r=0.5 using the exact same per-candidate-plan sample set that MAD uses: eight adversarially biased diffusion predictions and two unbiased normal predictions, with the same biasing loss, λ, and clipping. Also run the worst-case baseline over this same sample set. If either baseline matches or exceeds MAD's overall 86.6 CLS, the claim that the mixture-expected-cost aggregator is responsible for the improvement is unsupported. If MAD remains best across all three benchmarks, the concern is mitigated. Report results over multiple seeds and with error bars.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-C states that all baselines besides constant velocity use only unbiased diffusion predictions from the normal behavior distribution, whereas MAD evaluates expected cost over two normal and eight adversarial biased samples per candidate plan (Section III-F, wb=0.8). Therefore the headline 18.8% error-rate reduction over CVaR conflates two separate changes: (i) supplying the planner with adversarial samples generated specifically for each candidate plan, and (ii) aggregating those samples by expected cost under a mixture rather than by a risk measure such as CVaR. The paper's central claim is specifically about (ii), namely that expected cost under a mixture avoids the over-conservatism of risk-sensitive planners. The experiments never include a CVaR or worst-case baseline that consumes the same biased adversarial distribution or the same mixture of adversarial and normal samples. If CVaR over the identical eight-adversarial-two-normal sample mixture matches or beats MAD, then the proposed mixture-expected-cost mechanism is not what drives the reported robustness. The evidence for 'not overly conservative' is also narrow: the only normal-behavior sub-benchmark, slow jaywalking, is near-saturated (EC already scores 100 in the single-agent case), so MAD's advantage is concentrated in adversarial sub-benchmarks and does not demonstrate reduced conservatism in ordinary driving.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a robust planning method for autonomous driving, called mixed adversarial diffusion predictions (MAD). The authors train a diffusion motion predictor on normal driving data, then at test time bias the predictor to generate adversarial agent trajectories that are close to each candidate ego plan, and finally score candidate plans by expected cost under a mixture of normal and adversarial predictions. The method is evaluated in closed-loop NuPlan simulation on three constructed benchmarks: single-agent jaywalking, multi-agent jaywalking, and red light violation. The paper reports that MAD achieves the best overall closed-loop score (86.6 CLS), corresponding to an 18.8% error-rate reduction over the second-best baseline CVaR (83.5 CLS), and argues that this demonstrates robustness to adversarial behaviors without the over-conservatism of risk-sensitive planners.","tokens_in":18398,"tokens_out":5247,"duration_ms":52953,"significance":"If the claims are substantiated, the contribution is conceptually appealing: a test-time-only modification of a diffusion-based motion predictor, combined with a mixture expected-cost objective, that avoids offline adversarial scenario collection and hard safety constraints. The paper is clearly written, the method is well specified, and the choice of PDM-Closed as a base planner allows the effect of the prediction distribution to be isolated from planner architecture. However, the present experimental evidence does not fully support the central mechanism claim. The baselines consume a different input distribution than MAD, the mixture weight wb is selected on one of the evaluation benchmarks, and the 'not overly conservative' claim rests on near-saturated sub-benchmarks. With additional controlled ablations and statistical reporting, the contribution would be solid and of interest to the robotics and autonomous-driving communities.","major_comments":[{"comment":"The baseline comparison conflates the adversarial input distribution with the mixture-expected-cost rule. Section IV-C states that all baselines except constant velocity use only unbiased diffusion predictions from the normal behavior distribution, whereas MAD evaluates each candidate plan over two normal and eight adversarial biased samples (Section III-F, wb=0.8). The reported 18.8% error-rate reduction over CVaR therefore does not isolate the paper's central mechanism, which is the expected-cost aggregation under a mixture; the gain could be driven entirely by the presence of plan-specific adversarial samples. Please add a CVaR (or worst-case) baseline evaluated on the same biased adversarial samples, and an expected-cost baseline over the same eight-adversarial/two-normal sample mixture, to disentangle these effects.","section":"Section IV-C and Table I"},{"comment":"The mixture weight wb is selected by an ablation on the single-agent jaywalking benchmark (Table III), and the same benchmark is part of the headline overall comparison in Table I. This selection-on-evaluation creates a risk of overfitting the hyperparameter to that scenario, so the reported overall improvement may not generalize. Please report wb chosen on a separate validation set, or provide results for the full range of wb across all three benchmarks, so that the influence of this free parameter is transparent.","section":"Section IV-H and Table III"},{"comment":"The claim that MAD avoids over-conservatism is only weakly supported because the slow-jaywalker sub-benchmark is saturated. In the single-agent case, EC, CVaR, and Col-P 0.1 already achieve 100 CLS on the slow sub-benchmark, and EC achieves 100 CLS in the multi-agent slow case as well. Meanwhile, on the fast-jaywalker sub-benchmark, MAD (74.2 single-agent, 76.8 multi-agent) is not the best method, losing to WC (91.9 single-agent) and Col-P 0 (78.2 multi-agent). The evidence indicates a robustness/conservatism trade-off rather than a strict improvement in 'not overly conservative' behavior; additional non-saturated normal-behavior scenarios are needed to support the central claim.","section":"Section IV-B and Table II"},{"comment":"The adversarial distribution is generated by an L1-distance loss between the closest agent's predicted trajectory and the candidate ego plan, with no validation that the biased samples are physically plausible or representative of real jaywalking and red-light violations. The diffusion prior is the only realism constraint, and the paper provides no qualitative examples, kinematic-feasibility check, or comparison against recorded adversarial trajectories. This is load-bearing because if the biased samples are unrealistic, the planner's extra caution is spent on synthetic threats; please add such validation or a sensitivity analysis demonstrating that the biased predictions are plausible.","section":"Section III-C, Eq. (7)"},{"comment":"The experimental section reports no error bars, number of seeds, or number of scenarios per benchmark. With constructed benchmarks and one-decimal CLS scores, differences such as the 10.4% single-agent error-rate reduction need to be assessed for statistical significance. Please report means and variances over multiple seeds, or at minimum the number of scenarios per benchmark, so that the robustness of the claimed improvements can be evaluated.","section":"Section IV-A and Tables I-II"}],"minor_comments":[{"comment":"There is a typo in the sentence 'By computing expected cost using both normal and adverarial agent behaviors'; 'adverarial' should be 'adversarial'.","section":"Section III-D"},{"comment":"The term 'unbiased distribution' is used to describe the trained diffusion model; since the training data is NuPlan driving logs, the distribution is 'normal' relative to that dataset, but calling it 'unbiased' may overstate its neutrality. Consider using 'normal behavior distribution' throughout for clarity.","section":"Section III-C"},{"comment":"The paper states that the planner considers 10 sampled predictions per candidate plan (two normal, eight adversarial, wb=0.8), but it does not report the computational cost of generating eight biased samples per plan per step; a brief runtime comparison against the baselines would help assess practical deployability.","section":"Section III-F"},{"comment":"In the ablation caption, the sentence 'all methods that use a mixture of normal and adversarial agent behaviors outperform the methods that use only normal agent predictions (EC) or only adversarial predictions' is not fully supported by the table, since wb=1.0 (only adversarial) is worse than EC; the text should clarify that the claim applies to mixture weights strictly between 0 and 1.","section":"Section IV-H"},{"comment":"Reference [13] contains a typo in the title: 'validatio' should be 'validation'.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the manuscript is generally well written, but the experimental evidence does not yet isolate the proposed mixture-expected-cost mechanism from the effect of simply supplying adversarial samples, and the hyperparameter wb is selected on the evaluation benchmark. I recommend major revision; the control experiments and statistical reporting requested in the major comments are within scope and should resolve the concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: MAD is a simple, sensible idea—train a standard diffusion motion predictor, bias it at test time to produce adversarial trajectories for each candidate plan, then score plans by expected cost under a mixture of normal and adversarial predictions. The writing is clear and the direction is plausible. The ablation showing that intermediate mixture weights beat both normal-only and adversarial-only is a good sanity check.\n\nThe real soft spot is the baseline setup. Section IV-C says all baselines except constant velocity use only unbiased diffusion predictions from the normal behavior distribution. CVaR, worst-case, and collision-probability baselines never see the adversarial samples that MAD generates. So the reported 18.8% error-rate reduction over CVaR conflates two things: giving the planner adversarial samples in the first place, and aggregating those samples by expected cost under a mixture rather than by a risk measure. The central claim is about the second, but the experiments don't isolate it. You would need CVaR and worst-case evaluated on the same two-normal-plus-eight-adversarial sample mixture to back the claim that mixture-expected-cost is what avoids over-conservatism.\n\nOther weaknesses are more conventional but still matter. The 'not overly conservative' evidence is thin: on slow jaywalking, expected-cost already hits 100 in single-agent, so the normal-behavior sub-benchmark is saturated. The hyperparameters wb and lambda were tuned on the same single-agent benchmark used as a headline result. There are no error bars, seeds, or scenario counts. The adversarial biasing loss in Eq. (7), minimizing L1 distance to the closest agent, is a heuristic with no validation that the biased samples are realistic or representative of real jaywalking and red-light violations. No code or data is provided, so the exact numbers are hard to trust.\n\nNone of this kills the idea. The mixture approach is worth pursuing, and the paper is honest about what it does and does not compare against. It just needs a cleaner experimental comparison to make the claimed mechanism believable.\n\nBottom line: this deserves a serious referee. I'd send it to review, but ask for a CVaR/worst-case baseline on the same adversarial mixture, proper statistics across seeds, and ideally code or data. If the authors can show that mixture-expected-cost beats risk measures when everyone sees the same predictions, this becomes a useful result for the autonomous-driving planning community.","headline":"A clean mixture-of-predictions idea for robust driving planning, but the experiments confound the adversarial input distribution with the risk measure, so the headline gain over CVaR is not yet pinned to the claimed mechanism.","tokens_in":18911,"tokens_out":3200,"would_cite":true,"duration_ms":35171,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that expected-cost planning under a mixture of normal and collision-biased diffusion predictions makes autonomous driving robust to rare adversarial behaviors without the over-conservatism of risk-sensitive planners.","keywords":["autonomous driving","robust planning","diffusion models","motion prediction","adversarial prediction","risk-sensitive planning","closed-loop evaluation","jaywalking scenarios"],"falsifier":"Run MAD and a normal-prediction expected-cost planner on logged real-world jaywalking and red-light-violation events; if the normal-only planner matches or exceeds MAD's closed-loop score on those logs, the reported benefit is an artifact of the synthetic biased samples rather than genuine robustness.","tokens_in":17947,"feed_emoji":"🚗","tokens_out":9012,"duration_ms":85697,"temperature":0.7,"pith_summary":"The paper tries to establish that an autonomous-driving planner can be made robust to rare adversarial behaviors such as jaywalking pedestrians and red-light runners without becoming overly cautious in normal traffic. Its method trains a diffusion model to predict normal agent motion, then at test time biases the same model to generate predictions that are likely to collide with the candidate ego plan. Candidate plans are scored by expected cost under a mixture of the normal and biased prediction distributions. The authors report that this mixture planner outperforms risk-sensitive and safety-constrained baselines in simulated adversarial scenarios, cutting the error rate by 18.8 percent relative to the best baseline. The value of the claim is that a simple test-time mechanism, with no offline adversarial scenario collection and no hard safety constraints, can deliver robustness to out-of-distribution agent behaviors.","feed_headline":"Mixed normal and adversarial predictions beat conservative planners","feed_subtitle":"Expected cost over a normal-plus-adversarial mixture cut closed-loop errors by 18.8 percent.","key_machinery":"The load-bearing object is the mixture distribution $p_{\\mathrm{mix}}(y^a_t \\mid s_t, y^{\\mathrm{ego}}_t) = (1-w_b)p_\\theta(y^a_t \\mid s_t) + w_b p_{\\theta,b}(y^a_t \\mid s_t, y^{\\mathrm{ego}}_t)$, where $p_\\theta$ is the trained diffusion predictor for normal behavior and $p_{\\theta,b}$ is the test-time biased distribution. The bias is implemented by adding a score term $\\lambda \\nabla_{y^a_t} \\mathcal{L}(D_\\theta(y^a_t;s_t,\\sigma), y^{\\mathrm{ego}}_t)$ to the diffusion sampling ODE, with $\\mathcal{L}$ the average $\\ell^1$ distance between the candidate plan and the closest predicted agent trajectory. This makes the adversarial samples plan-specific and collision-seeking, while the diffusion model's learned distribution keeps them realistic; expected cost under the mixture then places nonzero weight on both behavior types. An ablation shows the mixture weight matters: $w_b=0.8$ yields the best score, while $w_b=1.0$ (adversarial predictions only) performs poorly.","core_discovery":"The paper's central claim is that a planner can be made robust to rare adversarial agent behaviors at test time without training on offline adversarial scenarios or imposing hard safety constraints. The proposed method, MAD (mixed adversarial diffusion predictions), trains a diffusion model to produce an unbiased distribution of normal agent motions, then biases the same model during inference toward trajectories that are likely to collide with the candidate plan. Plans are scored by Monte-Carlo expected cost under a mixture of the normal and biased distributions, with mixture weight $w_b$ controlling how much probability mass is placed on adversarial behavior. The authors report a closed-loop score of 86.6 across single-agent jaywalking, multi-agent jaywalking, and red-light violation benchmarks, an 18.8 percent error-rate reduction over the second-best baseline (CVaR at 83.5), and show through ablations that mixtures outperform both normal-only and adversarial-only planning.","pith_inferences":["If real logged adversarial trajectories were available, a natural next test would be whether the biased diffusion samples match those logs in acceleration, timing, and route choice; the paper's experiments use synthetic adversarial scenarios, so the realism of the biased distribution remains an open question.","The bias objective targets the single closest agent via average L1 distance, which is a proxy for collision; a planner-aware objective that optimizes actual collision margin or accounts for multiple simultaneous threats could behave differently in dense scenes, though it would likely require more computation.","Because the mixture distribution is defined per candidate plan, the method's cost scales with the number of plans times the number of diffusion samples; sharing adversarial samples across similar plans or distilling the biased distribution could make the approach cheaper in practice."],"forward_implications":["Because adversarial predictions are generated at test time rather than from offline scenario collections, the planner can in principle respond to adversarial behaviors never seen during training.","The mixture weight $w_b$ acts as a tuning knob for conservatism: $w_b=0.8$ outperformed both the normal-only baseline ($w_b=0$) and the adversarial-only baseline ($w_b=1.0$), so operators can trade robustness against normal driving performance.","The mechanism is planner-agnostic in that it only changes how agent predictions are sampled and how candidate plans are scored; the underlying candidate-plan generation and cost function remain those of the base planner.","The same recipe transfers across different adversarial failure modes, since the results cover single-agent jaywalking, multi-agent jaywalking, and red-light violation without scenario-specific hyperparameters."],"supporting_citations":[{"why":"Supplies the diffusion motion predictor and the differentiable-guidance mechanism that MAD adapts to sample plan-specific adversarial predictions.","marker":"[4]"},{"why":"Supplies the base rule-based planner, its candidate plan generation and cost function, and the closed-loop score used for evaluation.","marker":"[119]"},{"why":"Supplies the simulator and training data in which the prediction model is trained and all planners are evaluated.","marker":"[122]"},{"why":"Defines the CVaR risk functional used by the strongest baseline that MAD claims to outperform.","marker":"[28]"},{"why":"Provides the diffusion sampling and denoiser framework, including the ODE solver and score parameterization, used to draw normal and biased predictions.","marker":"[84]"},{"why":"Provides the Monte Carlo estimator used to implement the CVaR baseline.","marker":"[124]"}],"fun_headline_variants":["Mixing normal and adversarial predictions cuts driving errors 18.8%","MAD planner: robust to rare agents without hard safety rules","Expected-cost diffusion mixture beats CVaR by 18.8%","Autonomous driving planner mixes adversarial predictions for robust behavior"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach depends on the test-time biased diffusion samples being realistic adversarial behaviors that match the real threats; if those samples are physically implausible or aimed at the wrong failure mode, the planner's extra caution is spent on synthetic hazards rather than real ones.","fun_headline_variants_meta":{"raw":{"variants":["Mixing normal and adversarial predictions cuts driving errors 18.8%","MAD planner: robust to rare agents without hard safety rules","Expected-cost diffusion mixture beats CVaR by 18.8%","Autonomous driving planner mixes adversarial predictions for robust behavior"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000939,"raw_usage":{"total_tokens":3977,"prompt_tokens":871,"completion_tokens":3106,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":3033}},"tokens_in":487,"tokens_out":3106,"duration_ms":21817,"temperature":1.0,"reasoning_tokens":3033,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:35:44.175200+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MAD and a normal-prediction expected-cost planner on logged real-world jaywalking and red-light-violation events; if the normal-only planner matches or exceeds MAD's closed-loop score on those logs, the reported benefit is an artifact of the synthetic biased samples rather than genuine robustness.","supporting_citations":[{"cited_title":"Parting with misconceptions about learning-based vehicle motion planning,","cited_arxiv_id":null,"evidence_quote":"Supplies the base rule-based planner, its candidate plan generation and cost function, and the closed-loop score used for evaluation."},{"cited_title":"Nuplan: A closed-loop ml- based planning benchmark for autonomous vehicles,","cited_arxiv_id":null,"evidence_quote":"Supplies the simulator and training data in which the prediction model is trained and all planners are evaluated."}],"review_version":1}