{"id":"63a0363d-f676-419f-98fa-e4dd6eb04753","arxiv_id":"2412.01234","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An end-to-end planner that jointly learns prediction, lane-selection decisions, and trajectory optimization with a differentiable optimizer reports lower collision rates and higher progress than imitation-based baselines on Waymo.","lead":"This paper adds a lane-selection decision step into an end-to-end autonomous driving planner, where a differentiable optimizer and a neural network are trained together on the Waymo dataset. The system chooses lanes that differ from human demonstrations and reports fewer collisions and more progress in closed-loop simulation than baseline planners.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eqs. (20)-(21) do not enforce one-hot lane decisions: for any b in [0,1] with sum at most 1 both penalties vanish, so the reported 100% constraint compliance cannot follow from the written formulation and the central decision-making claim rests on unverified optimizer/initialization behavior.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing weakness: the relaxed constraints in Eqs. (20)-(21) do not force a one-hot decision for b in [0,1]. I read the paper in good faith: the architecture is coherent, the closed-loop results are promising, and the ablation study does support the importance of learned initialization. The concern is not that relaxing binary variables is illegitimate; it is that the specific penalties written do not push relaxed variables toward 0 or 1 at all inside the unit simplex. Since the paper's stated novelty is integrating decision-making constraints into a differentiable optimization and preserving them throughout learning, this is the load-bearing link between the method and the reported decision-making capability. The proposed check would separate the empirical claim from the formal one: if corrected penalties change the Table III numbers, the published mechanism is doing no real work; if the numbers are unchanged, the paper should correct the equations and state that integrity is achieved empirically rather than by construction. I would not move the verdict to REJECT, because the flaw is localized and fixable, and because the empirical comparison could still hold after retraining. I also agree with the reader's secondary concerns about the missing reference route and missing error bars, but they do not displace the constraint-formulation issue. Thus the reader's CONDITIONAL verdict should stand.","tokens_in":19143,"tokens_out":5601,"duration_ms":54319,"concrete_test":"Run the differentiable optimizer in isolation, with the reported weights wbi=10 and weq=1000 and the same Gauss-Newton settings, initialized with b=(1/3,1/3,1/3) and the expert trajectory on the scenes used in Tables II and V. Record the raw converged b values before any rounding or thresholding. If max(b) remains below 0.99 or b collapses toward (0,0,0) for a substantial fraction of timesteps, the written penalties do not enforce integrality. Then retrain with corrected penalties wbi*max(0,b(1-b)) and weq*(sum(b)-1)^2 and re-evaluate Table III; if the collision rate and progress move outside the reported 4% and 72.77 m, the headline result depends on the current non-enforcing penalties.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's distinguishing contribution is that intrinsic decision-making constraints are \"preserved throughout the learning process.\" That claim is undermined by the soft penalties written in Eqs. (20)-(21). For any continuous b in [0,1], b(b-1) is non-positive, so max(0, b(b-1)) is identically zero. Likewise, for any b with b_-1 + b_0 + b_1 <= 1, the equality penalty max(0, sum(b)-1) is zero. Thus the relaxed feasible set contains all fractional mixtures with sum at most 1, including b=(0,0,0), and neither penalty creates any gradient toward integrality. The paper's assertion that these penalties \"ensure that the associated constraints (3d) and (3c) are still respected\" is therefore not justified by the equations as written. Table V's 100% compliance must arise from the learned initialization landing near binary values or from a post-hoc rounding/thresholding metric, not from the optimization formulation itself. This matters because lane selection is the claimed mechanism behind the safety and efficiency gains: if b remains fractional, the \"decision\" is a soft blend of lane costs, and the resulting trajectory is not a discrete lane-change decision. The missing reference route in the DIPP comparison is a secondary fairness concern, but the constraint flaw is the more load-bearing issue for the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an end-to-end trainable framework for autonomous vehicle planning that couples a transformer-based motion predictor with a differentiable nonlinear optimizer. The optimizer jointly solves lane-selection decisions and trajectory planning using costs for safety, traveling efficiency, riding comfort, and penalties intended to enforce the integer and equality constraints of the decision problem. The framework is trained on the Waymo Open Motion Dataset and evaluated in open-loop and closed-loop settings against Vanilla IL, IL+OPT, and DIPP. The authors report the lowest collision rate (4.50% open-loop, 4% closed-loop) and the largest closed-loop progress (72.77 m) among the compared methods, and they argue that the integrated decision-making capability enables planning that goes beyond imitation of expert demonstrations. Ablation studies examine the role of learned initialization and learnable cost weights.","tokens_in":106,"tokens_out":6466,"duration_ms":97263,"significance":"If the results are reproducible and the decision-constraint issue is resolved, the integration of discrete lane-selection decisions into differentiable optimization-guided learning is a useful step toward end-to-end planning that goes beyond imitation. The closed-loop progress improvement over DIPP (72.77 m vs 47.58 m) and the low collision rates are notable, and the closed-loop log-replay evaluation is appropriate for the claimed driving-performance gains. The ablation study for decision initialization is also informative. However, the paper's central structural claim that the decision-making constraints are 'preserved throughout the learning process' is not supported by the penalty formulation as written, and the open-loop 'consistent improvement' claim is contradicted by parts of Table II. These issues must be addressed before the results can be taken at face value.","major_comments":[{"comment":"The penalty terms do not enforce the claimed decision constraints. For any continuous b in [0,1], b(b-1) <= 0, so max(0, b(b-1)) is identically zero; for any b with b_-1 + b_0 + b_1 <= 1, max(0, sum(b)-1) is also zero. The relaxed feasible set therefore contains all fractional mixtures with sum at most 1, including b=(0,0,0), and neither penalty creates any gradient toward integrality. The statement in Section IV-A6 that constraints (3d) and (3c) are 'still respected' is not justified by these equations. Table V's 100% compliance must come from the learned initialization, a rounding/thresholding evaluation, or some other mechanism, not from the formulation as written. This is load-bearing because the paper's distinguishing contribution is that discrete decision-making constraints are preserved through learning. Please either replace the penalties with a form that actually penalizes fractional values (e.g., a penalty on b(1-b) plus a two-sided equality penalty), or demonstrate that the optimizer's solutions are binary by construction, and report the compliance metric accordingly.","section":"IV-A6, Eqs. (20)-(21), Table V"},{"comment":"The abstract's claim that open-loop outcomes 'consistently outperform' baselines is contradicted by Table II. At 3s and 5s the planning error is larger than DIPP (2.105 vs 1.715 m and 4.763 vs 4.630 m), and the off-route rate is higher (8.85% vs 7.68%). The statement in Section V-B that the method 'consistently outperforming both Vanilla IL and IL+OPT, while closely matching DIPP' is more accurate. Please revise the abstract and the summary of open-loop results to reflect the mixed outcome on planning accuracy, or supply an explicit weighting/aggregation that justifies the 'consistent' claim.","section":"Abstract and Section V-B, Table II"},{"comment":"The DIPP comparison may be unfair. The authors state that a notable modification in their processed data is the absence of a reference route, while DIPP [15] is a framework that plans with respect to a preconfigured reference route. If DIPP is evaluated without the reference route it was designed to use, its off-route and control-smoothness numbers in Table II and Fig. 3 could be degraded. Please clarify how the DIPP baseline obtains its reference line under the shared data pipeline, or include an additional DIPP variant with a reference route.","section":"Section V, first paragraph; Section V-B"},{"comment":"All results are reported as single point estimates without error bars, repeated-seed statistics, or significance tests. Collision rates such as 4%, 5%, and 9% and progress differences of several meters may be within scenario-level noise. Because the central support is empirical, please report means and standard deviations (or bootstrap confidence intervals) over multiple training runs and/or over scenario subsets.","section":"Section V, Tables II and III"}],"minor_comments":[{"comment":"There is an extra closing parenthesis in Eq. (21): 'max(0, b_-1 + b_0 + b_1 - 1))' should be 'max(0, b_-1 + b_0 + b_1 - 1)'.","section":"Eq. (21)"},{"comment":"The ablation row header 'No learnbale cost function' contains a typo; it should read 'No learnable cost function'.","section":"Table III"},{"comment":"The algorithm says gradients are computed with respect to θ and {ω_i}, but θ is the optimization variable; line 13 updates ϕ and {ω_i}. The gradient should be with respect to network parameters ϕ (and cost weights), not θ. Please correct this inconsistency.","section":"Algorithm 1, lines 12-13"},{"comment":"The description of dataset sampling is slightly confusing: 'randomly sample 10% (i.e., 100 data files)' followed by filtering 'resulting in a total of 88,123 frames' would benefit from stating whether the 88,123 frames are frames after filtering from the 100 files or from the full dataset.","section":"Section V, first paragraph"}],"recommendation":"major_revision","confidential_remarks":"The paper is closely related to the authors' own DIPP [15] and two-stage optimization [3]; Section IV-A says 'Building upon our previous work [3]' but the relationship to [15] could be clearer. For a journal submission, the novelty relative to [15] should be stated more sharply, and the fairness of the DIPP baseline should be addressed. These are editorial and novelty concerns, not allegations of misconduct."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, this is a real extension of DIPP: it adds a differentiable lane-selection variable to the optimization and trains the predictor with the optimizer in the loop. The closed-loop numbers are good — 4% collision vs 5% for DIPP, and 72.77 m progress vs 47.58 m. Second, the paper's central claim about preserving the discrete decision constraints is not backed by its own equations.\n\nThe new piece is the learned initialization for the decision variables plus the joint training. The ablation showing that no decision initialization drops optimizer convergence from 100% to 15% is informative. The closed-loop experiments are the right way to evaluate this kind of planner, and the comparisons are mostly fair in setup.\n\nThe soft spot is Eqs. (20)-(21). For any b in [0,1], b(b-1) is non-positive, so max(0, ...) is zero. For any sum at most 1, the equality penalty is also zero. So the penalties do not drive b toward one-hot; they only discourage b outside [0,1] or sums above 1. The 100% compliance in Table V therefore cannot come from the formulation as written. It must come from the optimizer landing near binary values because the lane costs are linear in b, or from post-processing. The paper should say this. As written, the claim that constraints (3c) and (3d) are 'still respected' is wrong. This matters because the discrete decision is the claimed mechanism for the safety and efficiency gains. If b remains fractional, the trajectory is a blend, not a decision.\n\nAlso worth flagging: the open-loop results do not show consistent superiority. Planning error at 3s and 5s is worse than DIPP (2.105 vs 1.715 m, 4.763 vs 4.630 m), and off-route is higher. The paper rationalizes this as the cost of optimizing decisions, which is fair, but 'consistently outperform' is an overstatement. No error bars or repeated-seed stats are reported, so we can't tell if the closed-loop margins are real. And removing the reference route from the data may hurt DIPP more than the other baselines.\n\nBottom line: the work is worth refereeing. The empirical story is interesting and the extension is genuine, but the constraint formulation needs a fix and the claims need to be toned down. If the authors fix Eqs. (20)-(21) (e.g., with a true penalty on fractional b or an explicit rounding step) and add variance information, this could be a solid paper. I'd send it out.","headline":"A genuine DIPP extension with promising closed-loop results, but the central claim about preserving one-hot decision constraints is not supported by the paper's own penalty equations.","tokens_in":20070,"tokens_out":3446,"would_cite":true,"duration_ms":30613,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that integrating lane-selection decisions into a differentiable optimization-guided learning framework produces safer and more efficient autonomous driving plans than imitation learning baselines.","keywords":["autonomous driving","end-to-end planning","differentiable optimization","decision-making","trajectory planning","lane selection","imitation learning","motion prediction"],"falsifier":"Run the trained optimizer on a diverse set of held-out scenes and record the optimized decision variables $b_\\alpha$: the claim that the formulation enforces a single lane choice is falsified if any converged solution mixes two lanes with strictly fractional values while the penalty terms are zero, or if removing the penalty weights $w_{\\mathrm{bi}}$ and $w_{\\mathrm{eq}}$ leaves all optimized decisions unchanged.","tokens_in":18959,"feed_emoji":"🚗","tokens_out":8703,"duration_ms":69080,"temperature":0.7,"pith_summary":"The paper aims to show that autonomous vehicles can be trained end-to-end to make lane-selection decisions and plan trajectories by optimizing explicit safety, efficiency, and comfort objectives rather than merely imitating human demonstrations. The authors build a differentiable optimizer that consumes predicted futures from a transformer-based predictor and jointly solves for the best lane choice and the best trajectory; the whole pipeline is trainable by backpropagation through the optimizer. If the central claim is right, end-to-end planning systems can go beyond expert trajectories, perform discretionary lane changes that avoid obstacles, improve progress, and retain interpretability through the optimization objectives. The reported results show a 4.50% open-loop collision rate versus 7.55% for the DIPP baseline and 72.77 m closed-loop progress versus 47.58 m for DIPP.","feed_headline":"Autonomous planner that chooses lanes beats imitation driving","feed_subtitle":"Differentiable optimization cuts open-loop collisions to 4.5% and nearly doubles closed-loop progress.","key_machinery":"The load-bearing machinery is a differentiable constrained nonlinear optimization problem solved with the Gauss-Newton algorithm inside a bilevel optimization loop: the inner loop solves for the ego vehicle's lane-selection decision variables $b_\\alpha(\\tau) \\in \\{0,1\\}$ (relaxed to continuous), states $x(\\tau)$, and controls $u(\\tau)$; the outer loop trains the transformer-based predictor, whose outputs initialize the decision and trajectory for the inner loop. The cost function combines position tracking, decision-dependent safety costs relative to leading and neighboring vehicles, traveling efficiency, comfort, and hinge-loss penalties for collisions and traffic-light violations. The discrete nature of the decision is handled by relaxing $b_\\alpha$ and adding penalty terms $\\ell_{\\mathrm{binary}}$ and $\\ell_{\\mathrm{equality}}$ with large weights, with the claimed effect of preserving the integer and equality constraints throughout learning. The entire pipeline is trained end-to-end, with the cost weights themselves also learnable, using a combination of prediction, score, decision, planning, and imitation losses.","core_discovery":"The paper's central discovery is that lane-selection decisions and trajectory plans can be jointly optimized in a differentiable constrained nonlinear program, and that training a transformer predictor end-to-end with this optimizer yields driving behavior that is safer, more efficient, and more comfortable than imitation-based planning. The optimization objectives explicitly encode tracking, longitudinal and lateral safety, traveling efficiency, riding comfort, and collision/traffic-light compliance, with the decision variable relaxed from binary to continuous but accompanied by penalty terms intended to enforce the one-hot and equality constraints. On the Waymo Open Motion Dataset, open-loop testing gives the method the lowest collision rate (4.50%) among the compared methods, and closed-loop testing gives the lowest collision rate (4%) and the largest progress (72.77 m) against baselines including DIPP. The paper also claims that the learned initialization of decisions and actions from the prediction module is essential for the optimizer to converge and for overall performance, as demonstrated by ablations.","pith_inferences":["The paper's reported 100% constraint compliance is likely not guaranteed by the penalty terms alone: for any continuous $b_\\alpha \\in [0,1]$ with $\\sum_\\alpha b_\\alpha \\le 1$, both $\\ell_{\\mathrm{binary}}$ and $\\ell_{\\mathrm{equality}}$ are zero, so the one-hot property depends on the optimizer and learned initialization landing on near-binary values rather than on the formulation.","An immediate testable extension is to inspect the optimized decision variables across many held-out scenes; if any converged solution mixes two lanes with comparable weights at zero penalty, the hard-constraint claim would be weakened.","The higher open-loop planning error at 3 s and 5 s relative to DIPP suggests the gains are concentrated in closed-loop re-planning; in settings where the planner is not re-run frequently, the benefit may be smaller.","The closed-loop evaluation replays recorded agent trajectories (non-reactive), so the collision-rate improvement may not fully reflect performance in fully interactive traffic where other vehicles respond to the ego vehicle's decisions."],"forward_implications":["End-to-end planners can be built without a predefined reference route, with lane selection emerging from the optimization objectives instead of from HD-map waypoints.","Joint training of prediction, decision-making, and trajectory planning through a differentiable optimizer can reduce closed-loop collision rates and increase travel progress relative to imitation-only baselines.","Providing learned initialization for both the decision and the control inputs is a critical design choice: the ablation shows convergence drops to 15% without decision initialization.","Because the cost weights are learnable, the trade-off between safety, efficiency, and comfort can be tuned from data during end-to-end training."],"supporting_citations":[{"why":"Supplies the differentiable integrated prediction-and-planning baseline (DIPP) with learnable cost function that this work extends by adding decision-making.","marker":"[15]"},{"why":"Provides the differentiable nonlinear optimization library used to implement the Gauss-Newton solver in the joint training.","marker":"[44]"},{"why":"Supplies the Waymo Open Motion Dataset used for training and both open-loop and closed-loop evaluation.","marker":"[46]"},{"why":"The authors' prior two-stage optimization work, whose integrated objective for decision-making and trajectory planning is generalized into the differentiable end-to-end framework.","marker":"[3]"}],"fun_headline_variants":["Differentiable decision planning beats imitation driving","Joint lane and trajectory optimization improves safety","End-to-end planner with integrated decisions cuts crashes","Optimizing decisions and paths enhances autonomous driving","Decision-aware optimization outperforms imitation in AV planning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that the penalty terms actually force the planner to settle on exactly one lane; in fact those penalties are zero for any fractional combination of lanes that does not exceed one lane in total, so the guarantee really comes from the optimizer happening to land on clean choices, not from the math of the penalties.","fun_headline_variants_meta":{"raw":{"variants":["Differentiable decision planning beats imitation driving","Joint lane and trajectory optimization improves safety","End-to-end planner with integrated decisions cuts crashes","Optimizing decisions and paths enhances autonomous driving","Decision-aware optimization outperforms imitation in AV planning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1474,"prompt_tokens":947,"completion_tokens":527,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":461}},"tokens_in":563,"tokens_out":527,"duration_ms":5809,"temperature":1.0,"reasoning_tokens":461,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:33:27.614339+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained optimizer on a diverse set of held-out scenes and record the optimized decision variables $b_\\alpha$: the claim that the formulation enforces a single lane choice is falsified if any converged solution mixes two lanes with strictly fractional values while the penalty terms are zero, or if removing the penalty weights $w_{\\mathrm{bi}}$ and $w_{\\mathrm{eq}}$ leaves all optimized decisions unchanged.","supporting_citations":[{"cited_title":"Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving,","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable integrated prediction-and-planning baseline (DIPP) with learnable cost function that this work extends by adding decision-making."},{"cited_title":"Theseus: A library for differentiable nonlinear optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the differentiable nonlinear optimization library used to implement the Gauss-Newton solver in the joint training."},{"cited_title":"Scalability in Perception for Autonomous Driving: Waymo Open Dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the Waymo Open Motion Dataset used for training and both open-loop and closed-loop evaluation."},{"cited_title":"Synergizing Decision Making and Trajectory Planning Using Two-Stage Optimization for Autonomous Vehicles","cited_arxiv_id":"2411.18974","evidence_quote":"The authors' prior two-stage optimization work, whose integrated objective for decision-making and trajectory planning is generalized into the differentiable end-to-end framework."}],"review_version":1}