{"id":"54500947-cf6e-4dac-a7b8-f27e8bd1adc0","arxiv_id":"2507.09836","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Multi-residual Mixture of Experts Learning adds a learned residual to a gated mixture of nominal driving policies and reports additional 4%-9% emission reductions in simulated eco-driving at signalized intersections.","lead":"This paper introduces a learning framework for self-driving cars that combines known tricks: a mixture of simple driving policies plus a learned correction. In simulated eco-driving tests across three U.S. cities, it reports 4%-9% extra emission reductions over the strongest comparison policy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No held-out scenario split is described; the 4%-9% advantage may be in-distribution memorization, not generalization.","rationale":"The reader's weakest assumption was simulator fidelity; I agree that is a genuine uncertainty, but I see a more direct and more easily settled threat: the evaluation protocol may not include any held-out traffic scenarios. The abstract and introduction frame MRMEL as improving generalization across diverse traffic scenarios, and Section IV-B defines policy generalization within a city. Yet Section VI-B/C never state that the evaluation scenarios are disjoint from training scenarios. If they are not disjoint, then even a perfect simulator would not support the generalization claim, and Table I would only show in-distribution fit. The lack of multiple seeds and code release compounds this, but the split is the load-bearing issue because it determines what Table I can mean. I am not claiming the authors actually trained and evaluated on identical scenarios; the manuscript simply omits the information, and no appendix or limitation statement acknowledges the omission. The correct disposition remains conditional, as the reader concluded, but the revision should now also require a clear held-out evaluation. I therefore keep the verdict unchanged. My agreement with the reader is partial: we share the view that the empirical evidence is under-supported, but I weight the missing train/test split more heavily than the simulator-fidelity question.","tokens_in":10787,"tokens_out":6003,"duration_ms":77184,"concrete_test":"Re-run MRMEL and all baselines with a random 80/20 intersection-level split per city, train on the 80% and evaluate only on the held-out 20%, repeated over at least five seeds. Report Table I emission improvements on held-out scenarios with mean and standard deviation. If the mean additional reduction over the strongest baseline drops below 4% or the 95% confidence interval includes zero, the central generalization claim does not survive; if it remains above 4% in all three cities, the no-split concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section VI-B describes training on the IntersectionZoo scenarios across roughly 1,670, 621, and 282 intersections in the three cities, but Section VI-C reports results 'across all scenarios' with no explicit statement of a train/validation/test split. Figure 3 even maps emission benefits over all intersections. If the same scenarios used for PPO training are also used to compute Table I, the headline 'additional 4%-9% reduction' is an in-distribution training score, and the paper's claimed policy generalization across diverse traffic scenarios is untested. This is the weakest link because the paper's central contribution is a gating mechanism that is supposed to generalize over a context space (Sections I, IV-A, IV-C), not merely fit training scenarios. The absence of seeds and released code (Section VI-C) prevents checking whether a split exists, and the manuscript nowhere states that evaluation scenarios were unseen at training time. The simulator-fidelity issue raised by the reader is real, but it is secondary: if the policy does not generalize to held-out intersections, the simulator's fidelity is irrelevant to the generalization claim. This concern is concretely checkable, unlike a broad debate about emission-model realism, and should be settled before the generalization claim is credited.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Multi-Residual Mixture of Experts Learning (MRMEL), a framework that extends residual reinforcement learning with a gated mixture of nominal policies conditioned on traffic-scenario context. The method is instantiated for cooperative eco-driving at signalized intersections using IntersectionZoo scenarios for Dallas–Fort Worth, Atlanta, and Salt Lake City, at 30% and 100% AV penetration. The paper reports that MRMEL reduces aggregate emissions by an additional 4%–9% over the strongest baseline in each setting and also improves intersection throughput.","tokens_in":10962,"tokens_out":4848,"duration_ms":55787,"significance":"MRMEL addresses a real and timely problem: extending residual RL from single-task settings to contextual, multi-agent Lagrangian control. The architecture is clearly motivated, and evaluating on thousands of real-world-data-derived intersection scenarios across three cities is more ambitious than most prior benchmark studies. The paper also compares against a reasonable set of baselines, including the previously published MRTL, which strengthens the empirical setup. The central weakness is that the evaluation protocol does not currently establish the generalization the method is designed for: there is no stated train/validation/test split, no uncertainty quantification over seeds, and no cross-city policy-transfer test for the claimed algorithmic generalization across cities. If the authors add held-out evaluations and seed-level statistics, the contribution would be solid and likely useful to the Lagrangian control and contextual RL communities.","major_comments":[{"comment":"The central claim is policy generalization across traffic scenarios (Section IV-A), but the paper never states whether the scenarios used to produce Table I and Figure 3 were held out during PPO training. Section VI-B describes training on IntersectionZoo scenarios in each city, and Section VI-C reports results \"across all scenarios\" with no mention of a split. Because MRMEL's gating network and residual are trained on contextual features of the same intersections, the 4%–9% improvement could be an in-distribution training score rather than evidence of generalization. Please specify the split (e.g., by intersection or by scenario), and report evaluation metrics on held-out intersections or scenarios only. This is the load-bearing fix for the paper's main claim.","section":"Section VI-C, Table I; Section IV-A"},{"comment":"All numbers in Table I are point estimates with no standard deviations, number of seeds, or statistical significance tests. This matters because the RRL (Const. acc.) row shows an isolated 8.66% emission improvement for Atlanta at 100% penetration while the same method produces 0.00%–0.05% in nearly all other cells; this pattern is consistent with high variance or an anomalous run, and it makes the headline \"4%–9% additional reduction\" unverifiable. Please report mean ± std (or confidence intervals) over multiple random seeds for every method and setting, and state whether the difference between MRMEL and the strongest baseline is statistically significant.","section":"Table I; Section VI-C"},{"comment":"Section IV-A defines \"algorithmic generalization across cities\" and says MRMEL is designed to display it, but all experiments train and evaluate within the same city; there is no train-in-one-city/test-in-another experiment, and no analysis of how the learned gating transfers across city context distributions. As written, the paper demonstrates within-city scoring on the training distribution, not algorithmic generalization across cities. Please either add a cross-city transfer experiment or rephrase the claim to match the evidence.","section":"Section IV-A; Section VI"},{"comment":"The formal objective in Eq. (5) seeks a set of at most k policies with the best per-context return, but the implemented method learns a single policy per city whose gating mechanism selects among nominal experts. The relationship between the set-of-policies objective and the learned gating policy is never made precise. Please clarify whether Eq. (5) is intended as a conceptual target or as the actual optimization objective, and derive how the actor-critic training approximates it.","section":"Section IV-B, Eq. (5); Section IV-C"}],"minor_comments":[{"comment":"There is a typo in \"it's own variants\" — it should be \"its own variants.\"","section":"Section VI-A"},{"comment":"The gating function is introduced as g(s,c) in Section III-D but then written as g_k^rho(s,c) in Section IV-C; align the notation to avoid confusion about parameters.","section":"Section IV-C"},{"comment":"The reward weights w1=30, w2=15, w3=10 and the cooperation probability p=0.2 are stated without a sensitivity analysis; please state whether these values come from prior work or justify them with an ablation.","section":"Section V-A"},{"comment":"The statement that jointly controlling residual actions and nominal policy selection \"performs better than using a separate gating network\" is an empirical claim with no supporting ablation; add the comparison or soften the claim.","section":"Section VI-B"},{"comment":"The colorbar is labeled \"emission benefit percentage,\" but the caption does not state explicitly that this is relative to the IDM baseline; please make that explicit.","section":"Figure 3"},{"comment":"The interpretation of the nominal-policy usage pattern as an \"implicit curriculum\" is speculative; label it as a qualitative observation rather than a demonstrated property of the method.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The architecture is the real contribution: MRMEL extends residual RL from a single nominal policy to a gated mixture of multiple nominals, with gating and residual learned jointly and conditioned on scenario context. That is a natural, well-motivated next step after the group's own MRTL, and I don't see it in the cited RRL, MoE, or MRTL papers. The empirical scope is also substantial: roughly 5,000 scenarios across three cities, two penetration levels, and a reasonable baseline list including GLOSA, IDM, multi-task from scratch, and single-nominal RRL variants. The paper is honest about its continuous-control limitation, and Figure 5's implicit-curriculum story is a nice observation, even if speculative.\n\nThe soft spots, though, are real and load-bearing. The stress-test note is right: Section VI-B describes training on the IntersectionZoo scenarios, and Section VI-C reports results \"across all scenarios\" with no mention of a train/validation/test split. Figure 3 plots every intersection. If the same scenarios used for PPO training produced Table I, the 4%-9% is an in-distribution score and the paper's central generalization claim is untested. The manuscript nowhere states that evaluation intersections were unseen during training. That must be fixed before the generalization language is credible.\n\nThe reader's other concerns also hold. There are no seeds, no standard deviations, and Table I contains a suspicious outlier: RRL with constant acceleration in Atlanta at 100% penetration shows an 8.66% throughput gain against near-zero values everywhere else. No code or data is released, which makes it impossible to check. The IntersectionZoo emission-model fidelity question is real but secondary; even a perfect simulator doesn't help if the split problem is unresolved.\n\nWho gets value from this? Researchers working on residual RL and on eco-driving or Lagrangian control. The method deserves serious attention; the evidence does not yet. With a proper held-out evaluation, multi-seed results, and code/data release, this could be a solid contribution. As it stands, it reads like a strong workshop paper with a conference-level claim. I'd recommend sending it to peer review rather than desk-rejecting: the architecture is a genuine extension, and the evaluation gaps are addressable in revision. A serious referee should ask for the split, the seeds, and the code before signing off on the 4%-9%.","headline":"A genuine architecture extension of residual RL with a gated mixture of nominal policies, but the headline 4-9% claim rests on an evaluation that lacks a held-out scenario split and any uncertainty quantification.","tokens_in":11555,"tokens_out":1954,"would_cite":true,"duration_ms":23377,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a gated mixture of residual-corrected prior policies lets autonomous vehicles learn one eco-driving controller that generalizes across real-world traffic scenarios and beats every tested baseline by 4–9% in emissions.","keywords":["autonomous vehicles","Lagrangian traffic control","eco-driving","residual reinforcement learning","mixture of experts","multi-task reinforcement learning","contextual MDP","signalized intersections"],"falsifier":"Deploy the trained MRMEL policy in a field test, or in a high-fidelity modal-emissions simulator with measured trajectories, over the same intersections and compare aggregate emissions against the strongest single-prior baseline under matched traffic; if the additional reduction does not reach the claimed 4–9% range or is not statistically significant, the paper's central generalization claim would be contradicted.","tokens_in":10536,"feed_emoji":"🚗","tokens_out":8894,"duration_ms":93605,"temperature":0.7,"pith_summary":"This paper claims that a fleet of autonomous vehicles can learn one eco-driving policy that generalizes across thousands of real-world signalized intersections by combining two existing ideas: residual reinforcement learning and a gated mixture of experts. The proposed framework, MRMEL, keeps a small pool of prior control policies—ranging from the GLOSA green-light-optimized-speed-advisory heuristic to a zero-action policy—and learns a context-dependent selector that picks the best prior for each traffic scenario while a neural residual corrects the chosen action. In simulated cooperative eco-driving calibrated to Dallas, Atlanta, and Salt Lake City, MRMEL consistently beats every baseline, achieving an additional 4–9% reduction in aggregate vehicle emissions over the strongest competing method in each city at both 30% and 100% AV penetration. If these simulation results hold, the framework would let cities deploy one learned controller rather than hand-tuning policies for each intersection or traffic regime.","feed_headline":"Gated residual experts cut fleet emissions 4-9% beyond best baselines","feed_subtitle":"One policy adapts to thousands of real intersections in three cities and beats every single-prior baseline.","key_machinery":"The central object is the gated mixture-of-nominal-policies residual policy. A set of five nominal policies—the GLOSA green-light-optimized-speed-advisory heuristic, the Intelligent Driver Model, constant acceleration, constant deceleration, and a zero-action policy—are combined by a learned gating network that outputs a one-hot selection given the state and scenario context; the selected nominal action is added to a context-conditioned residual $f_\\theta(s,c)$ learned with the PPO algorithm. The gating lets the method switch among priors depending on the scenario (for example, GLOSA in free-flow conditions, constant deceleration or zero action for gliding and idling reduction), and joint training of gating and residual creates the implicit curriculum observed by the authors: early reliance on constant acceleration to learn to move, then a shift toward deceleration and zero-action to refine emission behavior.","core_discovery":"At its center, the paper claims that a policy of the form $\\pi^*(s,c)=\\sum_{k=1}^{K}g^\\rho_k(s,c)\\,\\pi^n_k(s)+f_\\theta(s,c)$ generalizes across traffic scenarios better than the alternatives it compares against. Here $s$ is the vehicle's observation, $c$ is a context vector describing the traffic scenario (signal timing, lane length, grade, vehicle mix, and so on), $\\pi^n_k$ are fixed nominal policies, $g^\\rho$ is a learned gating distribution over them, and $f_\\theta$ is a learned residual correction. The paper calls this architecture MRMEL (Multi-residual Mixture of Experts Learning) and evaluates it on cooperative eco-driving at signalized intersections. Its reported result is that MRMEL 'consistently yields superior performance—achieving an additional 4%–9% reduction in aggregate vehicle emissions relative to the strongest baseline in each setting' across Dallas, Atlanta, and Salt Lake City at both 30% and 100% AV penetration, with similar advantages in intersection throughput. The authors also show that emission benefits spread across the spatial distribution of intersections rather than concentrating in easy cases, which they interpret as evidence that the policy generalizes rather than overfits.","pith_inferences":["A field trial with instrumented vehicles, or a re-evaluation in a modal emissions model that uses measured trajectories, would test whether the 4–9% margin survives contact with reality; the simulated emission model is the main place the result could evaporate.","The reported gating dynamics suggest an experiment: freeze the learned gating schedule as a fixed curriculum and train only the residual; if performance matches MRMEL, the gain may come from exploration structure rather than online expert selection.","Because the paper evaluates one reward weighting and a fixed pool of five nominal policies, the framework's advantage over multi-task RL should be re-tested as the expert pool and reward weights change; the current result does not delimit that sensitivity."],"forward_implications":["A single MRMEL policy per city outperforms the strongest single-prior baseline in every city and AV-penetration combination reported, so scenario-specific retraining is not needed for the tested cities.","Because one nominal policy is a zero-action policy, the residual can learn the full action from scratch when all priors are poor, protecting against badly chosen priors.","The observed shift in nominal-policy usage over training suggests an implicit curriculum, with early training relying on constant acceleration and later training shifting to deceleration and zero-action policies.","Since MRMEL is formulated for general continuous-control Lagrangian traffic tasks, the same architecture can be applied to other AV-based traffic objectives such as congestion smoothing or emergency-vehicle clearance.","Higher AV penetration magnifies the reported emission and throughput improvements, which the authors attribute to more vehicles being able to adopt the eco-driving strategy."],"supporting_citations":[{"why":"Introduces residual policy learning, the foundation that MRMEL extends.","marker":"[6]"},{"why":"Independently introduces residual reinforcement learning and motivates the residual structure used here.","marker":"[7]"},{"why":"Proposes the single-nominal residual eco-driving policy (MRTL) that MRMEL generalizes and compares against.","marker":"[8]"},{"why":"Provides the pre-built eco-driving CMDPs and real-world calibrated traffic scenario distributions used for training and evaluation.","marker":"[23]"},{"why":"Proposes the GLOSA controller used as one nominal expert and as an industry-standard baseline.","marker":"[4]"},{"why":"Defines the IDM car-following model used as a nominal expert and as the human-like driving baseline.","marker":"[24]"},{"why":"Provides PPO, the actor-critic algorithm used to train the gating and residual networks.","marker":"[25]"}],"fun_headline_variants":["Mixture-of-experts residual policy cuts emissions 4-9% at intersections","Adaptive AV policy beats baselines with 4-9% emission cuts","Multi-residual experts learn eco-driving across three cities","MRMEL: Mixing experts and residuals for greener traffic","Residual expert mixture shaves 4-9% emissions from fleet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pre-built simulation, including its instantaneous-emissions model and human car-following behavior, faithfully reproduces real-world traffic and emissions; if the simulator exaggerates the benefits of smooth acceleration and platooning, the measured 4–9% additional emission reduction could shrink or vanish in the real world.","fun_headline_variants_meta":{"raw":{"variants":["Mixture-of-experts residual policy cuts emissions 4-9% at intersections","Adaptive AV policy beats baselines with 4-9% emission cuts","Multi-residual experts learn eco-driving across three cities","MRMEL: Mixing experts and residuals for greener traffic","Residual expert mixture shaves 4-9% emissions from fleet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000645,"raw_usage":{"total_tokens":3044,"prompt_tokens":1103,"completion_tokens":1941,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":719,"completion_tokens_details":{"reasoning_tokens":1862}},"tokens_in":719,"tokens_out":1941,"duration_ms":14935,"temperature":1.0,"reasoning_tokens":1862,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:47:00.290937+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Deploy the trained MRMEL policy in a field test, or in a high-fidelity modal-emissions simulator with measured trajectories, over the same intersections and compare aggregate emissions against the strongest single-prior baseline under matched traffic; if the additional reduction does not reach the claimed 4–9% range or is not statistically significant, the paper's central generalization claim would be contradicted.","supporting_citations":[{"cited_title":"Residual reinforcement learning for robot control","cited_arxiv_id":null,"evidence_quote":"Independently introduces residual reinforcement learning and motivates the residual structure used here."},{"cited_title":"Generalizing cooperative eco-driving via multi-residual task learning","cited_arxiv_id":null,"evidence_quote":"Proposes the single-nominal residual eco-driving policy (MRTL) that MRMEL generalizes and compares against."},{"cited_title":"Intersectionzoo: Eco-driving for bench- marking multi-agent contextual reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Provides the pre-built eco-driving CMDPs and real-world calibrated traffic scenario distributions used for training and evaluation."},{"cited_title":"Performance study of a green light optimized speed advisory (glosa) application using an integrated cooperative its simulation plat- form","cited_arxiv_id":null,"evidence_quote":"Proposes the GLOSA controller used as one nominal expert and as an industry-standard baseline."},{"cited_title":"Traffic flow dynamics , volume 1","cited_arxiv_id":null,"evidence_quote":"Defines the IDM car-following model used as a nominal expert and as the human-like driving baseline."}],"review_version":1}