Pith. sign in

REVIEW 3 major objections 3 minor 64 references

SelectLight: Learning to Select Signal Plans Generated by Distributed Model Predictive Control for Urban Traffic Networks

T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SelectLight shows that a learned selector choosing among MPC-generated signal plans beats fixed-rule, direct-RL, and traffic-responsive baselines on delay-related metrics, and the margin widens as demand rises.

desk verdict A clean, well-engineered MPC-RL integration paper that earns a careful review; the load-bearing caveat is the unvalidated stationary propagation tensor, not the learning setup. read the letter →

arxiv 2608.12256 v1 pith:WNASY2GV submitted 2026-08-12 eess.SY cs.SY

Lyuzhou Luo , Chaopeng Tan , Zhengyong Gao , Hong Zhu , Andrea D'Ariano , Keshuang Tang This is my paper · ORCID
classification eess.SYcs.SY
keywords NetworktrafficsignalcontrolDistributedmodelpredictiveMulti-agentreinforcementlearningPost-optimizationselectionMulti-objectivedynamicprogrammingAttentionpolicyZero-shottransferQueueingdelay
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a multi-agent reinforcement learning policy can profit by choosing among the feasible signal plans that a distributed model predictive controller generates online, instead of synthesizing phase decisions itself or tuning the optimizer's weights. It proposes post-optimization selection: SP-MODP, a state-pruned multi-objective dynamic program over a Newellian point-spatial queue model, returns a bounded set of mutually nondominated plans at every control update, and a topology-aware attention policy trained with IPPO selects one plan without modifying it. The paper argues this division of labor keeps the signal timing constraints and inspectability of MPC while letting the selection rule adapt to realized closed-loop outcomes. On two 28-intersection networks, SelectLight achieves the best delay-related performance among seven controllers, and its advantage widens with demand: at twice the baseline demand it reduces average cumulative queue by 5.57% and average waiting time by 6.44% relative to the strongest baseline. It also incurs the lowest zero-shot transfer loss in every tested demand shift.

What carries the argument

The load-bearing mechanism is the pair formed by SP-MODP and the attention selector. SP-MODP searches a stage-time lattice of phase end times, carrying a plan-dependent label state--cumulative departures and a queue-dissipation front state--instead of inflating the node index; objective-space dominance pruning and per-node and terminal label caps bound the candidate set to at most $K$ plans. The Newellian point-spatial queue model provides the deterministic rollout that evaluates each partial plan, distinguishing the point queue from the spatial queue through the dissipation front. The selector encodes lanes, phases, intersection context, and each candidate's objective vector and ordered stage structure, then applies masked attention so that candidate scores depend only on the plans actually available in the current set.

What would settle it

Run SelectLight under the same training protocol on a third network or on a time-varying demand profile where the flow-propagation tensor is known to be miscalibrated; if its queueing-delay advantage over the fixed-rule MO-DMPC baseline shrinks to zero, or its zero-shot ACQ transfer loss rises above the direct-RL baselines, the central claim that online candidate generation plus learned selection transfers across conditions would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that MPC and RL should meet at the level of the plan set rather than the plan or the objective weights: the lower level constructs a state-dependent bounded set of feasible candidate signal plans, and the upper level learns to pick one plan from that set. SelectLight instantiates this by having SP-MODP return mutually nondominated plans evaluated on total queueing delay, peak queue accumulation, and total number of stops, and by having an attention policy select one plan with masked scaled dot-product scoring over the currently available candidates. The selection is constrained by construction, because the selector cannot invent a plan; it only chooses an unmodified feasible one. The paper reports that this yields the best delay-related performance on both test networks, that the advantage grows as demand increases, and that the learned selection rule transfers to unseen demand levels and network configurations with lower loss than direct-RL baselines.

Load-bearing premise

The whole approach assumes the calibrated, fixed flow-propagation model and the simplified queue model stay accurate enough at the demand levels and networks where the controller runs; if those predictions drift, the candidate plans become uninformative and the learned selector has nothing useful to choose among.

Editorial extensions

If this is right

  • Any Pareto-based MPC that can emit multiple feasible plans can be upgraded to learned selection without changing the optimizer, preserving feasibility and timing guarantees by construction.
  • The widening gap under heavier demand indicates that the value of learned selection concentrates in congested conditions rather than in light traffic.
  • Because online MPC regenerates the candidate plans for each deployment condition, the policy can be trained in one scenario and deployed in another with modest degradation, reducing the need for per-network or per-demand retraining.
  • The per-intersection p99 SP-MODP solution time of 5.408 ms at the default 120 s horizon shows that generating a diverse candidate set costs almost nothing against the 5 s control interval.
  • Selected plans and their predicted objective trade-offs remain visible to traffic managers, preserving an audit trail for why one plan was preferred over another.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the reported transfer advantage is not decomposed between candidate diversity and selector skill; a testable extension is to vary the candidate set size or objectives and measure how much of the zero-shot gain remains.
  • Beyond the paper: the rank and percentile analysis suggests the learned policy implements a state-dependent preference among delay, queue, and stops; probing the attention weights could yield an interpretable rule for when the controller protects against spillback instead of minimizing raw delay.
  • Beyond the paper: post-optimization selection is not specific to traffic signals; any constrained planner that returns a Pareto set of feasible plans could expose that set to a learned selector, though the paper does not claim this transfer.
  • Beyond the paper: since the calibrated stationary flow-propagation tensor is the weakest input, randomizing or re-estimating it during training is a natural robustness extension that the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes SelectLight, a traffic signal controller that couples distributed multi-objective MPC with MARL: at each 5 s control update, SP-MODP generates a bounded set of nondominated candidate signal plans using a Newellian point–spatial queue model, and an IPPO-trained attention policy selects one unmodified plan. The authors claim that this post-optimization selection pattern confines learning to selection, preserves signal constraints, and yields the best delay-related performance on two 28-intersection SUMO networks, with the advantage growing under doubled demand (5.57% ACQ and 6.44% AWT relative to the strongest baseline), the lowest zero-shot transfer loss, and p99 solve times below the control interval.

Significance. If the results hold, post-optimization selection is a useful and well-positioned MPC–RL integration pattern. Strengths include a careful experimental protocol with eight seeds, a fixed training/evaluation procedure without checkpoint selection, ablations, transfer tables, and latency measurements. The technical device of masked attention over variable-size candidate sets is elegant, and the separation between constraint-aware plan construction and learned selection is conceptually clean. The main caveats are that the baseline-demand margins are small and not significance-tested, and the demand-robustness and transfer claims depend on a stationary propagation tensor W whose calibration is inherited and not validated under shifted demand. These are addressable with additional analyses rather than conceptual errors.

major comments (3)
  1. [§8.2, Table 5; §8.3, Fig. 10; §8.4, Fig. 11; Tables B.10–B.12] The headline comparisons are reported as mean ± SD over eight seeds, but no significance tests are given. At baseline demand on Lianyungang, SelectLight's ACQ advantage over SO-DMPC is only 1.68% (9720.03 ± 77.03 vs. 9886.19 ± 159.47), and its ATT advantage is below 0.5% (283.38 vs. 284.61 s), so with N=8 the reported intervals do not by themselves establish that these differences are reliable. The same issue affects the demand-robustness and transfer claims: Figure 10 and Tables B.10–B.12 report means without paired comparisons between methods. Because 'best delay-related performance' and 'advantage widens with demand' are the paper's central claims, please add paired per-seed tests (paired t-test or Wilcoxon signed-rank) for the headline metrics, and report effect sizes or confidence intervals.
  2. [§5.1, Eq. (7); §8.1.4; §8.3; §8.4; §9] The demand-robustness and transfer results rest on the stationary flow-propagation tensor W in Eq. (7), which converts upstream departures into downstream arrival profiles. Section 8.1.4 states that the prediction module was calibrated before policy training and then held fixed, and Section 9 lists only a calibrated, stationary AVI-based propagation model as a limitation. No calibration dataset, validation accuracy, or demand-sensitivity analysis for W is reported. Because SP-MODP evaluates every candidate plan using arrival profiles produced by Eq. (7), the informativeness of the candidate set under demand scales 1.5 and 2.0 is an unverified structural input to the claim that SelectLight's advantage widens under congestion. I request either (a) a report of the prediction error of Eq. (7) at each demand scale, (b) an ablation in which W is recalibrated at scales 1.5 and 2.0 and the comparisons are repeated, or (c) an explicit demonstration that the AVI feedback in Section 5.2 keeps the point queue consistent with observed counts under shifted demand.
  3. [§8.6.1, Fig. 12] The corridor-progression analysis used to explain SelectLight's gains is a single 500 s window (2700–3200 s) at demand scale 2.0, drawn from a single corridor with no indication of the seed. The text states that SelectLight 'demonstrates stronger corridor-level coordination' and 'fewer stop-and-go interruptions,' but the reported mean speeds and stopped-sample shares (e.g., 8.9 vs. 8.0 m/s south-to-north) are not accompanied by multi-seed or multi-window statistics. Please report aggregate statistics across the eight evaluation seeds and multiple time windows, or explicitly label this as an illustrative case study.
minor comments (3)
  1. [Fig. 10(b); Table B.10] Figure 10(b) appears inconsistent with Table B.10: at demand scale 2.0, all methods' ATT values exceed 300 s (e.g., SelectLight 313.77 s, MP 367.55 s), but the y-axis is drawn to 300 s. Please rescale or annotate the axis.
  2. [§8.3 vs. §8.2] Section 8.3 says the scale-1.0 ACQ improvement over SO-DMPC is 1.65%, while Section 8.2 and Table 5 imply 1.68% (1 − 9720.03/9886.19). Please make the number consistent.
  3. [§6.2, Eq. (13)] In Eq. (13), the sign convention for upstream distance ξ could be stated explicitly (e.g., ξ increases upstream) so that the direction of the Newellian time shift is unambiguous for readers unfamiliar with the coordinate system.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the learned selector is trained on realized queueing delay and evaluated on held-out seeds, demand levels, and a second network.

full rationale

The derivation chain is self-contained with respect to the central claim. The SelectLight policy is trained by IPPO on realized local queueing delay (Eq. 52) and evaluated on held-out seeds, demand scales not used for training, and a second network. The SP-MODP candidate set is generated online from a Newellian queue model and predicted arrivals (Eqs. 7, 17-22, 24-27), and no evaluation metric is equal by construction to a quantity fitted to that metric. The only notable self-citation is the inherited calibrated propagation tensor W from Luo et al. (2025a), whose calibration is not reported here; this is a stated limitation (Section 9) and a robustness concern, not a circular reduction, because W is held fixed before policy training and does not encode the learned selection outcome. No step reduces the headline performance claim to its inputs.

Assumptions & free parameters 8 free parameters · 7 assumptions · 0 invented entities

No new physical entities are introduced. The central claim rests chiefly on calibrated inputs, notably the propagation tensor W and queue model parameters, which are not fully specified, and on hand-chosen SP-MODP caps. The learned-selection component is trained on realized SUMO delay and evaluated on separate seeds, so the headline numbers are not fitted constants.

free parameters (8)
  • Flow-propagation tensor W and platoon dispersion parameters = Not reported (calibrated following Luo et al. 2025a)
    Equation (7) uses W to propagate departures to arrivals; W encodes travel-time offsets, link travel-time distributions, and turning fractions calibrated from AVI data and held fixed during training and evaluation.
  • Newellian queue model parameters (v_f, h_jam, s_i,m) = Not reported (lane-specific effective values)
    Equations (13), (14), and (18) define free-flow speed, jam spacing, and saturation discharge capacity that determine the queue rollouts and all three candidate objectives.
  • Macro decision interval M = 5 s
    Length of the executed plan prefix and control period; a design choice, not tuned in the paper.
  • Prediction horizon H = 120 s (sensitivity 60-150 s)
    Default horizon for SP-MODP; sensitivity in Appendix B.4 shows stable performance around the default.
  • Maximum candidate set size K = 25
    Cap on the number of candidate plans returned to the selector; sensitivity at 10, 25, and 50 shows small effects.
  • Per-node label cap K_pf = 50
    Cap on labels propagated at each stage-time node in SP-MODP; no sensitivity experiment is reported.
  • SP-MODP timing discretization = 2 s
    Granularity of phase end times in the stage-time lattice; no sensitivity experiment is reported.
  • Maximum stages Rbar = 8
    Upper bound on phase transitions per candidate plan; no sensitivity experiment is reported.
assumptions (7)
  • domain assumption The deterministic Newellian point-spatial queue model with arrivals-before-departures is an adequate within-horizon proxy for real queue dynamics.
    Invoked in Section 6.2 to compute queue rollout, delay, peak queue, and stops for every candidate plan.
  • domain assumption The calibrated flow-propagation tensor W and the AVI feedback rules correctly predict future arrivals and correct prediction errors.
    Equations (7)-(10) generate the predictive arrival profiles that drive SP-MODP; calibration is referenced to Luo et al. (2025a) and not reported.
  • domain assumption The signal timing constraints, including fixed phase order, green minimum and maximum, intergreen, and inter-update phase-end tolerance, define the feasible plan space exactly.
    Section 6.4 encodes these as Equations (28)-(36); all claims of feasibility preservation rest on them.
  • domain assumption Negative realized local queueing delay over the executed interval is a sufficient online surrogate for the system-level travel-delay objective.
    Sections 3.3 and 7.4 define the reward; the authors state the surrogate relationship but do not prove alignment.
  • domain assumption SUMO microsimulation on the Lianyungang and Jiading networks is an adequate ground-truth environment for evaluating the controllers.
    All headline numbers in Tables 5 and B.10 come from SUMO simulations described in Section 8.1.
  • domain assumption IPPO with parameter sharing and the common PPO protocol trains a selector that generalizes across intersections.
    Training details are in Appendix A.2; no convergence guarantee is provided, which is standard for MARL.
  • standard math The area under the point-queue trajectory equals total queueing delay.
    Equation (24) follows the input-output accounting identity of Daganzo (1983).

how reviews work

0 comments
Cite this review

Pith. "Pith review of SelectLight: Learning to Select Signal Plans Generated by Distributed Model Predictive Control for Urban Traffic Networks." pith.science (2026). https://pith.science/paper/WNASY2GV

@misc{pith2026260812256,
  author       = {Pith},
  title        = {Pith review of: SelectLight: Learning to Select Signal Plans Generated by Distributed Model Predictive Control for Urban Traffic Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WNASY2GV}},
  note         = {Machine review of arXiv:2608.12256}
}
read the original abstract

Coordinated traffic signal control across urban networks must adapt to changing demand while satisfying operational constraints. Multi-objective distributed model predictive control (DMPC) can construct feasible signal plans online, but prescribed rules for selecting among trade-off solutions cannot learn from realized closed-loop outcomes. We propose SelectLight, which implements post-optimization selection by allowing a multi-agent reinforcement learning (MARL) policy to choose directly from plans generated online by DMPC. At each control update, state-pruned multi-objective dynamic programming (SP-MODP) evaluates plans with a Newellian point--spatial queue model and returns a bounded set of mutually nondominated candidate signal plans for total queueing delay, peak queue accumulation, and total number of stops. A topology-aware attention policy trained with independent proximal policy optimization (IPPO) selects one unmodified plan from each variable-size set. This confines learning to candidate selection, preserves the prescribed signal timing constraints, and leaves the selected plan and its predicted objective trade-offs available for inspection. Experiments on two 28-intersection SUMO networks show that SelectLight achieves the best delay-related performance and that its advantage widens with demand. At twice the baseline demand, it reduces queueing delay and waiting time by 5.57% and 6.44%, respectively, relative to the strongest baseline. SelectLight also incurs the lowest transfer loss under every tested demand shift. With a 120 s prediction horizon, the per-intersection 99th-percentile SP-MODP solution time is 5.408 ms, well below the 5 s control interval.

Figures

Figures reproduced from arXiv: 2608.12256 by the authors.

Figure 1
Figure 1. Three deployment-time MPC–RL integration patterns distinguished by the decision object exposed at their interface. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Temporal structure of the hierarchical control framework. At each macro step, each agent solves its local multi-objective [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Deployment-aware overview of SelectLight. The cloud-deployed prediction module and per-intersection candidate genera [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: AVI sensing configuration and traffic propagation between adjacent signalized intersections. Devices near the stop lines [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the deterministic Newellian point–spatial queue model. (a) Physical and Newellian coordinates. (b) Point [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Phase structures and phase-based horizon representation. (a) Standard four-phase structure used as the running example. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Schematic illustration of SP-MODP on the stage–time lattice. Each node [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Observation encoders used by the candidate selector. The intersection encoder aggregates lane embeddings along the [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: SUMO networks representing urban areas in (a) Lianyungang, Jiangsu Province, and (b) Jiading District, Shanghai. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Performance across demand levels on the Lianyungang network. Each learning method is trained at the corresponding [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: Zero-shot transfer without fine-tuning: (a) directed transfer among Lianyungang demand scales and (b) bidirectional [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]
Figure 12
Figure 12. Figure 12: Vehicle trajectories under SelectLight and IPPO on a representative Lianyungang corridor during 2700–3200 s at demand [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: Candidate-selection behavior on the Lianyungang network at demand scale 2.0: (a) distribution of the ideal-point rank [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 30 canonical work pages

  1. [6]

    Transportation Research Part C: Emerging Technologies 85, 732–752

    Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events. Transportation Research Part C: Emerging Technologies 85, 732–752. doi:10.1016/j.trc.2017.09.020. Berg, M.V.D., Hegyi, A., De Schutter, B., Hellendoorn, H.,

    work page doi:10.1016/j.trc.2017.09.020 2017

  2. [12]

    Proceedings of the AAAI Conference on Artificial Intelligence 34, 3414–3421

    Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. Proceedings of the AAAI Conference on Artificial Intelligence 34, 3414–3421. doi:10.1609/aaai.v34i04.5744. Chu, T., Wang, J., Codeca, L., Li, Z.,

    doi:10.1609/aaai.v34i04.5744

  3. [13]

    IEEE Transactions on Intelligent Transportation Systems 21, 1086–1095

    Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Transactions on Intelligent Transportation Systems 21, 1086–1095. doi:10.1109/tits.2019. 2901791. Daganzo, C.F.,

    doi:10.1109/tits.2019 2019

  4. [15]

    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.,

    Is independent learning all you need in the StarCraft multi-agent challenge? doi:10.48550/ARXIV.2011.09533, arXiv:2011.09533. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.,

    arXiv doi:10.48550/arxiv.2011.09533 2011

  5. [18]

    Expert Systems with Applications 229, 120535

    Multi-objective traffic signal control using network-wide agent coordinated reinforcement learning. Expert Systems with Applications 229, 120535. doi:10.1016/j. eswa.2023.120535. Feng, Y., Head, K.L., Khoshmagham, S., Zamanipour, M.,

    arXiv 2023

  6. [20]

    IEEE Transactions on Smart Grid 15, 4349–4363

    A novel reinforcement-learning-based compensation strategy for DMPC-based day-ahead energy management of shipboard power systems. IEEE Transactions on Smart Grid 15, 4349–4363. doi:10.1109/TSG.2024.3382213. Ghezzi, A., Hoffmann, J., Frey, J., Boedecker, J., Diehl, M.,

    arXiv 2024

  7. [21]

    Imitation learning from nonlinear MPC via the exact Q-Loss and its Gauss-Newton approximation, in: 2023 62nd IEEE Conference on Decision and Control (CDC), IEEE. pp. 4766–4771. doi:10.1109/CDC49753.2023.10383323. Gros, S., Zanon, M.,

    arXiv 2023

  8. [22]

    IEEE Transactions on Automatic Control 65, 636–648

    Data-driven economic NMPC using reinforcement learning. IEEE Transactions on Automatic Control 65, 636–648. doi:10.1109/TAC.2019.2913768. Guilliard, I., Sanner, S., Trevizan, F.W., Williams, B.C.,

    arXiv 2019

Show all 64 references
  1. [24]

    Control Engineering Practice 110, 104758

    An integrated MPC and deep reinforcement learning approach to trams-priority active signal control. Control Engineering Practice 110, 104758. doi:10.1016/j.conengprac.2021.104758. Han, X., Zhao, X., Zhang, L., Wang, W.,

    2021

  2. [25]

    Mitigating action hysteresis in traffic signal control with traffic predictive reinforcement learning, in: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery. pp. 673–684. doi:10.1145/3580305. 3599528. van ...

    doi

  3. [26]

    doi:10.48550/arXiv.1602.07714,arXiv:1602.07714

    Learning values across many orders of magnitude. doi:10.48550/arXiv.1602.07714,arXiv:1602.07714. Huber, P.J.,

    doi arXiv

  4. [28]

    Transportation Research Part C: Emerging Technologies 154, 104258

    A survey on urban traffic control under mixed traffic environment with connected automated vehicles. Transportation Research Part C: Emerging Technologies 154, 104258. doi:10.1016/j.trc.2023.104258. Li, K., Wang, Z., Dinh, T.Q., Yoon, J.,

    2023

  5. [29]

    IEEE Transactions on Vehicular Technology 74, 12058–12071

    Reinforcement learning-based hyperparameter tuning for adaptive model predictive controllers in battery thermal management. IEEE Transactions on Vehicular Technology 74, 12058–12071. doi:10.1109/TVT.2025.3552968. Li, X., Sun, J.Q.,

    2025

  6. [32]

    Energy Conversion and Management 341, 120081

    Novel energy management strategy for fuel cell/battery hybrid energy systems combining MPC and deep reinforcement learning. Energy Conversion and Management 341, 120081. doi:10.1016/j.enconman.2025.120081. 34 Lopez, P.A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flötteröd, ...

    2025

  7. [34]

    IEEE Transactions on Intelligent Transportation Systems 22, 5577–5588

    A back-pressure-based model with fixed phase sequences for traffic signal optimization under oversaturated networks. IEEE Transactions on Intelligent Transportation Systems 22, 5577–5588. doi:10.1109/TITS.2020.2987917. Newell, G.F.,

    2020

  8. [35]

    Transportation Research Part B: Methodological 36, 195–205

    A simplified car-following theory: A lower order model. Transportation Research Part B: Methodological 36, 195–205. doi:10.1016/S0191-2615(00)00044-8. Papageorgiou, M., Kiakaki, C., Dinopoulou, V., Kotsialos, A., Yibing Wang,

    doi

  9. [37]

    IEEE Transactions on Intelligent Transportation Systems 24, 8079–8096

    Distributed stochastic MPC traffic signal control for urban networks. IEEE Transactions on Intelligent Transportation Systems 24, 8079–8096. doi:10.1109/tits.2023.3262580. Qadri, S.S.S.M., Gökçe, M.A., Öner, E.,

    2023

  10. [38]

    European Transport Research Review 12, 55–77

    State-of-art review of traffic signal control methods: Challenges and opportunities. European Transport Research Review 12, 55–77. doi:10.1186/s12544-020-00439-1. Reiter, R., Hoffmann, J., Reinhardt, D., Messerer, F., Baumgärtner, K., Sawant, S., Bödecker, J., Diehl, M., Gros, S.,

    doi

  11. [39]

    Annual Reviews in Control 61, 101045

    Synthesis of model predictive control and reinforcement learning: Survey and classification. Annual Reviews in Control 61, 101045. doi:10.1016/j.arcontrol.2026.101045. Remmerswaal, W., Sun, D., Jamshidnejad, A., De Schutter, B.,

    2026

  12. [40]

    Combined MPC and reinforcement learn- ing for traffic signal control in urban traffic networks, in: 2022 26th International Conference on System The- ory, Control and Computing (ICSTCC), IEEE, Sinaia, Romania. pp. 432–439. doi:10.1109/ICSTCC55426. 2022.9931771. Rinaldi, M., Pi...

    2022

  13. [41]

    Mixed hybrid and electric bus dynamic fleet management in urban networks: A model predictive control approach, in: 2019 6th International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS), IEEE. pp. 1–8. doi:10.1109/ MTITS.2019.8883387. Saik...

    2019

  14. [42]

    IEEE Access 11, 75875–75883

    Flexible traffic signal control via multi-objective reinforcement learning. IEEE Access 11, 75875–75883. doi:10.1109/ACCESS.2023.3296537. Schulman, J., Moritz, P., Levine, S., Jordan, M.I., Abbeel, P.,

    2023

  15. [43]

    doi:10.48550/arXiv.1707.06347,arXiv:1707.06347

    Proximal policy optimization algorithms. doi:10.48550/arXiv.1707.06347,arXiv:1707.06347. Sen, S., Head, K.L.,

    doi arXiv

  16. [45]

    IEEE Open Journal of Control Systems 4, 316–331

    Integrating reinforcement learning and model pre- dictive control for mixed-logical dynamical systems. IEEE Open Journal of Control Systems 4, 316–331. doi:10.1109/OJCSYS.2025.3601435. Sun, D., Jamshidnejad, A., De Schutter, B., 2024a. Adaptive parameterized model predictive c...

    2025

  17. [46]

    IEEE Transactions on Intelligent Trans- portation Systems 26, 21635–21650

    Connected vehicle data-driven robust optimization for traffic signal timing: Modeling traffic flow variability and errors. IEEE Transactions on Intelligent Trans- portation Systems 26, 21635–21650. doi:10.1109/TITS.2025.3606559. Tan, C., Luo, L., Yang, K., Zhu, H., Tang, K., 2...

    2025

  18. [47]

    Actuators 13, 251–265

    Traffic signal control optimization based on neural network in the framework of model predictive control. Actuators 13, 251–265. doi:10.3390/act13070251. Tang, K., Cao, Y., Chen, C., Yao, J., Tan, C., Sun, J.,

    doi

  19. [48]

    Computer-Aided Civil and Infrastructure Engineering 36, 31–46

    Dynamic origin-destination flow estimation using automatic vehicle identification data: A 3D convolutional neural network approach. Computer-Aided Civil and Infrastructure Engineering 36, 31–46. doi:10.1111/mice.12559. Varaiya, P.,

    doi

  20. [50]

    IEEE Transactions on Intelligent Transportation Systems 23, 333–343

    Optimizing signal timing control for large urban traffic networks using an adaptive linear quadratic regulator control strategy. IEEE Transactions on Intelligent Transportation Systems 23, 333–343. doi:10.1109/tits.2020.3010725. Wang, X., Abdulhai, B., Sanner, S.,

    2020

  21. [51]

    Edward Elgar Publishing, pp

    A critical review of traffic signal control and a novel unified view of reinforcement learning and model predictive control approaches for adaptive traffic signal control, in: Handbook on Artificial Intelligence and Transport. Edward Elgar Publishing, pp. 482–532. doi:10.4337/...

  22. [53]

    CoLight: Learning network-level cooperation for traffic signal control, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Association for Computing Machinery, New York, NY, USA. pp. 1913–1922. doi:10.1145/3357384.3357902. Wei, H....

    1913

  23. [54]

    IntelliLight: A reinforcement learning approach for intelligent traffic light control, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery, New York, NY, USA. pp. 2496–2505. doi:10.1145/32198...

  24. [55]

    Engineering Optimization 56, 2579–2599

    Distributed predictive motion planning of automated guided vehicles: Serial versus parallel schemes. Engineering Optimization 56, 2579–2599. doi:10.1080/0305215X.2024.2318342. Xin, J., Wu, X., D’Ariano, A., Negenborn, R., Zhang, F.,

    2024

  25. [56]

    IEEE Transactions on Intelligent Transporta- tion Systems 24, 6943–6954

    Model Predictive Path Planning of AGVs: Mixed Logical Dynamical Formulation and Distributed Coordination. IEEE Transactions on Intelligent Transporta- tion Systems 24, 6943–6954. doi:10.1109/TITS.2023.3254147. Xu, B., Wang, Y., Wang, Z., Jia, H., Lu, Z.,

    2023

  26. [57]

    Proceedings of the AAAI Conference on Artificial Intelligence 35, 669–677

    Hierarchically and cooperatively learning traffic signal control. Proceedings of the AAAI Conference on Artificial Intelligence 35, 669–677. doi:10.1609/aaai.v35i1.16147. Ye, B.L., Wu, P., Li, L., Wu, W., Song, B., Zhang, X.,

    doi

  27. [58]

    IEEE Open Journal of Intelligent Transportation Systems 6, 1424–1438

    Deep reinforcement learning based on spatiotem- poral information for network-wide traffic signal coordination control. IEEE Open Journal of Intelligent Transportation Systems 6, 1424–1438. doi:10.1109/OJITS.2025.3627135. Ye, B.L., Wu, W., Mao, W.,

    2025

  28. [60]

    (NeurIPS), New Orleans, Louisiana, USA

    The surprising effectiveness of PPO in cooperative multi-agent games, in: Advances in Neural Information Processing Systems 35, Neural Information Processing Systems Foundation, Inc. (NeurIPS), New Orleans, Louisiana, USA. pp. 24611–24624. doi:10.52202/068431-1787. Zang, X., Y...

    doi

  29. [62]

    Transportation Research Part C: Emerging Technologies 148, 104033

    Scalable multi-region perimeter metering control for urban networks: A multi- agent deep reinforcement learning approach. Transportation Research Part C: Emerging Technologies 148, 104033. doi:10.1016/j.trc.2023.104033. Zhu, J., Ma, C., Shi, Y., Yang, Y., Guo, Y., Yang, X.,

    2023

  30. [63]

    Computer-Aided Civil and Infrastructure Engineering 40, 895–916

    Asynchronous decentralized traffic signal coor- dinated control in urban road network. Computer-Aided Civil and Infrastructure Engineering 40, 895–916. doi:10.1111/mice.13362. Zhu, Y., Wen, H., Min, G., Luo, M.,

    doi

  31. [64]

    HALO: Hierarchical reinforcement learning for large-scale adaptive traffic signal control, in: Proceedings of the ACM Web Conference 2026, ACM. pp. 5368–5379. doi:10.1145/ 3774904.3792352. 37

    2026

  32. [560]

    Aslani, M., Mesgari, M.S., Wiering, M.,

    doi:10.3390/machines14050560. Aslani, M., Mesgari, M.S., Wiering, M.,

    doi

  33. [1160]

    Zarrouki, B., Spanakakis, M., Betz, J., 2024a

    doi:10.1609/aaai.v34i01.5467. Zarrouki, B., Spanakakis, M., Betz, J., 2024a. A safe reinforcement learning driven weights-varying model predictive control for autonomous vehicle motion control, in: 2024 IEEE Intelligent Vehicles Symposium (IV), IEEE. pp. 1401–1408. doi:10.1109...

    2024

  34. [1306]

    Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., Li, Z.,

    doi:10.1038/s41467-024-45427-4. Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., Li, Z.,

    doi

  35. [1964]

    The Annals of Mathematical Statistics 35, 73–101

    Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics 35, 73–101. doi:10.1214/aoms/1177703732. Husch, D., Albeck, J.,

  36. [1983]

    Transportation Research Part A: General 17, 341–342

    Derivation of delays based on input-output analysis. Transportation Research Part A: General 17, 341–342. doi:10.1016/0191-2607(83)90001-8. de Witt, C.S., Gupta, T., Makoviichuk, D., Makoviychuk, V., Torr, P.H.S., Sun, M., Whiteson, S.,

    doi

  37. [1997]

    Transportation Science 31, 5–17

    Controlled optimization of phases at an intersection. Transportation Science 31, 5–17. doi:10.1287/trsc.31.1.5. Oliveira da Silva, C.F., Dabiri, A., De Schutter, B.,

    doi

  38. [2002]

    IEEE Transactions on Evolutionary Computation 6, 182–197

    A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 182–197. doi:10.1109/4235.996017. Devailly, F.X., Larocque, D., Charlin, L.,

  39. [2003]

    Proceedings of the IEEE 91, 2043–2067

    Review of road traffic control strategies. Proceedings of the IEEE 91, 2043–2067. doi:10.1109/JPROC.2003.819610. Pham, V.H., Ahn, H.S.,

    2003

  40. [2007]

    European Journal of Transport and Infrastructure Research doi:10.18757/ejtir.2007.7.3.3390

    Integrated traffic control for mixed urban and freeway networks: A model predictive control approach. European Journal of Transport and Infrastructure Research doi:10.18757/ejtir.2007.7.3.3390. Bøhn, E., Gros, S., Moe, S., Johansen, T.A.,

    2007 doi

  41. [2009]

    Transportation Research Part C: Emerging Technologies 17, 163–174

    Store-and-forward based methods for the signal control problem in large-scale congested urban road networks. Transportation Research Part C: Emerging Technologies 17, 163–174. doi:10.1016/j.trc.2008.10.002. Aboudolas, K., Papageorgiou, M., Kouvelas, A., Kosmatopoulos, E.,

    2008 doi

  42. [2010]

    Trans- portation Research Part C: Emerging Technologies 18, 681–694

    A rolling-horizon quadratic- programming approach to the signal control problem in large-scale congested urban road networks. Trans- portation Research Part C: Emerging Technologies 18, 681–694. doi:10.1016/j.trc.2009.06.003. Airaldi, F., De Schutter, B., Dabiri, A.,

    2009 doi

  43. [2013]

    Transportation Research Part C: Emerging Technologies 36, 177–195

    Max pressure control of a network of signalized intersections. Transportation Research Part C: Emerging Technologies 36, 177–195. doi:10.1016/j.trc.2013.08.014. Wang, H., Zhu, M., Hong, W., Wang, C., Tao, G., Wang, Y.,

    2013 doi

  44. [2014]

    Asian Journal of Control 17, 775–790

    Distributed model predictive control method for optimal coordination of signal splits in urban traffic networks. Asian Journal of Control 17, 775–790. doi:10.1002/asjc.1011. 36 Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., Wu, Y.,

    doi

  45. [2015]

    Transportation Research Part C: Emerging Technologies 55, 460–473

    A real-time adaptive signal control in a connected vehicle environment. Transportation Research Part C: Emerging Technologies 55, 460–473. doi:10. 1016/j.trc.2015.01.007. Fu, J., Sun, D., Peyghami, S., Blaabjerg, F.,

    2015

  46. [2016]

    Transportation Research Record: Journal of the Trans- portation Research Board 2595, 128–138

    Nonhomogeneous time mixed integer linear programming formulation for traffic signal control. Transportation Research Record: Journal of the Trans- portation Research Board 2595, 128–138. doi:10.3141/2595-14. Guo, G., Wang, Y.,

    doi

  47. [2017]

    Transportation Research Part C: Emerging Technologies 80, 272–285

    Distributed coordinated signal timing optimization in connected trans- portation networks. Transportation Research Part C: Emerging Technologies 80, 272–285. doi:10.1016/j. trc.2017.04.017. Allahloh, A.S., Ghaleb, A.M., Sarfraz, M., Alrashdan, A., Ali, M.A.H., Al-Shayea, A.,

    2017 doi

  48. [2018]

    Microscopic traffic simulation using SUMO, in: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), IEEE. pp. 2575–2582. doi:10.1109/itsc.2018. 8569938. Luo, L., Tan, C., Tang, K., Zhu, H., 2025a. A distributed model predictive approach for network ...

    2018

  49. [2019]

    Journal of Intelligent Transportation Systems 23, 370–388

    Multi-objective optimal predictive control of signals in urban traffic network. Journal of Intelligent Transportation Systems 23, 370–388. doi:10.1080/15472450.2018.1504294. Liu, S., Chen, J., Cheng, H., Jung, S., Kim, Y.B.,

    2018

  50. [2020]

    IEEE Robotics and Automation Letters 5, 2897–2904

    MPC-net: A first principles guided policy search. IEEE Robotics and Automation Letters 5, 2897–2904. doi:10.1109/lra.2020.2974653. Chen, C., Cao, Y., Tang, K., Li, K.,

    2020

  51. [2021]

    Journal of Advanced Transportation 2021, 8877138

    Dynamic path flow estimation using automatic vehicle identifi- cation and probe vehicle trajectory data: A 3D convolutional neural network model. Journal of Advanced Transportation 2021, 8877138. doi:10.1155/2021/8877138. Chen, C., Huang, Y.P., Lam, W.H.K., Pan, T.L., Hsu, S.C...

    2021 doi

  52. [2022]

    Transportation Research Part C: Emerging Technologies 142, 1–29

    Data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics. Transportation Research Part C: Emerging Technologies 142, 1–29. doi:10.1016/j.trc.2022.103759. Chen, C., Wei, H., Xu, N., Zheng, G., Yang, M., Xiong, Y., Xu, K., Li, Z.,

    2022

  53. [2023]

    Engineering Applications of Artificial Intelligence 123, 106211

    Optimizationofthemodelpredictivecontrolmeta-parameters through reinforcement learning. Engineering Applications of Artificial Intelligence 123, 106211. doi:10.1016/ j.engappai.2023.106211. Carius, J., Farshidian, F., Hutter, M.,

    2023

  54. [2024]

    IEEE Open Journal of Intelligent Transportation Systems 5, 238–250

    Model-based graph reinforcement learning for inductive traffic signal control. IEEE Open Journal of Intelligent Transportation Systems 5, 238–250. doi:10.1109/ojits. 2024.3376583. 33 Dowling, R.,

    2024

  55. [2025]

    IEEE Transactions on Intelligent Transportation Systems 26, 5988–6004

    Reinforcement learning with model predictive control for highway ramp metering. IEEE Transactions on Intelligent Transportation Systems 26, 5988–6004. doi:10.1109/TITS. 2025.3549227. Al Islam, S.B., Hajbabaie, A.,

    2025

  56. [2026]

    International Journal of Hydrogen Energy 216, 153914

    Hierarchical model predictive control with multi-agent reinforcement learning for eco-driving of fuel cell hybrid electric vehicles. International Journal of Hydrogen Energy 216, 153914. doi:10.1016/j.ijhydene.2026.153914. Liu, S., Cheng, H., Jung, S., Kim, Y.B.,

    2026

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.