{"id":"154a5c3c-5c35-4ff9-8cb7-7ef214a93d8f","arxiv_id":"2505.21818","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An adaptive dynamic programming controller makes a two-region macroscopic traffic network track a time-varying desired accumulation trajectory without knowing the traffic dynamics, beating fixed set-point control in simulation.","lead":"The paper proposes a learning-based traffic controller that makes a two-region city network follow a desired time-varying congestion trajectory, rather than aiming at a single fixed congestion target. The method learns its feedback rule without a calibrated traffic model and, in simulation, cuts total travel time by about 20% compared with a fixed-target controller.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Example 2's reference trajectory (Eq. 27) is not a fully specified four-dimensional tracking target and is asserted, without derivation, to maximize trip completion; the paper's optimality and robustness conclusions for the proposed TPC therefore lack support.","rationale":"Good-faith reading: the paper's contribution is an ADP/IRL solution to trajectory tracking for two-region MFD networks, with the central empirical support for optimality appearing in Example 2, which claims robust tracking of a reference that 'maximizes trip completion.' The weakest link in the chain is the reference itself. The command generator (27) is written for scalar totals nd_1 and nd_2, but the value function in (8)-(10) depends on the four-dimensional tracking error ed. No equations or initial conditions are given for nd_11, nd_12, nd_21, and nd_22; the ratios nd_21/nd_2 and nd_12/nd_1 in (27) therefore refer to quantities whose evolution is undefined. The optimality claim is also unsupported: tracking a nominal-demand trajectory is not equivalent to maximizing trip completion, and no formal connection is drawn. This matters because the headline claims of robust tracking and trip-completion maximization rest on Example 2; if the reference is arbitrary or ill-posed, the experiment demonstrates only that the IRL loop can track a contrived two-dimensional signal. The reader's verdict of CONDITIONAL is appropriate: the control-theoretic derivation of Algorithm 1 is a standard IRL argument and may be correct, but the empirical and optimality conclusions need the missing reference definition and a proper benchmark. I see no reason to move the verdict; the requested revisions in the reader's report address exactly this gap.","tokens_in":13125,"tokens_out":4946,"duration_ms":51606,"concrete_test":"Reproduce Example 2 with a fully specified four-dimensional reference: derive the missing OD-component equations (or set them consistently with demand shares and initial conditions), then solve the nominal-demand trip-completion maximization problem (e.g., by optimal control or dynamic programming over the calibrated two-region MFD model) and compare its optimal total-accumulation trajectory with the one generated by (27). If the (27) trajectory is not within a small tolerance of the optimum, or if no four-dimensional reference consistent with (27) exists, the claim that tracking (27) 'maximizes trip completion' is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of Section IV-B is that the adaptive TPC regulates the four-dimensional accumulation state n to a desired trajectory that 'maximizes trip completion' under nominal demand. For this to hold, the command generator (27) must define a complete four-dimensional reference nd = col(nd_11, nd_12, nd_21, nd_22), and the resulting path must be an actual optimizer of trip completion. Neither condition is met in the text. Equation (27) gives only two differential equations, for nd_1 and nd_2, while the tracking error ed = n - nd in (5) is four-dimensional; the OD components nd_12 and nd_21 appear as ratios in (27), but their own dynamics and initial values are never specified, and the stated initial conditions nd_1(0) = 0, nd_2(0) = 0 also leave the OD split undefined. The statement that tracking (27) 'maximizes trip completion' (Section IV-B) is asserted without proof; no argument shows that this trajectory is the trip-completion-maximizing one. Hence the robust-optimality conclusion of Example 2 is not established, even if the IRL feedback loop tracks a two-dimensional total-accumulation reference. This is not a numerical quibble: without a well-defined optimal reference, the claim that the controller tracks an optimal trajectory and thereby maximizes trip completion is untestable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends set-point perimeter control for a two-region macroscopic fundamental diagram (MFD) traffic system to an optimal tracking perimeter control problem (OTPCP). The authors formulate an augmented system composed of the tracking error and the reference trajectory, introduce a nonquadratic cost to handle actuator constraints, and propose an adaptive dynamic programming (ADP) algorithm based on integral reinforcement learning (IRL) that is claimed to solve the OTPCP without knowledge of the system dynamics. Numerical experiments compare the proposed tracking perimeter control (TPC) with set-point perimeter control (SPC) in a time-varying demand scenario, reporting a 20.01% reduction in total time spent and a 3.15% improvement in cumulative trip completion, and a second example is intended to demonstrate robust tracking of a time-varying reference that is asserted to maximize trip completion.","tokens_in":13363,"tokens_out":7281,"duration_ms":73590,"significance":"If the claims are validated, the paper makes a useful contribution by generalizing MFD perimeter control from equilibrium stabilization to trajectory tracking, which is better suited to time-varying demand. The IRL derivation and the equivalence proof in Proposition III.1 follow standard policy-iteration arguments and are internally consistent apart from notation slips. The use of a nonquadratic performance function for constrained inputs is appropriate, and the numerical comparison in Example 1 gives a concrete performance improvement over SPC. The main advertised strengths are the model-free learning formulation and the ability to track a time-varying reference under demand disturbances. However, the strongest version of these claims, presented in Example 2, is not supported as written because the reference trajectory is incompletely specified and its optimality is asserted rather than proved; the 'model-free' label also requires qualification.","major_comments":[{"comment":"The command generator (27) defines differential equations only for nd,1 and nd,2, the regional accumulations, whereas the tracking error ed = n - nd in Eq. (5) is four-dimensional and nd(t) was defined in Section II-B as col(nd_11, nd_12, nd_21, nd_22). The OD-specific reference components nd_12 and nd_21 enter (27) as ratios, but their dynamics and initial conditions are never specified, so (27) is not a well-posed four-dimensional reference. Consequently, Example 2 cannot be implemented as stated and the claim that the controller regulates the four-dimensional accumulation state to the desired reference trajectory is not testable. The authors should either specify a complete four-dimensional command generator or reformulate the tracking problem in terms of the two regional accumulations n1 and n2 and correspondingly reduce the dimension of ed.","section":"Section IV-B, Eq. (27)"},{"comment":"It is asserted without proof or citation that tracking (27) 'maximizes trip completion.' The right-hand side of (27) is the free-running accumulation dynamics under a nominal demand pattern; no argument shows that this trajectory maximizes cumulative trip completion or the integral of the trip completion rates. In fact, a free-running trajectory generally settles at a demand-determined equilibrium rather than at the trip-completion-maximizing accumulation, which is typically near the critical accumulation of the MFD. Since the optimality of the tracked reference is load-bearing for the robust-optimality conclusion of Example 2, an explicit optimality argument or a precise definition of the trip-completion-maximizing reference is required.","section":"Section IV-B"},{"comment":"The label 'model-free' overstates what Algorithm 1 actually achieves. The total perimeter control is u = us + µ, and the feedforward part us is computed from (6) or (7) using f(nd), s(nd), and θ(nd). If the system dynamics are unknown, us cannot be computed from these formulas. Furthermore, the policy improvement relation (17) still contains S(N); the text states that V(k+1) and D(k+1) are solved simultaneously from (21), but does not explain how S(N) is eliminated or approximated in the actor update. Please clarify the exact meaning of 'model-free' in this context and state which quantities are assumed known and which are learned from data.","section":"Section II-B and Section III"},{"comment":"The paper states that the convergence of the policy iteration sequence (V(k), µ(k)) to (V*, µ*) 'has been checked by [21]', but no convergence theorem or sufficient conditions (initial admissibility, excitation condition, approximation error bounds) are provided in this manuscript. Because Algorithm 1's optimality claim is central, the authors should either state the relevant convergence result with assumptions or give a precise reference to a theorem in [21] that covers the constrained, augmented-system setting used here.","section":"Section III, Proposition III.1 and Algorithm 1"}],"minor_comments":[{"comment":"In the second-to-last line of (20), the term '−(∇V(k))^T S(µ(k)−µ)' should be '−(∇V(k+1))^T S(µ(k)−µ)' to be consistent with the preceding line; also, the notation tanh^{-T}(υ/λ) is unusual and should be defined.","section":"Eq. (20)"},{"comment":"The proof divides (21) by Δt and takes the limit Δt → 0; this interchange requires differentiability of V(k+1) and continuity of the integrand. These regularity conditions are not stated.","section":"Proposition III.1 proof"},{"comment":"Equation (7) requires s(nd)^T s(nd) to be invertible, i.e., s(nd) must have full column rank; this assumption should be stated explicitly.","section":"Section II-A"},{"comment":"The legend labels in Figures 5 and 6 use abbreviated subscripts 'ntpc', 'nref', and 'nspc' that are not defined in the captions; please spell out the quantities being plotted.","section":"Figures 5 and 6"},{"comment":"Reference [31] is a preprint under review, and it is used to motivate MFD uncertainty; please provide a published version or additional details so that the reader can verify the claim.","section":"Reference [31]"},{"comment":"The notation nd,1 and nd,2 in (27) is inconsistent with the four-dimensional nd(t) defined earlier; clarify that nd,i = nd,i1 + nd,i2 and specify the relation between the regional reference accumulations and the OD-specific reference components.","section":"Eq. (27)"},{"comment":"The step 'According to the control policy D(k), D(k+1) and V(k+1) can be solved simultaneously' is presented as an equation, not as an implementable algorithm; please describe the function approximators and the least-squares or recursive least-squares procedure used to solve (21).","section":"Algorithm 1"},{"comment":"The abstract reports a 20.01% TTS reduction and a 3.15% CTC improvement without stating that these numbers are for a single scenario (Example 1); please qualify them accordingly and, ideally, provide multiple simulation runs or error bars.","section":"Abstract and Table II"}],"recommendation":"major_revision","confidential_remarks":"This is a revise-and-resubmit candidate. The core IRL derivation and the equivalence proposition are sound, and the problem studied is of interest to the traffic-control and ADP communities. However, the paper's strongest advertised result, Example 2, is not reproducible as written because the reference generator is incomplete and its trip-completion optimality is unproved; the 'model-free' claim also needs qualification. These are substantive but fixable issues, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a workmanlike extension of the authors' ADP perimeter-control work to trajectory tracking for two-region MFD networks. The constrained formulation (tracking plus IRL plus nonquadratic input cost, no linearization) is genuinely new relative to the cited traffic literature, and the IRL derivation in Proposition III.1 is standard and internally consistent, modulo a notation slip in the derivation of (20) where V(k) and V(k+1) get mixed.\n\nThat said, the strongest claims do not survive close reading. Example 2's reference trajectory (27) supplies differential equations only for the regional totals nd,1 and nd,2. The tracking error in (5) is four-dimensional, so the OD-specific components nd,12 and nd,21 are never defined as trajectories; the reader cannot tell what the controller is actually tracking. The paper asserts that this trajectory 'maximizes trip completion' without any derivation or even a stated optimization problem. That is a load-bearing gap: without a well-defined optimal reference, the robustness and optimality conclusions of Section IV-B are untestable. This is not a numerical quibble; it is a missing piece of the problem statement.\n\nTwo more soft spots. The 'model-free' label overstates: the feedforward term us in (6)/(7) uses the calibrated f and s at the reference trajectory. Only the feedback part of the controller is model-free. And the 20.01% TTS reduction is against a fixed set-point baseline that the paper's own cited literature would update in the face of changing demand; a fairer comparison would include an adaptive set-point scheme.\n\nThe core algorithmic idea—using IRL to solve a constrained tracking HJB equation without linearization—is plausible and grounded in external IRL results, so the paper is not circular. It deserves a serious referee, but a revision needs to define the reference trajectory completely, justify or drop the trip-completion optimality claim, compare against a non-strawman baseline, and release the simulation details (architecture, parameters, data). Without those, the experimental section is only illustrative.\n\nFor a reader in the MFD perimeter-control subfield, this is a useful formulation to know, but I would not rely on the numbers until the above is fixed.\n\nRecommendation: send to peer review, with major revisions expected.","headline":"A plausible extension of ADP to tracking perimeter control whose headline optimality and robustness claims rest on an under-specified reference trajectory.","tokens_in":14024,"tokens_out":4595,"would_cite":true,"duration_ms":44057,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["93C40","49L20","90B20"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that an adaptive dynamic programming controller can solve optimal tracking perimeter control for a two-region macroscopic fundamental diagram network without knowing the system dynamics, and that in simulation this…","keywords":["macroscopic fundamental diagram","perimeter control","adaptive dynamic programming","trajectory tracking","model-free control","integral reinforcement learning","two-region traffic network","trip completion"],"falsifier":"Run Example 2 again with several distinct four-dimensional reference trajectories that all reduce to the same aggregate accumulations $n_{d,1}$ and $n_{d,2}$ from (27), for example by reallocating the internal OD components. If the resulting tracking errors or cumulative trip completion differ appreciably, the claim that tracking (27) maximizes trip completion depends on reference components the paper never defines. A direct check is to compare the reported trip completion against a numerical search over admissible trajectories on the same demand pattern.","tokens_in":12809,"feed_emoji":"🚦","tokens_out":7402,"duration_ms":78371,"temperature":0.7,"pith_summary":"The paper tries to establish that perimeter control for a two-region macroscopic traffic network should regulate accumulation along a desired time-varying trajectory rather than toward a fixed set point, and that the resulting optimal tracking problem can be solved without knowing the system dynamics. The proposed adaptive dynamic programming algorithm learns the value function and control policy from state and control data through an integral reinforcement learning Bellman equation, which the authors prove equivalent to model-based policy iteration. If the claim holds, traffic managers could adapt perimeter metering to within-day demand swings without calibrating a macroscopic fundamental diagram model. In simulation the tracking controller reduces total time spent by 20.01% and increases cumulative trip completion by 3.15% relative to set-point perimeter control, and it keeps the accumulation state on a desired reference under demand disturbance.","feed_headline":"Model-free traffic control cuts travel time by 20 percent","feed_subtitle":"Perimeter metering learns to track a moving congestion target and outperforms fixed set-point control in simulation.","key_machinery":"The load-bearing object is the augmented system $N = \\mathrm{col}(e_d, n_d)$ with dynamics $\\dot{N} = F(N) + S(N)\\mu$, together with the nonquadratic performance function whose tanh integrand enforces the actuator bound $|\\mu_i| \\le \\lambda$. The tracking Hamilton-Jacobi-Bellman equation is solved iteratively through the integral reinforcement learning Bellman equation (21), which replaces $F(N)$ and $S(N)$ with an integral over data collected along the trajectory. Proposition III.1 proves that this data-driven update is equivalent to the model-based policy iteration, which is what lets the controller be model-free while still respecting input constraints and tracking a time-varying reference.","core_discovery":"The central claim is that the optimal tracking perimeter control problem for the two-region MFD system is solved by building an augmented system whose state combines the tracking error and the reference trajectory, then applying an integral reinforcement learning Bellman equation derived from a nonquadratic cost with tanh-saturated control. Proposition III.1 states that this IRL update is equivalent to model-based policy iteration, so the value function and control policy converge to the optimal tracking solution without knowledge of the drift and input dynamics. In the paper's simulations this yields 20.01% lower total time spent and 3.15% more cumulative trip completion than set-point perimeter control, and the controller keeps accumulation near a time-varying reference under demand noise. The paper also asserts that the reference trajectory used in Example 2 maximizes trip completion under the nominal demand pattern, so tracking it is presented as the practical traffic-management target.","pith_inferences":["The paper leaves the design of the reference trajectory largely open; defining it from economic or OD-specific objectives would be a natural next step and would also make the trip-completion optimality claim checkable.","The same augmented-system and IRL derivation does not use the two-region structure, so it should transfer to multi-region or trip-based MFD networks, though the convergence guarantee would need to be re-established.","The chattering in the metering inputs under demand noise suggests the learned policy could benefit from a control-rate penalty or a filter on the demand estimate; the paper does not test these modifications."],"forward_implications":["Calibrated MFD dynamics become unnecessary: the IRL Bellman equation collects data along the actual trajectory, so the same algorithm can be applied when the drift and input dynamics are unknown or time-varying.","Trajectory tracking changes the control objective from stabilizing one equilibrium to following a demand-fitted path, which the simulations show avoids unnecessary queueing when congestion is offset.","The nonquadratic tanh cost keeps the feedback correction inside the actuator limits, so the learned policy respects $0 \\le u_{ij} \\le 1$ without post-hoc clipping.","Because the policy is updated online from measured state and control data, the controller can compensate for demand disturbances that differ from the pattern used to design the reference."],"supporting_citations":[{"why":"supplies the two-region MFD flow-conservation dynamics that the tracking problem is built on.","marker":"[4]"},{"why":"provides the cubic MFD curve and the robust constrained-control setting used in the numerical experiments.","marker":"[9]"},{"why":"gives the affine form and the robust perimeter-control background with desired invariant set from which the tracking formulation departs.","marker":"[10]"},{"why":"supplies the control-systems analysis of MFD boundary conditions and the rationale for targeting the critical accumulation during peak periods.","marker":"[11]"},{"why":"establishes convergence of the policy-iteration sequence that Algorithm 1's IRL update is shown equivalent to.","marker":"[21]"},{"why":"supplies the command-generator reference dynamics and the steady-state plus feedback decomposition of the tracking controller.","marker":"[27]"},{"why":"provides the least-squares steady-state control used when the input dynamics inverse does not exist.","marker":"[28]"},{"why":"is the offline policy-iteration method for HJB equations on which the model-free algorithm is based.","marker":"[29]"},{"why":"is the open-source traffic simulator used to generate the microscopic data shown for MFD hysteresis and demand uncertainty.","marker":"[30]"}],"fun_headline_variants":["Adaptive tracking control cuts travel time 20%","Model-free perimeter control beats set-point by 20%","Learning control tracks moving congestion target","ADP-based traffic control improves trip completion 3.15%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Example 2 conclusions assume the reference trajectory (27) is fully specified and truly maximizes trip completion, but the paper defines it only for the two aggregate accumulations and does not prove the optimality claim.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive tracking control cuts travel time 20%","Model-free perimeter control beats set-point by 20%","Learning control tracks moving congestion target","ADP-based traffic control improves trip completion 3.15%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000441,"raw_usage":{"total_tokens":2245,"prompt_tokens":965,"completion_tokens":1280,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":1217}},"tokens_in":581,"tokens_out":1280,"duration_ms":11287,"temperature":1.0,"reasoning_tokens":1217,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:24:31.684848+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Example 2 again with several distinct four-dimensional reference trajectories that all reduce to the same aggregate accumulations $n_{d,1}$ and $n_{d,2}$ from (27), for example by reallocating the internal OD components. If the resulting tracking errors or cumulative trip completion differ appreciably, the claim that tracking (27) maximizes trip completion depends on reference components the paper never defines. A direct check is to compare the reported trip completion against a numerical search over admissible trajectories on the same demand pattern.","supporting_citations":[{"cited_title":"Optimal perimeter control for two urban regions with macroscopic fundamental diagrams: A model predictive approach,","cited_arxiv_id":null,"evidence_quote":"supplies the two-region MFD flow-conservation dynamics that the tracking problem is built on."},{"cited_title":"Robust constrained control of uncertain macroscopic fun- damental diagram networks,","cited_arxiv_id":null,"evidence_quote":"provides the cubic MFD curve and the robust constrained-control setting used in the numerical experiments."},{"cited_title":"Robust perimeter control for two urban regions with macroscopic fundamental diagrams: a control-lyapunov function approach,","cited_arxiv_id":null,"evidence_quote":"gives the affine form and the robust perimeter-control background with desired invariant set from which the tracking formulation departs."},{"cited_title":"Boundary conditions and behavior of the macroscopic fundamental diagram based network traffic dynamics: A control systems perspective,","cited_arxiv_id":null,"evidence_quote":"supplies the control-systems analysis of MFD boundary conditions and the rationale for targeting the critical accumulation during peak periods."},{"cited_title":"Data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics,","cited_arxiv_id":null,"evidence_quote":"establishes convergence of the policy-iteration sequence that Algorithm 1's IRL update is shown equivalent to."},{"cited_title":"Tracking control optimization scheme of continuous-time nonlinear system via online single network adaptive critic design method,","cited_arxiv_id":null,"evidence_quote":"supplies the command-generator reference dynamics and the steady-state plus feedback decomposition of the tracking controller."},{"cited_title":"Near-optimal output tracking controller design for nonlinear systems using an event-driven adp approach,","cited_arxiv_id":null,"evidence_quote":"provides the least-squares steady-state control used when the input dynamics inverse does not exist."},{"cited_title":"Reinforcement learning and adaptive dynamic programming for feedback control,","cited_arxiv_id":null,"evidence_quote":"is the offline policy-iteration method for HJB equations on which the model-free algorithm is based."}],"review_version":1}