Deep Reinforcement Learning for Dynamic Origin-Destination Matrix Estimation in Microscopic Traffic Simulations Considering Credit Assignment

Donggyu Min; Dong-Kyu Kim; SeongJin Choi

REVIEW 5 major objections 5 minor 44 references

Deep Reinforcement Learning for Dynamic Origin-Destination Matrix Estimation in Microscopic Traffic Simulations Considering Credit Assignment

T0 review · 5 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reinforcement learning can estimate dynamic origin-destination matrices in microscopic traffic simulation so accurately that reproduced link flows are statistically indistinguishable from ground truth.

desk verdict A plausible toy experiment that is structurally well-matched to its own data generator, with an abstract that overclaims the results. read the letter →

arxiv 2511.06229 v3 pith:5AJNRQVD submitted 2025-11-09 cs.LG

Donggyu Min , Seongjin Choi , Dong-Kyu Kim This is my paper · ORCID

classification cs.LG

keywords dynamicorigin-destinationmatrixestimationmicroscopictrafficsimulationreinforcementlearningPPOcreditassignmentlinkflowsMDPSUMO

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that dynamic origin-destination matrix estimation (DODE) in microscopic traffic simulation is best treated as a sequential decision problem, not a one-shot optimization. It models the problem as a Markov decision process in which the agent decides every five seconds, for each OD pair, whether to dispatch a vehicle, and learns this policy with multi-binary PPO interacting directly with the simulator. The central claim, demonstrated on the Nguyen-Dupuis network with SUMO, is that the learned policy reproduces ground-truth five-minute link flows with an MSE 43.2% lower than the best conventional simultaneous Bayesian-optimization baseline and that its outputs are statistically indistinguishable from data generated by the true demand. A sympathetic reader would care because this offers a way to sidestep the credit-assignment problem—the difficulty of attributing observed link flows to time-lagged OD inputs—that plagues conventional bi-level and sequential calibration methods. The claim is conditional on a toy scenario whose ground-truth demand is generated as independent Bernoulli draws, so the result is a proof-of-mechanism rather than a field demonstration.

What carries the argument

The central machinery is the multi-binary PPO agent: an actor-critic network whose policy is a factorized Bernoulli distribution over per-OD-pair dispatch decisions at each 5-second step. The critic's value estimates feed generalized advantage estimation (GAE), which propagates the sparse, 5-minute-aggregated link-flow reward back to the fine-grained actions that caused it—this credit-assignment mechanism is what sets the method apart from baselines. The state is a fixed-dimension vector of link vehicle counts and average speeds plus a context of current timestep and recent detector counts, keeping the MDP Markovian enough to train without a belief state.

What would settle it

Sample a different ground-truth generation process on the same Nguyen-Dupuis network—e.g., Poisson arrivals with a time-varying rate shared across OD pairs, or route choice depending on congestion—and run the identical multi-binary PPO training. If the learned policy's link-flow MSE then clearly exceeds the stochastic noise floor of the true-demand process, the claim that the method solves credit assignment generally is falsified. Alternatively, in the current setup, a policy that simply samples independent Bernoulli draws from the known true rates should achieve an MSE at the true-demand nois

Watch

Extended reading notes

Core claim

The paper's core discovery is that a model-free deep RL policy can perform fine-grained DODE in a stochastic microscopic simulator at a decision resolution (5-second actions) that defeats conventional simultaneous and sequential optimization methods. By decomposing the OD matrix into per-pair Bernoulli decisions and using PPO with generalized advantage estimation, the agent learns to assign credit across the long, stochastic time lag between a departure decision and its effect on detector counts. The resulting demand trajectory yields link flows that are not just closer to the observed data than the baselines (MSE reduction of 43.2% versus ST-BO 5min) but statistically indistinguishable from

Load-bearing premise

The load-bearing premise is that the true demand process is independent Bernoulli per OD pair per 5-second step, so the factorized Bernoulli policy can represent the true data-generating distribution; if real demand has correlations across pairs, time-of-day trends, or a different arrival process, the paper gives no evidence the learned policy or its credit-assignment mechanism transfers.

Editorial extensions

If this is right

If the central claim holds, DODE in microscopic simulation can be performed at a decision resolution (seconds) that is normally considered intractable, because the RL agent learns a policy rather than searching the joint input space.
The method removes the need for an explicit dynamic assignment model or an analytical mapping from OD to link flows; the policy learns that mapping implicitly from simulation interactions.
The framework's success depends on the action space being factorizable into independent binary decisions per OD pair; networks with strongly correlated demand across OD pairs or modes may require richer policy parameterizations.
The paper's statistical tests suggest that, on this toy network, the residual error of RL-PPO is at the noise floor: it matches the stochastic variability of the simulator itself under true demand, so further accuracy gains would have to come from reducing simulation noise, not better demand estimation.
Because the policy is trained for one specific network, detectors, and simulation setup, each new network requires retraining; no transfer across scenarios is demonstrated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The abstract reports a 23.7% toy MSE reduction while the body reports 43.2%, and the abstract's real-world case-study numbers (59.2–88.3%) are not backed by the body; reconciling these numbers would clarify the paper's headline claim.
The factorized Bernoulli policy class exactly matches the assumed Bernoulli ground-truth demand generation, which means the toy result may overstate performance relative to real demand that has temporal correlation or time-of-day structure; a test with non-Bernoulli or correlated demand would clarify the generality.
The approach implicitly trades computation during training for computation during calibration: training the PPO agent requires many simulation episodes, which may be acceptable for offline calibration of a fixed network but could be prohibitive for frequently re-calibrated large networks.
A natural next experiment would be to compare against a 'true-demand sampler' baseline—a policy that emits independent Bernoulli draws with the known true rates—to see whether the learned policy achieves the same link-flow MSE; if it does not, there is room to improve the credit-assignment mechanism, and if it does, the method is essentially recovering the true distribution.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look if you work on demand calibration. The paper applies PPO with a factorized Bernoulli action head to dynamic OD estimation in a SUMO microscopic simulation on the Nguyen–Dupuis network. The toy results are coherent: the RL policy fits the 5-minute link-flow counts better than simultaneous and sequential Bayesian-optimization baselines (roughly 43% MSE reduction), and the demand-recovery check in Fig. 9 is a nice diagnostic that goes beyond the link-flow objective. That is a legitimate empirical data point.

Now the soft spots. First, the abstract and the body disagree. The abstract claims a Santa Clara/San José case study with 59.2–88.3% improvements; the body has no such experiment, and the body's toy improvement is 43.2%, not 23.7%. That is a serious reporting error and has to be fixed before anyone can trust the rest.

Second, and more fundamental: the experiment is structured so that the true data generator belongs to the policy class. Ground-truth demand is generated by independent Bernoulli draws per OD pair per 5-second step, and the agent's action is exactly a factorized Bernoulli decision over those same binary choices. The optimal policy is a constant vector of probabilities; there is no actual dynamic complexity for the MDP or credit assignment to resolve. The BO baselines were given continuous flow-rate variables, so they were never in the same hypothesis space. That doesn't make the 43% number fake, but it does mean the claim that the method works because of DRL's credit assignment is unsupported. The improvement could be entirely due to choosing an action representation that matches the data generator.

Also: the real-world case study is missing from the body, the statistical tests are nine uncorrected detector-wise comparisons, and no code or hyperparameters are released. All are addressable. The conclusion's limitation statement is honest about needing real-world validation.

Who is this for? Traffic simulation modelers who want to see whether DRL can be dropped into a calibration loop. If the authors fix the reporting and add a binary-space BO or a simple maximum-likelihood baseline, the toy result becomes a useful comparison. As it stands, I'd treat the central claim as unproven in the paper's own terms, but the direction is plausible.

Recommendation: send it to a serious referee, because the experiment is reproducible in principle and the application is reasonably novel, but expect major revision. The issues are large but not load-bearing in the sense that they can be fixed.

Referee Report

5 major / 5 minor

Summary. The paper proposes a model-free deep reinforcement learning (DRL) framework for dynamic origin-destination matrix estimation (DODE) in microscopic traffic simulation. The authors formulate DODE as a Markov decision process: the state is a concatenation of link-level network state and detector data context; the action is a binary dispatch decision for each OD pair every 5 seconds, parameterized as a factorized Bernoulli policy; the reward is the negative link-flow MSE aggregated every 5 minutes. They solve this with multi-binary PPO and GAE. Experiments on the Nguyen-Dupuis network in SUMO compare RL-PPO with Bayesian-optimization simultaneous and sequential baselines at 5-minute and 5-second input intervals. The body reports a 43.2% MSE reduction versus the best baseline and claims that the proposed method's link flows are statistically indistinguishable from ground truth, while a direct OD-demand comparison also shows lower error. The abstract additionally claims a real-world Santa Clara / San Jose case study with 59.2–88.3% improvements, but this case study is absent from the manuscript body.

Significance. The paper addresses a practically important and technically difficult problem: calibrating stochastic microscopic traffic simulators through dynamic OD demand estimation. The MDP/DRL framing with a multi-binary action head and GAE-based credit assignment is a reasonable and scalable design, and the toy experiment is internally coherent, including multiple error metrics, a direct OD-demand evaluation, and a comparison to BO baselines. If the body-level results hold, the approach is a promising alternative to conventional optimization-based DODE on small networks. However, the central claim about resolving the credit-assignment problem is not yet convincingly supported: the ground-truth demand is generated by exactly the same Bernoulli process that the policy class represents, so the performance gap may reflect a structural match rather than a learned sequential mechanism. Moreover, the abstract promises a real-world case study that is absent from the body, and the statistical equivalence claim is based on an uncorrected per-detector analysis with very small sample sizes. The contribution is therefore best viewed, at this stage, as a well-matched synthetic demonstration rather than a

major comments (5)

[Abstract / Section 5] The abstract claims a real-world case study on a Santa Clara / San Jose highway subnetwork with 59.2–88.3% improvements, but the manuscript body contains only the Nguyen-Dupuis toy experiment. Section 5 explicitly states that validation on realistic-scale networks and real-world data is future work. This is a substantial inconsistency in the paper's central claims and must be corrected.
[Sec. 3.4, 3.2.4, 3.3] The ground-truth demand is generated by independent probabilistic departure decisions per OD pair at every 5-second step (Sec. 3.4), while the action is a binary dispatch decision per OD pair per step (Sec. 3.2.4) modeled with a factorized Bernoulli policy (Sec. 3.3). Thus the true data-generating process is a member of the policy class: an agent setting constant Bernoulli logits to the true departure probabilities exactly represents the generating distribution, and the MDP state and credit assignment are unnecessary in principle. The 43.2% improvement over BO baselines may therefore be attributable to action-space matching rather than to the sequential credit-assignment mechanism. The paper should test with ground truth generated outside this class (e.g., correlated or time-varying probabilities, continuous flows) or include a simple direct estimator of the Bernoulli marginals as a base
[Sec. 4 / Fig. 8] The claim that RL-PPO is 'statistically indistinguishable from ground truth' is based on 9 separate paired tests, one per detector, with no correction for multiple comparisons and only 6 observations per detector. Failing to reject the null hypothesis does not demonstrate equivalence, and with n=6 the tests are severely underpowered. A proper equivalence test (TOST) or simultaneous confidence intervals is needed before this claim can be made.
[Sec. 4 / Fig. 7] The reported performance metrics, including the headline MSE of 30.69 (veh/5min)^2, appear to be selected from the best of 5 random seeds ('the points with the lowest error were primarily considered'). No confidence intervals or distribution of outcomes across seeds are reported for the main comparison. Best-of-seed selection optimistically biases the comparison; the authors should report all runs or provide seed-averaged results with variance.
[Abstract / Sec. 4] The abstract reports a 23.7% improvement in the toy experiment, while the body reports a 42.0% reward improvement and a 43.2% MSE reduction. The body values are consistent with the reported MSEs (ST-BO 5min: 54.04; RL-PPO: 30.69), but the abstract number is unexplained and appears incorrect. This must be reconciled.

minor comments (5)

[Table 1] Table 1 is referenced ('notations are summarized in Table 1'), but the actual content of the table is not provided in the manuscript. Please include the notation table.
[Eq. (2), (3)] The vectors d'_k and d_k are used before being defined; please define the detector-flow vector. Also define the stochastic simulator map F and the mapping functions ψ and φ for a general k; the current notation 'M = ψ(k=1) = {t ∈ Z | 1 ≤ t ≤ 60}' is confusing.
[Sec. 3.4] The ground-truth generation says a total of 300 vehicles were created over 30 minutes, but also says a departure decision was made probabilistically for each OD pair at each 5-second step. Please clarify how these are consistent (e.g., are the Bernoulli probabilities scaled to yield an expected total of 300?).
[Sec. 4, True demand baseline] The 'true demand' baseline uses a different random seed and therefore has nonzero MSE due to microscopic stochasticity. This is a legitimate point, but the text should clarify that this does not measure demand-estimation error, only simulation noise.
[Fig. 5] The description 'dotted lines show the average maximum reward for each methodology' is ambiguous: is this the average over seeds of the maximum reward per seed, or the maximum over training of the average reward per seed? Please specify.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the shared reward/evaluation metric is an objective alignment, not a circular derivation.

full rationale

Walking the derivation chain: Sec. 3.2.2 defines DODE as minimizing L(a)=||d'-d||^2 (Eq. 2), and Sec. 3.2.6 sets the RL reward to the negative of the same error (Eq. 3). This is an alignment of objective with evaluation metric, not a circular reduction: the optimization can fail, as the BO baselines, evaluated on the same metric, show. Sec. 3.4 generates ground truth by Bernoulli departures per OD pair per 5-second step, while Sec. 3.2.4 and Sec. 3.3 use binary Bernoulli actions per OD pair per step; this is a favorable model-class match but does not make the empirical result true by construction—the agent still estimates probabilities from noisy link counts, and OD-demand recovery (Fig. 9) is not part of the reward. The only self-citation ([37], a previous DNN OD-estimation paper) appears in a literature-review list of surrogate-model work and is not load-bearing. The abstract's real-world 59.2-88.3% reduction claims are absent from the body and conflict with the conclusion's statement that 'validation using real-world data is necessary'; this is a reporting inconsistency, not circularity. No fitted parameter is renamed as a prediction, and no uniqueness/ansatz result is imported from the authors' prior work.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several domain assumptions about Markov structure, state sufficiency, the expressiveness of the factored Bernoulli policy, and the fidelity of SUMO as a stand-in for reality. The strongest circularity is that the ground-truth demand is generated by the same distribution family the agent uses, making the toy problem well-matched to the model class. All RL hyperparameters besides gamma are unstated, adding uncontrolled degrees of freedom.

free parameters (6)

Discount factor gamma = 0.995
Set in Sec. 3.2.7 'to improve the performance of the proposed method and ensure convergence'; controls the effective credit-assignment horizon.
Link length scaling factor = 3
Sec. 3.4: 'we multiplied all link lengths by a factor of 3' so that OD influence is evident over a wide time range; alters the toy network's dynamics.
Ground-truth total vehicles = 300
Sec. 3.4: the ground-truth dataset was generated by creating 300 vehicles over 30 min; the difficulty of the calibration task depends on this choice.
Per-OD Bernoulli departure probabilities for ground truth
Sec. 3.4: 'a departure decision was made probabilistically for each OD pair'; these probabilities define the target OD demand but are not reported in the paper.
Input interval / action timestep = 5 s
Sec. 3.2.4: chosen to 'reduce the probability of unobservable events'; this granularity is central to the method's claimed advantage.
PPO training hyperparameters (learning rate, clip epsilon, entropy coefficient, GAE lambda, network architecture, batch
Not reported anywhere in the manuscript; the experimental results depend on them.

assumptions (5)

domain assumption The traffic dynamics can be modeled as a discrete-time Markov process (the Markov property is a reasonable approximation)
Sec. 3.1: 'allows the Markov property ... to serve as a reasonable approximation of real-world traffic dynamics.'
domain assumption The proposed state (link vehicle counts, average link speeds, current timestep, detector accumulations) is sufficient for optimal decision-making
Sec. 3.2.3: the state is defined by these measurements; no argument establishes sufficiency beyond intuition.
domain assumption A factorized Bernoulli policy can represent the optimal or true demand-generation process
Sec. 3.3: the policy samples each OD pair independently; the ground-truth is also independent Bernoulli draws, so this assumption holds by construction in the toy experiment.
domain assumption SUMO's default car-following and 5-second shortest-path route choice reproduce the behavior that generated the observed data
Sec. 3.4: the same simulator and assignment logic produce both ground-truth and simulated output; no real-world validation is provided.
domain assumption The DODE problem is identifiable from aggregate 5-minute link flows; minimizing link-flow MSE recovers the true OD demand
Eq. 2 defines the objective only in terms of link flows; Sec. 4 (Fig. 9) interprets the fitted OD demand as an estimate of the true OD without an identifiability analysis.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Deep Reinforcement Learning for Dynamic Origin-Destination Matrix Estimation in Microscopic Traffic Simulations Considering Credit Assignment." pith.science (2026). https://pith.science/paper/5AJNRQVD

@misc{pith2026251106229,
  author       = {Pith},
  title        = {Pith review of: Deep Reinforcement Learning for Dynamic Origin-Destination Matrix Estimation in Microscopic Traffic Simulations Considering Credit Assignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5AJNRQVD}},
  note         = {Machine review of arXiv:2511.06229}
}

read the original abstract

This paper focuses on dynamic origin-destination matrix estimation (DODE), a crucial calibration process necessary for the effective application of microscopic traffic simulations. The fundamental challenge of the DODE problem in microscopic simulations stems from the complex temporal dynamics and inherent uncertainty of individual vehicle dynamics. This makes it highly challenging to precisely determine which vehicle traverses which link at any given moment, resulting in intricate and often ambiguous relationships between origin-destination (OD) matrices and their contributions to resultant link flows. This phenomenon constitutes the credit assignment problem, a central challenge addressed in this study. We formulate the DODE problem as a Markov Decision Process (MDP) and propose a novel framework that applies model-free deep reinforcement learning (DRL). Within our proposed framework, the agent learns an optimal policy to sequentially generate OD matrices, refining its strategy through direct interaction with the simulation environment. This approach was evaluated through a toy experiment on the Nguyen-Dupuis network and a case study utilizing an actual highway subnetwork spanning Santa Clara and San Jose. Experimental results show that the proposed method consistently improves calibration performance relative to the strongest conventional baseline, reducing link-flow MSE by 23.7% in the toy experiment and by 59.2-88.3% in the real-world case study. By reframing DODE as a sequential decision-making problem, our approach addresses the credit assignment challenge through a learned policy and provides a novel framework for calibration of microscopic traffic simulations.

Figures

Figures reproduced from arXiv: 2511.06229 by the authors.

**Figure 2.** Overall framework of the proposed method [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** State description therefore, the dimension of the network state is 24, and the dimension of the context state is 3. Therefore, the dimension of the state is 27. 3.2.4 Action In the proposed method, the input interval is set to a short interval of 5 seconds. This reduces the probability of unobservable events occurring between the current state and the next state, thereby improving the stability of the MDP framework.… view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Nguyen and Dupuis network TABLE 2 Ground-truth Detector Data rows and 9 columns. Second, determine the total number of cars that will be input into the target network during the defined analysis period. To generate the ground-truth data, a total of 300 vehicles were cr…

**Figure 5.** Figure 5: Reward graph of the proposed method containing maximum [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Scatter plots of detector data using estimated demand by method [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Comparison of multi-metric performance and stability across methods [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: p-values from tests of each method’s link flow deviations from true demand 5 CONCLUSION This study proposes a reinforcement learning-based approach to address the credit assignment problem, a persistent issue in DODE for microscopic traffic simulations. We address th…

**Figure 9.** Figure 9: Comparison of estimated versus true OD demands across different methods [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

**Figure 10.** Figure 10: Heatmap showing the values of the true OD matrix and the [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 6 linked inside Pith

[1]

Diakaki, V

Papageorgiou, M., C. Diakaki, V . Dinopoulou, A. Kotsialos, and Y. Wang. Review of Road Traffic Control Strategies.Proceedings of the IEEE, 2003. 91(12):2043–2067

2003
[2]

Li, and J

Ghiasi, A., X. Li, and J. Ma. A Mixed Traffic Speed Harmoniza- tion Model with Connected Autonomous Vehicles.Transportation Research Part C: Emerging Technologies, 2019. 104:210–233

2019
[3]

Xie, D. F., Z. Z. Fang, B. Jia, and Z. He. A Data-Driven Lane- Changing Model Based on Deep Learning.Transportation Research Part C: Emerging Technologies, 2019. 106:41–60

2019
[4]

Li, J., C. Yu, Z. Shen, Z. Su, and W. Ma. A Survey on Urban Traffic Control Under Mixed Traffic Environment with Connected Automated Vehicles.Transportation Research Part C: Emerging Tech- nologies, 2023. 154:104258

2023
[5]

Toledo, T., M. E. Ben-Akiva, D. Darda, M. Jha, and H. N. Kout- sopoulos. Calibration of Microscopic Traffic Simulation Mod- els with Aggregate Data.Transportation Research Record, 2004. 1876(1):10–19

2004
[6]

Markou, and C

Papathanasopoulou, V ., I. Markou, and C. Antoniou. Online Cal- ibration for Microscopic Traffic Simulation and Dynamic Multi- Step Prediction of Traffic Speed.Transportation Research Part C: Emerging Technologies, 2016. 68:144–159

2016
[7]

Patwary, A. U., W. Huang, and H. K. Lo. Metamodel-Based Calibration of Large-Scale Multimodal Microscopic Traffic Simu- lation.Transportation Research Part C: Emerging Technologies, 2021. 124:102859

2021
[8]

Dynamic Origin–Destination Matrix Calibration for Large-Scale Network Simulators.Transportation Research Part C: Emerging Technologies, 2019a

Osorio, C. Dynamic Origin–Destination Matrix Calibration for Large-Scale Network Simulators.Transportation Research Part C: Emerging Technologies, 2019a. 98:186–206

Show all 44 references

[9]

Huo, J., C. Liu, J. Chen, Q. Meng, J. Wang, and Z. Liu. Simulation- Based Dynamic Origin–Destination Matrix Estimation on Free- ways: A Bayesian Optimization Approach.Transportation Research Part E: Logistics and Transportation Review, 2023. 173:103108

2023
[10]

Sasaki, Y

Yang, H., T. Sasaki, Y. Iida, and Y. Asakura. Estimation of Ori- gin–Destination Matrices from Link Traffic Counts on Congested Networks.Transportation Research Part B: Methodological, 1992. 26(6):417–434

1992
[11]

Tavana, H.Internally Consistent Estimation of Dynamic Network Origin–Destination Flows from Intelligent Transportation Systems Data Using Bi-Level Optimization. Ph.D. dissertation. University of Texas at Austin, Austin, Tex., 2001. PREPRINT 11

2001
[12]

Montero, J

Ros-Roca, X., L. Montero, J. Barcel ´o, K. N ¨okel, and G. Gen- tile. A Practical Approach to Assignment-Free Dynamic Ori- gin–Destination Matrix Estimation Problem.Transportation Re- search Part C: Emerging Technologies, 2022. 134:103477

2022
[13]

Antoniou, M

Balakrishna, R., C. Antoniou, M. Ben-Akiva, H. N. Koutsopoulos, and Y. Wen. Calibration of Microscopic Traffic Simulation Models: Methods and Application.Transportation Research Record, 2008. 1999:198–207

2008
[14]

Osorio.Efficient Offline Calibration of Ori- gin–Destination (Demand) for Large-Scale Stochastic Traffic Models

Zhang, C., and C. Osorio.Efficient Offline Calibration of Ori- gin–Destination (Demand) for Large-Scale Stochastic Traffic Models. MIT Tech. Rep., Massachusetts Institute of Technology, Cam- bridge, Mass., 2017

2017
[15]

Kesting.Traffic Flow Dynamics

Treiber, M., and A. Kesting.Traffic Flow Dynamics. Springer, Berlin, 2013

2013
[16]

Steps toward Artificial Intelligence.Proceedings of the IRE, 1961

Minsky, M. Steps toward Artificial Intelligence.Proceedings of the IRE, 1961. 49: 8-30

1961
[17]

Ferret, M

Pignatelli, E., J. Ferret, M. Geist, T. Mesnard, H. van Has- selt, O. Pietquin, and L. Toni.A Survey of Temporal Credit As- signment in Deep Reinforcement Learning. arXiv preprint, 2023. arXiv:2312.01072

2023 arXiv
[18]

Joe, W. and H. C. Lau. Deep Reinforcement Learning Approach to Solve Dynamic Vehicle Routing Problem with Stochastic Cus- tomers.Proceedings of the International Conference on Automated Planning and Scheduling, 2020. 30:394-402

2020
[19]

Arulkumaran, K., M. P . Deisenroth, M. Brundage, and A. A. Bharath.A Brief Survey of Deep Reinforcement Learning. arXiv preprint, 2017. arXiv:1708.05866

2017 arXiv
[20]

Wolski, P

Schulman, J., F. Wolski, P . Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms. arXiv preprint, 2017. arXiv:1707.06347

2017 arXiv
[21]

Moritz, S

Schulman, J., P . Moritz, S. Levine, M. Jordan, and P . Abbeel. High-dimensional Continuous Control Using Generalized Advantage Estimation. arXiv preprint, 2015. arXiv:1506.02438

2015 arXiv
[22]

Willumsen, L. G. Estimation of an OD Matrix from Traffic Counts – A Review. Working Paper. Institute of Transport Studies, Uni- versity of Leeds, Leeds, UK, 1978

1978
[23]

Cascetta, E., and S. Nguyen. A Unified Framework for Estimating or Updating Origin/Destination Matrices from Traffic Counts. Transportation Research Part B: Methodological, 1988. 22(6):437–455

1988
[24]

Cremer, M., and H. Keller. A New Class of Dynamic Methods for the Identification of Origin–Destination Flows.Transportation Research Part B: Methodological, 1987. 21(2):117–132

1987
[25]

Inaudi, and G

Cascetta, E., D. Inaudi, and G. Marquis. Dynamic Estimators of Origin–Destination Matrices Using Traffic Counts.Transportation Science, 1993. 27(4):363–373

1993
[26]

Ashok, K., and M. E. Ben-Akiva. Alternative Approaches for Real-Time Estimation and Prediction of Time-Dependent Ori- gin–Destination Flows.Transportation Science, 2000. 34(1):21–36

2000
[27]

Bierlaire, M., and F. Crittin. An Efficient Algorithm for Real- Time Estimation and Prediction of Dynamic OD Tables.Operations Research, 2004. 52(1):116–127

2004
[28]

F., and J

Bard, J. F., and J. T. Moore. A Branch and Bound Algorithm for the Bilevel Programming Problem.SIAM Journal on Scientific and Statistical Computing, 1990. 11(2):281–292

1990
[29]

Gu, and M

Shafiei, S., Z. Gu, and M. Saberi. Calibration and Validation of a Simulation-Based Dynamic Traffic Assignment Model for a Large- Scale Congested Network.Simulation Modelling Practice and Theory,
[30]

Maher, M. J., X. Zhang, and D. Van Vliet. A Bi-Level Programming Approach for Trip Matrix Estimation and Traffic Control Problems with Stochastic User Equilibrium Link Flows.Transportation Re- search Part B: Methodological, 2001. 35(1):23–40

2001
[31]

Lu, L., Y. Xu, C. Antoniou, and M. Ben-Akiva. An Enhanced SPSA Algorithm for the Calibration of Dynamic Traffic Assignment Models.Transportation Research Part C: Emerging Technologies, 2015. 51:149–166

2015
[32]

Rao, W., Y. J. Wu, J. Xia, J. Ou, and R. Kluger. Origin–Destination Pattern Estimation Based on Trajectory Reconstruction Using Au- tomatic License Plate Recognition Data.Transportation Research Part C: Emerging Technologies, 2018. 95:29–46

2018
[33]

Van Lint, T

Krishnakumari, P ., H. Van Lint, T. Djukic, and O. Cats. A Data Driven Method for OD Matrix Estimation.Transportation Research Part C: Emerging Technologies, 2020. 113:38–56

2020
[34]

Khoshkhah, and A

Pourmoradnasseri, M., K. Khoshkhah, and A. Hadachi. Lever- aging IoT Data Stream for Near-Real-Time Calibration of City- Scale Microscopic Traffic Simulation.IET Smart Cities, 2023. 5(4):269–290

2023
[35]

Tang, J., Y. Wang, C. Hu, Z. Li, and X. Zhang. A Spectral Clus- tering Enabled SPSA Algorithm for Dynamic Origin–Destination Demand Matrix Estimation.Transportmetrica B: Transport Dynamics,
[36]

Osorio, C. High-Dimensional Offline Origin–Destination (OD) Demand Calibration for Stochastic Traffic Simulators of Large- Scale Road Networks.Transportation Research Part B: Methodolog- ical, 2019b. 124:18–43
[37]

Min, D., H. Yun, S. W. Ham, and D. K. Kim. Real-Time Es- timation of Origin–Destination Matrices Using a Deep Neural Network for Digital Twins.Transportation Research Record, 2024. doi: 03611981241266837

2024
[38]

Value-function Approximations for Partially Ob- servable Markov Decision Processes.Journal of artificial intelligence research, 2000

Hauskrecht, M. Value-function Approximations for Partially Ob- servable Markov Decision Processes.Journal of artificial intelligence research, 2000. 13: 33-94

2000
[39]

Liang, and N

Shi, M., Y. Liang, and N. Shroff.Theoretical Hardness and Tractability of POMDPs in RL with Partial Online State Information. arXiv preprint, 2023. arXiv:2306.08762

2023
[40]

Farazi, N. P ., B. Zou, T. Ahamed, and L. Barua. Deep reinforcement Learning in Transportation Research: A Review.Transportation Research Interdisciplinary Perspectives, 2021. 11: 100425

2021
[41]

Kavukcuoglu, D

Mnih, V ., K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller.Playing Atari with Deep Reinforce- ment Learning. arXiv preprint, 2013. arXiv:1312.5602

2013 arXiv
[42]

Nguyen, S., and C. Dupuis. An Efficient Method for Computing Traffic Equilibria in Networks with Asymmetric Transportation Costs.Transportation Science, 1984. 18(2):185–202

1984
[43]

Springer, New York, 2010

Barcel ´o, J.Fundamentals of Traffic Simulation. Springer, New York, 2010

2010
[44]

I.A Tutorial on Bayesian Optimization

Frazier, P . I.A Tutorial on Bayesian Optimization. arXiv preprint arXiv:1807.02811, 2018

2018 arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

Diakaki, V

Papageorgiou, M., C. Diakaki, V . Dinopoulou, A. Kotsialos, and Y. Wang. Review of Road Traffic Control Strategies.Proceedings of the IEEE, 2003. 91(12):2043–2067

2003

[2] [2]

Li, and J

Ghiasi, A., X. Li, and J. Ma. A Mixed Traffic Speed Harmoniza- tion Model with Connected Autonomous Vehicles.Transportation Research Part C: Emerging Technologies, 2019. 104:210–233

2019

[3] [3]

Xie, D. F., Z. Z. Fang, B. Jia, and Z. He. A Data-Driven Lane- Changing Model Based on Deep Learning.Transportation Research Part C: Emerging Technologies, 2019. 106:41–60

2019

[4] [4]

Li, J., C. Yu, Z. Shen, Z. Su, and W. Ma. A Survey on Urban Traffic Control Under Mixed Traffic Environment with Connected Automated Vehicles.Transportation Research Part C: Emerging Tech- nologies, 2023. 154:104258

2023

[5] [5]

Toledo, T., M. E. Ben-Akiva, D. Darda, M. Jha, and H. N. Kout- sopoulos. Calibration of Microscopic Traffic Simulation Mod- els with Aggregate Data.Transportation Research Record, 2004. 1876(1):10–19

2004

[6] [6]

Markou, and C

Papathanasopoulou, V ., I. Markou, and C. Antoniou. Online Cal- ibration for Microscopic Traffic Simulation and Dynamic Multi- Step Prediction of Traffic Speed.Transportation Research Part C: Emerging Technologies, 2016. 68:144–159

2016

[7] [7]

Patwary, A. U., W. Huang, and H. K. Lo. Metamodel-Based Calibration of Large-Scale Multimodal Microscopic Traffic Simu- lation.Transportation Research Part C: Emerging Technologies, 2021. 124:102859

2021

[8] [8]

Dynamic Origin–Destination Matrix Calibration for Large-Scale Network Simulators.Transportation Research Part C: Emerging Technologies, 2019a

Osorio, C. Dynamic Origin–Destination Matrix Calibration for Large-Scale Network Simulators.Transportation Research Part C: Emerging Technologies, 2019a. 98:186–206