Pith. sign in

REVIEW 4 major objections 5 minor 63 references

Automatic Calibration of Dynamic and Heterogeneous Parameters in Agent-based Model

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A calibration framework that adapts parameters over time and across agent clusters cuts real-estate simulation error to roughly half the human-calibrated baseline.

desk verdict A credible combined dynamic/heterogeneous ABM calibration framework with honest in-sample-only evidence; the digital-twin framing overreaches, but the method and synthetic checks justify a serious referee. read the letter →

arxiv 1908.03309 v1 pith:AV7FPCXL submitted 2019-08-09 cs.MA cs.CYcs.LG

Dongjun Kim , Tae-Sub Yun , Il-Chul Moon This is my paper · ORCID
classification cs.MAcs.CYcs.LG
keywords agent-basedmodelcalibrationdynamicheterogeneoushiddenMarkovvariationalautoencoderGaussianprocessBayesianoptimizationrealestatemarketsimulationin-samplevalidation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that parameter calibration for agent-based simulations need not be a one-time, all-agents-same exercise: both time and agent heterogeneity can be handled automatically if hidden structure is extracted from simulation output first. It proposes a framework in which a Hidden Markov Model splits the simulation period into regimes and a variational autoencoder plus mixture-model clustering splits agents into subpopulations; dynamic parameters are then calibrated regime by regime and heterogeneous parameters cluster by cluster. On a synthetic wealth-distribution model the framework recovers the planted regime alternation and cluster-specific consumption, and on a real Korean apartment market model the best combined run reports total MAPE of 0.105 against 0.214 for human calibration. The authors state that the real-world evaluation is in-sample, so the result is about reproducing the calibration period, not yet about forecasting.

What carries the argument

The load-bearing mechanism is the hidden-structure extraction loop. First, temporal regimes are inferred by an HMM over the sequence of per-timestep deviations (simulation mean minus validation statistic) for each candidate parameter hypothesis; regimes that agree across all hypotheses are merged. Each merged regime gets its own Beta-distribution posterior over the dynamic parameter, and next candidates are drawn from it, with exploration triggered when all likelihoods are low. Second, agent subpopulations are inferred by encoding each agent's time-series state into a low-dimensional latent vector with a variational autoencoder, then clustering those vectors with GMM or DPMM. The resulting clusters define which agents share a heterogeneous parameter, and Gaussian-process Bayesian optimization with a portfolio of acquisition functions (random, predictive variance, predictive mean, weighted expected improvement) proposes the next cluster-specific parameter values.

What would settle it

Take the calibrated parameters from the real-estate case and run the simulation over a holdout window of months not used in calibration; if the total MAPE rises to the level of the human-calibrated baseline or above while the in-sample MAPE stays near 0.105, the claimed ability to regenerate real-world observations is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that simulation parameters in an agent-based model can be automatically calibrated along two axes that static calibration ignores: time and agent heterogeneity. The authors propose that both axes become tractable when hidden structure is extracted from simulation output first. Dynamic calibration runs a Hidden Markov Model on the deviations between simulated summary statistics and validation data across a set of candidate parameter hypotheses, merges the resulting regimes across candidates, and then samples new parameter values from a per-regime Beta posterior, so poorly fitted time periods explore more and well-fitted periods exploit more. Heterogeneous calibration compresses agent-level state variables with a variational autoencoder, clusters the latent representations with a Gaussian or Dirichlet-process mixture model, and then runs Gaussian-process Bayesian optimization on each cluster's parameter values. On a synthetic wealth-distribution model the framework recovers the alternation between the two planted regimes and the cluster-specific consumption rates; on a real real-estate market model calibrated to Korean apartment price and transaction data, the best combined run reports a total MAPE of 0.105 compared with 0.214 for the human-calibrated baseline.

Load-bearing premise

All reported real-world error reductions are measured on the calibration data itself, so the framework's practical value rests on the unvalidated assumption that in-sample fit transfers to unseen time periods.

Editorial extensions

If this is right

  • Dynamic calibration lets a simulation track regime shifts such as boom and bust periods without resetting parameters by hand.
  • Heterogeneous calibration shrinks an infeasible per-agent parameter problem to per-cluster parameters, making heterogeneity calibration practical.
  • In the real estate case, the combined framework reaches the human-calibration error within about 20 iterations and then saturates at roughly half that error.
  • Separating well-fitted from poorly-fitted time regimes causes exploration where the simulation is far from validation, which is why random initialization eventually performs as well as a carefully chosen initial set.
  • Cluster count and regime count remain fixed hyperparameters; the nonparametric clustering variant selects cluster number automatically but not the regime count.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the in-sample fit is predictive, the same loop could be run online: as new validation data arrives, the HMM would re-detect regimes and the framework would re-calibrate, turning a one-time calibration into a digital-twin maintenance loop.
  • A natural test is to hold out the final eight to twelve months of the Korean apartment series and compare the calibrated framework's out-of-sample MAPE against human calibration, directly addressing the in-sample limitation.
  • Because dynamic and heterogeneous calibrations alternate with fixed iteration counts, one could treat these counts as hyperparameters and test whether an adaptive schedule (for example, calibrating only when regime boundaries move) improves final error.
  • The Beta posterior couples all candidate hypotheses through normalized likelihoods; replacing it with a likelihood-weighted resampling step, closer to sequential Monte Carlo ABC, would make the regime-wise exploration scheme directly comparable to that family.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an automatic calibration framework for agent-based models that extends static parameter calibration along two dimensions: dynamic calibration, which changes parameter values over simulation time by detecting hidden temporal regimes with an HMM and sampling regime-specific parameters from estimated Beta distributions; and heterogeneous calibration, which assigns different parameter values to agent sub-populations discovered by VAE-based dimension reduction followed by GMM/DPMM clustering, with a Gaussian-process Bayesian optimization loop selecting new cluster-specific parameter values. The framework alternates the two components over a fixed number of iterations. The authors evaluate the components and the combined framework on a synthetic Wealth Distribution Model with known ground-truth parameters and on a Real Estate Market Model calibrated to a 24-month Korean Appraisal Board dataset, reporting substantial MAPE reductions relative to human calibration and random search, with 30 replications and Welch t-tests.

Significance. If the claims were fully supported, the paper would make a useful contribution to ABM calibration practice: the synthetic experiment provides a ground-truth check, the experiments are replicated 30 times with statistical tests, and the combination of temporal regime detection (HMM), agent sub-population extraction (VAE plus clustering), and per-cluster Bayesian optimization is a genuinely integrative design that goes beyond static calibration. The interpreted cluster characteristics in the real-estate case (e.g., cluster-specific Willing to Pay and Purchase Rate values) are also plausible and illustrate the method's potential for diagnostic insight. However, the real-world evaluation is explicitly in-sample only, and several methodological choices (the 0.1 variance-scaling factor, the Beta posterior ansatz, and the hand-picked exploration/exploitation schedules) are not subjected to sensitivity analysis, so the strength of the evidence currently matches 'the method can fit the calibration data well' rather than 'the calibrated model reproduces unseen real-world observations' that the digital-twin framing implies.

major comments (4)
  1. [Section 1 and Section 5.2 (Table 10)] The real-world evaluation is in-sample only: the same KAB data are used as the calibration target in Eqs. (13)-(18) and Eqs. (21)-(22) and as the evaluation set for the Total MAPE values in Table 10. The reported reductions (e.g., 0.214 to 0.105 for the best framework run) are therefore training errors, not estimates of predictive accuracy. Section 1 explicitly states 'we limit ourselves to the in-sample validation' and then asserts that 'in-sample validated models to repeat the real-world scenario in the simulation,' but no out-of-sample or temporal holdout evidence is supplied. Because the digital-twin motivation in the abstract and introduction requires generalization to unseen periods, this load-bearing step is unsupported and should be either demonstrated with a holdout evaluation or removed from the claims.
  2. [Section 4.2.1, Eq. (13)] The dynamic calibration likelihood is computed with simulation variance set to sigma_{s,t}^i = 0.1 * mu_{s,t}^i. This fixed variance-scaling factor is arbitrary, yet it directly determines the likelihoods that drive both the HMM regime detection in Eq. (14) and the Beta-distribution weighting in Eqs. (16)-(17). No justification or sensitivity analysis is provided for the 0.1 multiplier, so it is unclear whether the reported dynamic-calibration improvements are robust to this assumption. The authors should either derive the variance from simulation replicates, test a range of scaling factors, or otherwise demonstrate that the regime detection and parameter updates are insensitive to this choice.
  3. [Section 4.2.3, Eqs. (16)-(18)] The 'posterior distribution' of each dynamic parameter inside a merged regime is asserted to be a Beta distribution fitted by maximum likelihood to normalized parameter values and normalized likelihoods. This is an ad-hoc construction rather than the posterior of a well-defined generative model, and the exploration threshold in Eq. (18) with ratio = 0.9^c is likewise hand-picked. The paper acknowledges in Table 2 that the dynamic calibration has no theoretical convergence guarantee, but no experiment or argument shows that the Beta ansatz and the cooling schedule work for models other than the two cases tested. At minimum, the authors should report sensitivity of the synthetic ground-truth results to the choice of Beta prior, threshold ratio, and number of regimes K_dyn.
  4. [Section 4.3.4 and Algorithm 3] The heterogeneous calibration relies on a portfolio of acquisition functions with probabilities xi_rand, xi_PV, xi_PM, the initial random-exploration count C0, and the cooling weight w_c = 0.99^{c/2}, all of which are set without sensitivity analysis. Since the real-estate performance gain is measured in-sample and the method has many tunable components, it is possible that the reported advantage over random search and human calibration is partly attributable to these choices rather than to the proposed dynamic/heterogeneous structure. A robustness study varying these schedules (and the parameter-free-looking claim that they are well-chosen) would materially strengthen the paper.
minor comments (5)
  1. [Eq. (19)] Equation (19) writes M(Phet ∪ Phet; omega) where the first argument should be the union of dynamic and heterogeneous parameters, presumably Pdyn ∪ Phet; this appears to be a typo and should be corrected.
  2. [Eq. (21)] In the definition of F in Eq. (21), the expression 'Shet(D(P∪Phet,c;omega_r))' contains a stray D; it should be Shet(M(Pdyn ∪ Phet,c;omega_r)).
  3. [Eq. (15)] The definition of merged regimes in Eq. (15) is unclear: the condition 'if t1,t2 in MRu' is used to define MRu itself, which is circular as written; the authors should restate the condition in terms of equal regime-index vectors over candidate hypotheses.
  4. [Table 10] Several entries in Table 10 report standard deviations rounded to 0.000 (e.g., the Frameworkb 'Apartment Lease Transaction Number in Noncapital' row), which obscures the actual variability; reporting at least three significant digits would be clearer.
  5. [Algorithm 3] The initialization step Phet,0 = AgentClustering(Pin_dyn ∪ Pin_het) is not fully specified: the clustering is run on one simulation's agent-level outputs, but how the cluster-specific parameter values are initialized for each cluster is not stated, which makes the framework harder to reproduce.

Circularity Check

2 steps flagged · score 6.0 of 10

Real-world validation is in-sample: the reported MAPE reductions are the minimized training objective, so the digital-twin expectation is unsupported.

  1. fitted input called prediction [Section 1; Section 4.3.1-4.3.4; Table 10]
    "Before we discuss the framework, we limit ourselves to the in-sample validation [60]. In-sample validation focuses on constructing a highly descriptive model with calibrated parameters, and we expect the in-sample validated models to repeat the real-world scenario in the simulation. ... In heterogeneous calibrations of our experiments, the simulation error function, d, is the Mean Absolute Percentage Error (MAPE) ..."

    The real-estate Dhet is the same 24-month KAB series used as the GP training target (the F vector in Eq. 21) and as the Table 10 evaluation target. The acquisition optimization in Eq. 22 selects Phet,c+1 to reduce exactly the MAPE that Table 10 reports. Hence the headline real-world improvements (Framework b 0.105 vs human calibration 0.214) are training losses, not predictive validations; they are forced by construction to improve on a baseline only in-sample. The Section 1 expectation that 'in-sample validated models to repeat the real-world scenario' is an untested extrapolation from these same-data numbers.

  2. fitted input called prediction [Section 4.2 (Eqs. 11, 13-14) and Table 10]
    "P*dyn = arg min_{Pdyn} d(Eω∈Ω[Sdyn(M(Pdyn∪Phet;ω))],Ddyn) ... Li_{s,t} = N(ds,t|μi_{s,t},σi_{s,t}) ... Ot = (μi_{1,t}−d1,t,...,μi_{|Sdyn|,t}−d_{|Sdyn|,t})"

    The HMM regime detector and likelihood weights in Eqs. 13-14 are computed from deviations of simulation from the same Ddyn that Eq. 11 minimizes, and the dynamic-calibration Total MAPE in Table 10 (0.167) is measured on that same KAB series. Thus the calibration produces parameters by fitting Ddyn and is then evaluated on Ddyn; the reported improvement over human calibration is an in-sample optimization result, not an independent estimate of predictive error. This does not vitiate the synthetic Wealth Distribution experiment, which checks parameter recovery against known synthetic parameters.

full rationale

The paper's methodological core—HMM regime extraction for dynamic calibration, VAE plus GMM/DPMM clustering for heterogeneous calibration, and GP-BO for cluster-wise parameter search—is described self-contained and is not derived from the validation data in a definitional way; the Wealth Distribution test case provides a genuine controlled check, since the validation series is generated from known synthetic parameters and the reported parameter MAE/Euclidean error is independent of the optimized objective. The self-citation [33] for dynamic parameter calibration is historical, not load-bearing, because Section 4.2 fully specifies the algorithm. However, the real-world real-estate evaluation is in-sample by the authors' own statement: Section 1 limits the paper to in-sample validation, and every reported Total MAPE in Table 10 is computed on the same KAB data used as Ddyn/Dhet in Eqs. 11-14 and 21-22. The dynamic likelihood and the heterogeneous MAPE are the very quantities being minimized; reporting their final values as 'validation' success therefore reduces, by construction, to reporting the training objective. The digital-twin expectation in Section 1—that in-sample validated models will repeat real-world scenarios—is an extrapolation that the same-data numbers cannot support. This is a partial circularity: the framework's components and the synthetic test retain independent content, but the headline real-world error reductions are not predictive evidence.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The central claim relies on hand-set constants (0.1 variance scaling, cooling rates), structural assumptions (regime-constant dynamic parameters, cluster-constant heterogeneous parameters), and a beta posterior assumption. The in-sample validation boundary is an explicit limitation that also constrains the claim. No invented entities are introduced.

free parameters (7)
  • variance scaling factor for likelihood = 0.1
    Hand-chosen in Section 4.2.1 to set σ_i_s,t = 0.1 × μ_i_s,t in Eq. 13; directly shapes regime detection and parameter generation.
  • exploration threshold cooling ratio = 0.9^c
    Ad hoc schedule in Eq. 18 that controls when dynamic calibration explores; no theoretical justification.
  • number of HMM regimes (Kdyn) = 3
    User-set for both experiments (Tabs. 5 and 9); the framework assumes the true parameter processes follow 3 regimes.
  • number of agent clusters (Khet) = 1, 2, 4, 8, or nonparametric
    User-set for GMM or learned by DPMM; the paper scans these values rather than deriving them.
  • weighted EI cooling weight = 0.99^(c/2)
    Ad hoc cooling schedule in Section 4.3.4 to shift from exploration to exploitation.
  • acquisition function portfolio probabilities = not reported
    Probabilities ξrand, ξPV, ξPM, and ξw-EI in Algorithm 3 are user-set but not specified.
  • calibration iteration counts (Cdyn, Chet, Ccal) = 100 to 200 iterations depending on case
    User-set hyperparameters (Tabs. 5 and 9) that determine how much each calibration component runs.
assumptions (6)
  • domain assumption Dynamic parameters are constant within each hidden temporal regime.
    Section 4.2 states this to reduce the T-dimensional dynamic parameter space to Kdyn regime values.
  • domain assumption Heterogeneous parameters take one value per agent sub-population.
    Section 4.3 assumes cluster-level values rather than agent-level values to make optimization feasible.
  • ad hoc to paper Simulation output variance is proportional to the mean, with constant 0.1.
    Section 4.2.1 sets σ_i_s,t = 0.1 × μ_i_s,t; no empirical basis is given.
  • ad hoc to paper The posterior distribution of a dynamic parameter inside a regime is a Beta distribution.
    Section 4.2.3 normalizes parameters and likelihoods and fits Beta(α_u, β_u); this is a modeling choice without derivation.
  • domain assumption In-sample validation is sufficient to demonstrate calibration quality.
    Section 1 explicitly states the limitation to in-sample validation; all conclusions inherit this boundary.
  • standard math HMM, VAE, GMM, DPMM, and GPR behave as standard methods.
    The paper relies on textbook and cited properties of these tools (Sections 3.1 and 3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automatic Calibration of Dynamic and Heterogeneous Parameters in Agent-based Model." pith.science (2026). https://pith.science/paper/AV7FPCXL

@misc{pith2026190803309,
  author       = {Pith},
  title        = {Pith review of: Automatic Calibration of Dynamic and Heterogeneous Parameters in Agent-based Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AV7FPCXL}},
  note         = {Machine review of arXiv:1908.03309}
}
read the original abstract

While simulations have been utilized in diverse domains, such as urban growth modeling, market dynamics modeling, etc; some of these applications may require validations based upon some real-world observations modeled in the simulation, as well. This validation has been categorized into either qualitative face-validation or quantitative empirical validation, but as the importance and the accumulation of data grows, the importance of the quantitative validation has been highlighted in the recent studies, i.e. digital twin. The key component of quantitative validation is finding a calibrated set of parameters to regenerate the real-world observations with simulation models. While this parameter calibration has been fixed throughout a simulation execution, this paper expands the static parameter calibration in two dimensions: dynamic calibration and heterogeneous calibration. First, dynamic calibration changes the parameter values over the simulation period by reflecting the simulation output trend. Second, heterogeneous calibration changes the parameter values per simulated entity clusters by considering the similarities of entity states. We experimented the suggested calibrations on one hypothetical case and another real-world case. As a hypothetical scenario, we use the Wealth Distribution Model to illustrate how our calibration works. As a real-world scenario, we selected Real Estate Market Model because of three reasons. First, the models have heterogeneous entities as being agent-based models; second, they are economic models with real-world trends over time; and third, they are applicable to the real-world scenarios where we can gather validation data.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 62 canonical work pages

  1. [33]

    In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) , IEEE, pp 1605–1610

    Moon IC, Kim D, Yun TS, Bae JW, Kang DO, Paik E (2018) Data-driven automatic calibration for validation of agent-based social simulations. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) , IEEE, pp 1605–1610

    work page 2018

  2. [1]

    Annual review of ecology, evolution, and systematics 41:379–406

    Beaumont MA (2010) Approximate bayesian computation in evolution and ecology. Annual review of ecology, evolution, and systematics 41:379–406

    work page 2010

  3. [2]

    Springer

    Beisbart C, Saam NJ (2018) Computer Simulation Validation: Fundamental Concepts, Methodological Frameworks, and Philosophical Perspectives. Springer

    work page 2018

  4. [3]

    springer

    Bishop CM (2006) Pattern recognition and machine learning . springer

    work page 2006

  5. [4]

    Pro- ceedings of the national academy of sciences 99(suppl 3):7280–7287

    Bonabeau E (2002) Agent-based modeling: Methods and techniques for simulating human systems. Pro- ceedings of the national academy of sciences 99(suppl 3):7280–7287

    work page 2002

  6. [5]

    Brenner T, Werker C (2006) A practical guide to inference in simulation models. Tech. rep., Papers on economics and evolution

    work page 2006

  7. [6]

    arXiv preprint arXiv:10122599

    Brochu E, Cora VM, De Freitas N (2010) A tutorial on bayesian optimization of expensive cost func- tions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:10122599

    work page 2010

  8. [7]

    Journal of Machine Learning Research 12(Oct):2879–2904

    Bull AD (2011) Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research 12(Oct):2879–2904

    work page 2011

Show all 63 references
  1. [8]

    In: Inter- national Workshop on Multi-Agent Systems and Agent-Based Simulation , Springer, pp 41–57

    Calvez B, Hutzler G (2005) Automatic tuning of agent-based models using genetic algorithms. In: Inter- national Workshop on Multi-Agent Systems and Agent-Based Simulation , Springer, pp 41–57

    2005

  2. [9]

    In:40th Annual Symposium on Foundations of Computer Science (Cat

    Dasgupta S (1999) Learning mixtures of gaussians. In:40th Annual Symposium on Foundations of Computer Science (Cat. No. 99CB37039) , IEEE, pp 634–644

    1999

  3. [10]

    Princeton University Press

    Epstein JM (2006) Generative social science: Studies in agent-based computational modeling . Princeton University Press

    2006

  4. [11]

    In: Proceedings of the 25th international conference on Machine learning , ACM, pp 312–319

    Fox EB, Sudderth EB, Jordan MI, Willsky AS (2008) An hdp-hmm for systems with state persistence. In: Proceedings of the 25th international conference on Machine learning , ACM, pp 312–319

    2008

  5. [12]

    arXiv preprint arXiv:180702811

    Frazier PI (2018) A tutorial on bayesian optimization. arXiv preprint arXiv:180702811

    2018

  6. [13]

    practice

    Fu MC (2002) Optimization for simulation: Theory vs. practice. INFORMS Journal on Computing 14(3):192–215

    2002

  7. [14]

    Colorado College Publication, General Series 208:73–79

    Gini C (1936) On the measure of concentration with special reference to income and statistics. Colorado College Publication, General Series 208:73–79

    1936

  8. [15]

    In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statis- tics, pp 273–280 30 Dongjun Kim et al

    Gr¨ unew¨ alder S, Audibert JY, Opper M, Shawe-Taylor J (2010) Regret bounds for gaussian process bandit problems. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statis- tics, pp 273–280 30 Dongjun Kim et al

    2010

  9. [16]

    The new palgrave dictionary of economics pp 1–7

    Hamilton JD (2016) Regime switching models. The new palgrave dictionary of economics pp 1–7

    2016

  10. [17]

    In: Agent-Based Approaches in Economic and Social Complex Systems VII , Springer, pp 45–59

    Hara S, Kita H, Ikeda K, Susukita M (2013) Configuring agents attributes with simulated annealing. In: Agent-Based Approaches in Economic and Social Complex Systems VII , Springer, pp 45–59

    2013

  11. [18]

    Ecology letters 14(8):816–827

    Hartig F, Calabrese JM, Reineking B, Wiegand T, Huth A (2011) Statistical inference for stochastic simulation models–theory and application. Ecology letters 14(8):816–827

    2011

  12. [19]

    Environment and Planning B: Planning and Design 34(6):1051–1070

    Heppenstall AJ, Evans AJ, Birkin MH (2007) Genetic algorithm optimisation of an agent-based model for simulating a retail market. Environment and Planning B: Planning and Design 34(6):1051–1070

    2007

  13. [20]

    Operations Research Letters 9(1):35–44

    Ho YC, Hu JQ (1990) An infinitesimal perturbation analysis algorithm for a multiclass g/g/1 queue. Operations Research Letters 9(1):35–44

    1990

  14. [21]

    In: Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence , AUAI Press, pp 327–336

    Hoffman M, Brochu E, de Freitas N (2011) Portfolio allocation for bayesian optimization. In: Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence , AUAI Press, pp 327–336

    2011

  15. [22]

    Cities 31:105–113

    Hosseinali F, Alesheikh AA, Nourian F (2013) Agent-based modeling of urban land-use development, case study: Simulating future scenarios of qazvin city. Cities 31:105–113

    2013

  16. [23]

    Methods in Ecology and Evolution 4(7):684–687

    Jabot F, Faure T, Dumoulin N (2013) Easy abc: performing efficient approximate b ayesian computation sampling schemes using r. Methods in Ecology and Evolution 4(7):684–687

    2013

  17. [24]

    Encyclopedia of machine learning pp 760–766

    Kennedy J (2010) Particle swarm optimization. Encyclopedia of machine learning pp 760–766

    2010

  18. [25]

    arXiv preprint arXiv:13126114

    Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint arXiv:13126114

    2013

  19. [26]

    science 220(4598):671– 680

    Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. science 220(4598):671– 680

    1983

  20. [27]

    Journal of Economic Dynamics and Control 90:366–389

    Lamperti F, Roventini A, Sani A (2018) Agent-based model calibration using machine learning surrogates. Journal of Economic Dynamics and Control 90:366–389

    2018

  21. [28]

    Journal of Artificial Societies and Social Simulation 18(4):4

    Lee JS, Filatova T, Ligmann-Zielinska A, Hassani-Mahmooei B, Stonedahl F, Lorscheid I, Voinov A, Pol- hill JG, Sun Z, Parker DC (2015) The complexities of agent-based modeling output analysis. Journal of Artificial Societies and Social Simulation 18(4):4

    2015

  22. [29]

    University of Alberta

    Lizotte DJ (2008) Practical bayesian optimization. University of Alberta

    2008

  23. [30]

    Pro- ceedings of the National Academy of Sciences 100(26):15324–15328

    Marjoram P, Molitor J, Plagnol V, Tavar´ e S (2003) Markov chain monte carlo without likelihoods. Pro- ceedings of the National Academy of Sciences 100(26):15324–15328

    2003

  24. [31]

    Complexity Eco- nomics 2(1):41–61

    Marks RE (2013) Validation and model selection: Three similarity measures compared. Complexity Eco- nomics 2(1):41–61

    2013

  25. [32]

    Springer Science & Business Media

    Mockus J (2012) Bayesian approach to global optimization: theory and applications, vol 37. Springer Science & Business Media

    2012

  26. [34]

    In: 2010 The 7th International Conference on Informatics and Systems (INFOS) , IEEE, pp 1–6

    Naiem A, Reda M, El-Beltagy M, El-Khodary I (2010) An agent based approach for modeling traffic flow. In: 2010 The 7th International Conference on Informatics and Systems (INFOS) , IEEE, pp 1–6

    2010

  27. [35]

    In: Proceedings of the 8th annual conference on Genetic and evolutionary computation , ACM, pp 183–190

    Nannen V, Eiben AE (2006) A method for parameter calibration and relevance estimation in evolutionary algorithms. In: Proceedings of the 8th annual conference on Genetic and evolutionary computation , ACM, pp 183–190

    2006

  28. [36]

    Applied Energy 113:1043–1058

    Nguyen AT, Reiter S, Rigo P (2014) A review on simulation-based optimization methods applied to building performance analysis. Applied Energy 113:1043–1058

    2014

  29. [37]

    In: Thirty-Sixth Southeastern Symposium on System Theory, 2004

    Noel MM, Jannett TC (2004) Simulation of a new hybrid particle swarm optimization algorithm. In: Thirty-Sixth Southeastern Symposium on System Theory, 2004. Proceedings of the , IEEE, pp 150–153

    2004

  30. [38]

    Entropy 20(3):201

    Preuss R, von Toussaint U (2018) Global optimization employing gaussian process-based bayesian surro- gates. Entropy 20(3):201

    2018

  31. [39]

    Management Science 54(5):998–1014

    Rahmandad H, Sterman J (2008) Heterogeneity and network structure in the dynamics of diffusion: Com- paring agent-based and differential equation models. Management Science 54(5):998–1014

    2008

  32. [40]

    In: 4th World Congress on Social Simulation, 2012

    Rand W (2012) When does simulated data match real data? comparing model calibration functions using genetic algorithms. In: 4th World Congress on Social Simulation, 2012

    2012

  33. [41]

    AdaptiveComputation and Ma- chine Learning

    Rasmussen C, Williams C (2006) GaussianProcessesforMachineLearning. AdaptiveComputation and Ma- chine Learning

    2006

  34. [42]

    Operations Research 64(6):1515–1528

    Ryzhov IO (2016) On the convergence rates of expected improvement methods. Operations Research 64(6):1515–1528

    2016

  35. [43]

    Engineering optimization 34(3):263–278

    Sasena MJ, Papalambros P, Goovaerts P (2002) Exploration of metamodeling sampling criteria for con- strained global optimization. Engineering optimization 34(3):263–278

    2002

  36. [44]

    PhD thesis, University of Waterloo , Waterloo, Ont., Canada, Canada, aAINQ22234

    Schonlau M (1997) Computer experiments and global optimization. PhD thesis, University of Waterloo , Waterloo, Ont., Canada, Canada, aAINQ22234

    1997

  37. [45]

    Journal of Econometrics 140(2):874–883

    Silver M, Heravi S (2007) Why elementary price index number formulas differ: Evidence on price dispersion. Journal of Econometrics 140(2):874–883

    2007

  38. [46]

    Proceedings of the National Academy of Sciences 104(6):1760–1765 Automatic Calibration of Dynamic and Heterogeneous Parameters in Agent-based Model 31

    Sisson SA, Fan Y, Tanaka MM (2007) Sequential monte carlo without likelihoods. Proceedings of the National Academy of Sciences 104(6):1760–1765 Automatic Calibration of Dynamic and Heterogeneous Parameters in Agent-based Model 31

    2007

  39. [47]

    In: Advances in neural information processing systems , pp 2951–2959

    Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems , pp 2951–2959

    2012

  40. [48]

    Journal of Global Optimization 33(1):31–59

    S´ obester A, Leary SJ, Keane AJ (2005) On the design of optimization strategies based on global response surface approximation models. Journal of Global Optimization 33(1):31–59

    2005

  41. [49]

    In: 2010 AAAI Fall Symposium Series

    Stonedahl F, Wilensky U (2010) Evolutionary robustness checking in the artificial anasazi model. In: 2010 AAAI Fall Symposium Series

    2010

  42. [50]

    In: International Workshop on Multi-Agent Systems and Agent-Based Simulation , Springer, pp 61–75

    Stonedahl F, Wilensky U (2010) Finding forms of flocking: Evolutionary search in abm parameter-spaces. In: International Workshop on Multi-Agent Systems and Agent-Based Simulation , Springer, pp 61–75

    2010

  43. [51]

    Journal of the ACM (JACM) 34(3):686–717

    Suri R (1987) Infinitesimal perturbation analysis for general discrete event systems. Journal of the ACM (JACM) 34(3):686–717

    1987

  44. [52]

    Genetics 145(2):505–518

    Tavar´ e S, Balding DJ, Griffiths RC, Donnelly P (1997) Inferring coalescence times from dna sequence data. Genetics 145(2):505–518

    1997

  45. [53]

    Encyclopedia of machine learning pp 280–287

    Teh YW (2010) Dirichlet process. Encyclopedia of machine learning pp 280–287

    2010

  46. [54]

    Thiele JC, Kurth W, Grimm V (2014) Facilitating parameter estimation and sensitivity analysis of agent- based models: A cookbook using netlogo and r.Journal of Artificial Societies and Social Simulation 17(3):11

    2014

  47. [55]

    Proceedings of building simulation and optimization pp 1–8

    Tresidder E, Zhang Y, Forrester A (2012) Acceleration of building design optimisation through the use of kriging surrogate models. Proceedings of building simulation and optimization pp 1–8

    2012

  48. [56]

    Journal of Statistical Planning and inference 140(11):3088–3095

    Vazquez E, Bect J (2010) Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. Journal of Statistical Planning and inference 140(11):3088–3095

    2010

  49. [57]

    arXiv preprint arXiv:160205149

    Wang J, Clark SC, Liu E, Frazier PI (2016) Parallel bayesian global optimization of expensive functions. arXiv preprint arXiv:160205149

    2016

  50. [58]

    Statistics and computing 4(2):65–85

    Whitley D (1994) A genetic algorithm tutorial. Statistics and computing 4(2):65–85

    1994

  51. [59]

    Wilensky U (1998) Netlogo wealth distribution model. Center for Connected Learning and Computer- Based Modeling, Northwestern University, Evanston, IL[En l´ ınea] Disponible en: http://ccl northwestern edu/NetLogo/models/WealthDistribution

    1998

  52. [60]

    Journal of Artificial Societies and Social Simulation 10(2):8

    Windrum P, Fagiolo G, Moneta A (2007) Empirical validation of agent-based models: Alternatives and prospects. Journal of Artificial Societies and Social Simulation 10(2):8

    2007

  53. [61]

    In: Advances in Neural Information Processing Systems , pp 3126–3134

    Wu J, Frazier P (2016) The parallel knowledge gradient method for batch bayesian optimization. In: Advances in Neural Information Processing Systems , pp 3126–3134

    2016

  54. [62]

    In: Advances in Neural Information Processing Systems, pp 5267–5278

    Wu J, Poloczek M, Wilson AG, Frazier P (2017) Bayesian optimization with gradients. In: Advances in Neural Information Processing Systems, pp 5267–5278

    2017

  55. [63]

    Journal of Artificial Societies and Social Simulation 12(2):5

    Yang C, Kurahashi S, Kurahashi K, Ono I, Terano T (2009) Agent-based simulation on women’s role in a family line on civil service examination in chinese history. Journal of Artificial Societies and Social Simulation 12(2):5

    2009

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.