REVIEW 3 major objections 4 minor 2 cited by
Navigating through Economic Complexity: Phase Diagrams & Parameter Sloppiness
T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper argues that agent-based macroeconomic models should be explored by mapping their phase diagrams first—finding the tipping points and 'dark corners'—before any calibration or quantitative prediction is attempted.
desk verdict A readable manifesto for phase-diagram-first ABM exploration, but all the substance is in the author's prior papers; the key robustness claim is asserted rather than demonstrated here. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the phase diagram of an agent-based model: a map of parameter space in which regions of qualitatively different macroscopic behaviour are separated by critical lines. In the Mark-0 model this is drawn in the plane of two control parameters—R, the ratio of hiring to firing adjustment speed, and Θ, the maximum debt-to-sales threshold (a proxy for credit supply). The work this diagram does is to turn a simulation zoo into an organised phenomenology: within each phase aggregate behaviour is qualitatively similar and robust to secondary parameters, while crossing a boundary can trigger deflationary collapse, endogenous crisis waves, or runaway inflation. The second piece of machinery is parameter sloppiness: computing the Hessian of a loss function over log-parameters and retaining only the top eigenvectors (the stiff directions) lets an algorithm explore the phase diagram efficiently and recover the same four phases, a procedure the paper claims applies universally to ABMs.
What would settle it
A systematic sweep of Mark-0 with several plausible alternative rule sets (different wage-update rules, demand functions, or price-adjustment asymmetries) should check whether the four phases persist in the same relative arrangement in the R–Θ plane and whether the tipping lines stay within similar ranges; if qualitative changes appear—phases disappearing, new phases emerging, or boundaries moving dramatically—the robustness premise fails.
Extended reading notes
Core claim
On its own account, the paper's core claim is that 'establishing the phase diagram of ABMs is a crucial first step' before any calibration, because the emergent macro behaviour of an agent-based economy is not predictable from its micro rules by inspection. Working through the Mark-0 model, the paper identifies four phases and shows that the phase boundaries coincide with discontinuity lines where small parameter changes cause runaway instabilities—Black Swans or Dark Corners. It further claims that these phase diagrams are 'extremely robust' to details of model specification and secondary parameters, and that the high-dimensional search can be tamed by the discovery that only a few stiff parameter directions (combinations of parameters with large effect on observables) govern the phase structure, while many sloppy directions have little effect. The paper's conclusion is a methodological manifesto: turn ABMs into 'telescopes for the mind' for generating qualitatively plausible scenarios and counterfactuals, rather than treating them as precise calibrated forecasting instruments.
Load-bearing premise
The phase diagram of Mark-0 is robust to changes in model specification and to the values of secondary parameters, so the phase boundaries are features of the model class rather than artifacts of particular heuristic choices.
Editorial extensions
If this is right
- Calibration of an ABM should be deferred until after the phase diagram is established; parameters should be restricted to regimes that produce 'reasonable' outcomes.
- Policy advice from ABMs should come from scenario and counterfactual analysis of phases, not from point predictions.
- Monetary policy rules that are stabilizing in rational-expectations DSGE models can be destabilizing in Mark-0: hyper-reactive Taylor rules trigger crises rather than suppress them.
- The identification of stiff and sloppy directions provides a practical algorithm for exploring high-dimensional ABM parameter spaces, recovering known phases with less computation.
- Strong wage/price indexation (gp = gw > 1) can push an economy into hyperinflation when inflation expectations are anchored to past inflation rather than the central bank's target.
Reading between the lines
- The phase-diagram-first methodology could be carried over to other agent-based models outside macroeconomics—finance, epidemiology, sociology—using the same sloppy-direction algorithm to locate the few stiff directions that organize their behaviour.
- If the robustness of Mark-0's phase diagram holds across rule variants, the diagram might serve as a qualitative 'stability map' for real economies, letting policy makers ask where the current state sits relative to a tipping line.
- A concrete test of the method would be to apply the stiff-direction exploration to a much larger ABM (hundreds of parameters) and see whether the number of stiff directions remains small and whether the recovered phase boundaries coincide with those found by brute-force scanning.
- The paper's preference for being 'roughly right' implies that the evaluation of macro models should shift from point-prediction accuracy in normal times to the ability to generate plausible crisis scenarios and counterfactuals.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript is a position essay arguing that before calibrating agent-based models (ABMs), researchers should first map the model's phase diagram in parameter space and identify 'stiff' and 'sloppy' directions. It illustrates the argument with the Mark-0 model, reporting four phases (FU, FE, RU, EC) and various tipping points from prior work, and advocates a qualitative, scenario-based approach over quantitative calibration. The paper contains no new derivations or simulations; it is a synthesis of the author's own published research.
Significance. If the phase-diagram-first approach is valid, it offers a practical response to the 'wilderness of high-dimensional spaces' problem in ABM macroeconomics. The paper's clear exposition of the analogy with statistical physics and its emphasis on endogenous tipping points are valuable, and the explicit disclaimers about not having predicted the post-COVID inflation are a sign of scientific honesty. The main weaknesses are evidential: the central robustness claim is supported only by self-citations, and the paper does not specify how 'phases' are defined in finite stochastic non-equilibrium models. These issues are fixable in revision.
major comments (3)
- [Section 2 and Fig. 1 caption] The sentence 'such phase diagram is found to be extremely robust against both details of the model specification and the value of the other parameters, which leave the qualitative emergent behavior unchanged but affect the precise location of the phase boundaries' is the load-bearing premise for the paper's central recommendation. However, no evidence is provided in this paper; the only support is a citation to [47]. Because the entire phase-diagram-first methodology would collapse if phase boundaries were artifacts of heuristic choices, the authors should either present a sensitivity analysis (e.g., varying the wage-update rule, demand function, bankruptcy rule, and secondary parameters) or substantially hedge the claim to 'robust for the specifications explored in [47,54]'.
- [Section 2] The paper uses the term 'phase diagram' without specifying how phases and phase boundaries are defined for a finite, stochastic, non-equilibrium ABM. It does not discuss order parameters, finite-size scaling, or the timescales over which 'long-time' behavior is assessed. Without such criteria, the identified FU/FE/EC/RU regions may depend on simulation length and system size, and the 'discontinuous phase transition' language is not justified. This is a conceptual gap in the methodological argument.
- [Section 4] The sloppiness claims are asserted rather than demonstrated. The paper states that 'both agent-based models and DSGE models have a sloppy phenomenology' and that 'stiff parameter directions point towards close phase transitions,' but provides no eigenvalue spectra, no comparison of stiff directions with phase boundaries, and no quantitative efficiency comparison for the exploration algorithm. Since parameter sloppiness is a main pillar of the proposed methodology, at least one illustrative quantitative result (or a clear pointer to a reproducible figure in [54]) is needed.
minor comments (4)
- [Section 1.2] The sentence 'calibration of an ABM using real data should only start to after such a qualitative investigation has been performed' contains a grammatical error; it should read 'should only start after' or 'should only begin after.'
- [Section 2] The phrase 'phase-diagram of the shown in Fig. 1' should be 'phase diagram shown in Fig. 1.'
- [Section 4] The statement that 'only a handful of parameters (or combination of parameters) turn out to be crucial' is vague; please specify which parameters and in which model this was found.
- [References] Some references are incomplete (e.g., [24] and [25] lack full publication details), and the list would benefit from consistent formatting.
Circularity Check
No circularity: the paper is a methodological essay that cites prior independently checkable simulations rather than rederiving or fitting its premises.
full rationale
This paper contains no formal derivation chain that could reduce to its own inputs; it is a perspective/essay advocating a phase-diagram-first methodology for agent-based models. The load-bearing empirical claims—the R–Theta phase diagram of Mark-0 and its robustness to secondary parameters—are explicitly attributed to earlier published work, e.g., "such phase diagram is found to be extremely robust against both details of the model specification and the value of the other parameters... see [47]." The sloppiness results are likewise attributed to prior work: "In Ref. [54], we have shown that both agent-based models and DSGE models have a sloppy phenomenology." These are self-citations, but they refer to externally checkable numerical studies rather than to quantities fitted inside the present paper and then renamed as predictions. No equation is defined in terms of its own output; no parameter is fitted to a subset of data and then called a prediction; no uniqueness theorem from the authors is invoked to force the conclusion. The absence of independent replication weakens the evidentiary base, but that is a robustness/correctness concern, not circularity under the stated criteria. The paper even explicitly disclaims having predicted the post-COVID inflation spike, further removing any concern about retrofitted prediction. Accordingly, the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption Phase diagrams from statistical physics transfer to macroeconomic agent-based models.
- domain assumption Mark-0's heuristic behavioral rules are reasonable representations of real economic behavior in normal times.
- domain assumption The Mark-0 phase diagram is robust to changes in secondary parameters and model details.
- domain assumption Sloppy/stiff structure is a universal property of multiparameter models, including ABMs.
Cite this review
Pith. "Pith review of Navigating through Economic Complexity: Phase Diagrams & Parameter Sloppiness." pith.science (2026). https://pith.science/paper/JNOBGHZB
@misc{pith2026241211259,
author = {Pith},
title = {Pith review of: Navigating through Economic Complexity: Phase Diagrams & Parameter Sloppiness},
year = {2026},
howpublished = {\url{https://pith.science/paper/JNOBGHZB}},
note = {Machine review of arXiv:2412.11259}
}
read the original abstract
We argue that establishing the phase diagram of Agent Based Models (ABM) is a crucial first step, together with a qualitative understanding of how collective phenomena come about, before any calibration or more quantitative predictions are attempted. Computer-aided *gedanken* experiments are by themselves of genuine value: if we are not able to make sense of emergent phenomena in a world in which we set all the rules, how can we expect to be successful in the real world? ABMs indeed often reveal the existence of Black Swans/Dark Corners i.e. discontinuity lines beyond which runaway instabilities appear, whereas most classical economic/finance models are blind to such scenarii. Testing for the overall robustness of the phase diagram against changes in heuristic rules is a way to ascertain the plausibility of such scenarii. Furthermore, exploring the phase diagrams of ABM in high dimensions should benefit enormously from the identification of ``stiff'' and ``sloppy'' directions in parameter space.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 2 Pith papers
-
Herding and Liquidity in Order-Book Markets. II. Fundamental Anchoring and the Resilience of Liquidity
In a two-market order-book agent-based model, fundamental anchoring is the stabilizer whose removal lets a leverage spiral self-sustain, while none of six coupling channels transmits liquidity stress between markets.
-
Herding and Liquidity in Order-Book Markets. I. A Robust Liquidity-Stress Crossover and its Reflexive Mechanism
High herding fraction and strength produce a null-verified, rule-robust liquidity-stress crossover in a continuous double-auction book, with a large self-reinforcing reflexive component under price-momentum herding.
Reference graph
Works this paper leans on
-
[47]
J Econ Dyn Control 50(C):29–61
Gualdi S, Tarzia M, Zamponi F, Bouchaud JP (2015) Tipping points in macroeconomic agent- based models. J Econ Dyn Control 50(C):29–61
work page 2015
-
[54]
S., Benzaquen, M., & Bouchaud, J
Naumann-Woleske, K., Knicker, M. S., Benzaquen, M., & Bouchaud, J. P. (2022). Exploration of the Parameter Space in Macroeconomic Models. Handbook of Complexity Economics
work page 2022
-
[1]
Sethna, J. P. (2021). Statistical mechanics: entropy, order parameters, and complexity (Vol. 14). Oxford University Press, USA
work page 2021
-
[2]
see e.g. S. Balibar, The Discovery of Superfluidity, Journal of Low Temperature Physics 146:441- 470 (2007)
work page 2007
-
[3]
Anderson, P. W. (1972). More Is Different: Broken symmetry and the nature of the hierarchical structure of science. Science, 177(4047), 393-396
work page 1972
-
[4]
Bouchaud, J. P. (2023). From statistical physics to social sciences: the pitfalls of multi- disciplinarity. Journal of Physics: Complexity, 4(4), 041001
work page 2023
-
[5]
A. Kirman, Whom or What Does the Representative Individual Represent?, Journal of Economic Perspectives, 6: 117-136 (1992)
work page 1992
- [6]
Show all 63 references
-
[7]
see e.g.: P. Ball. Why Society is a Complex Matter . Springer, Berlin (2012) for an introductory review. 11
2012
-
[8]
Bouchaud, Crises and Collective Socio-Economic Phenomena: Simple Models and Chal- lenges, J
J.-P. Bouchaud, Crises and Collective Socio-Economic Phenomena: Simple Models and Chal- lenges, J. Stat. Phys. 151: 567-606 (2013), and refs. therein
2013
-
[9]
Kirman, A. (2010). Complex economics: individual and collective rationality. Routledge
2010
-
[10]
Arthur, W. B. (2021). Foundations of complexity economics. Nature Reviews Physics, 3(2), 136- 145
2021
-
[11]
G., Benzaquen, M., Tarzia, M., & Bouchaud, J
Morelli, F. G., Benzaquen, M., Tarzia, M., & Bouchaud, J. P. (2020). Confidence collapse in a multihousehold, self-reflexive DSGE model. Proceedings of the National Academy of Sciences, 117(17), 9244-9249
2020
-
[12]
Dosi, G. (2023). The foundations of complex evolving economies: Part one: Innovation, organi- zation, and industrial dynamics. Oxford University Press
2023
-
[13]
Dessertaine, T., Moran, J., Benzaquen, M., & Bouchaud, J. P. (2022). Out-of-equilibrium dy- namics and excess volatility in firm networks. Journal of Economic Dynamics and Control, 138, 104362
2022
-
[14]
Goldstone, M
for a concise overview of ABM for collective phenomena, see: R. Goldstone, M. Janssen, Com- putational Models of Collective Behavior, Trends in Cognitive Science 9: 424-30 (2005)
2005
-
[15]
Epstein, J. M. (2012). Generative social science: Studies in agent-based computational modeling. Princeton University Press
2012
-
[16]
Farmer, D
D. Farmer, D. Foley, The economy needs agent-based modelling, Nature 460, 685-686 (2009)
2009
-
[17]
Dosi, G., & Roventini, A. (2019). More is different... and complex! the case for agent-based macroeconomics. Journal of Evolutionary Economics, 29, 1-37
2019
-
[18]
Sims, Macroeconomics and Reality, Econometrica, 48: 1-48 (1980)
C. Sims, Macroeconomics and Reality, Econometrica, 48: 1-48 (1980)
1980
-
[19]
see e.g. J. Gali, Monetary Policy, Inflation and the Business Cycle , Princeton University Press (2008)
2008
-
[20]
R ¨ohe, O. (2012). New Keynesian DSGE models: theory, empirical implementation, and specifi- cation (Doctoral dissertation)
2012
-
[21]
state of the art
W. Buiter, The unfortunate uselessness of most “state of the art” academic monetary economics, http://blogs.ft.com/maverecon/2009/03/
2009
-
[22]
Blanchard, O. (2018). On the future of macroeconomic models. Oxford Review of Economic Policy, 34(1-2), 43-54, and other papers in that volume
2018
-
[23]
Kaplan, G., Moll, B., & Violante, G. L. (2018). Monetary policy according to HANK. American Economic Review, 108(3), 697-743
2018
-
[24]
Reis, R (2022) The burst of high inflation in 2021–22: How and why did we get here?
2022
-
[25]
Rev Keynesian Econ 10(1):25–45
Rudd JB (2022) Why do we think that inflation expectations matter for inflation? (And should we?). Rev Keynesian Econ 10(1):25–45. https:// doi. org/ 10. 4337/ roke. 2022. 01. 02
2022
-
[26]
Stiglitz J, Regmi I (2022) The causes of and responses to today’s inflation (Tech. Rep.). Roosevelt Institute. 12
2022
-
[27]
Bernanke, B., & Blanchard, O. (2023). What caused the US pandemic-era inflation?. Peterson Institute for International Economics Working Paper, (23-4)
2023
-
[28]
G., & Turrell, A
Haldane, A. G., & Turrell, A. E. (2018). An interdisciplinary model for macroeconomics. Oxford Review of Economic Policy, 34(1-2), 219-251; Haldane, A. G., & Turrell, A. E. (2019) Drawing on different disciplines: macroeconomic agent-based models. J. Evol. Econ. 29(1):39–66
2018
-
[29]
G., Hommes, C., & Rabitsch, K
Poledna, S., Miess, M. G., Hommes, C., & Rabitsch, K. (2023). Economic forecasting with an agent-based model. European Economic Review, 151, 104306
2023
-
[30]
Farmer, J. D. (2024). Making Sense of Chaos: A Better Economics for a Better World. Yale University Press
2024
-
[31]
& Farmer, J
Wiese, S., Kaszowska-Mojsa, J., Dyer, J., Moran, J., Pangallo, M., Lafond, F. & Farmer, J. D. (2024). Forecasting Macroeconomic Dynamics using a Calibrated Data-Driven Agent-based Model. arXiv preprint arXiv:2409.18760
2024 arXiv
-
[32]
Deissenberg, S
C. Deissenberg, S. van der Hoog, H. Dawid, Eurace: A massively parallel agent-based model of the european economy, Applied Mathematics and Computation, 204: 541-552 (2008)
2008
-
[33]
Dawid, S
H. Dawid, S. Gemkow, P. Harting, S. van der Hoog, M. Neugarty; The Eurace@Unibi Model: An Agent-Based Macroeconomic Model for Economic Policy Analysis, Working paper, Bielefeld University. 2011
2011
-
[34]
G. Dosi, G. Fagiolo, A. Roventini, An evolutionary model of endogenous business cycles. Com- putational Economics, 27: 3-24 (2005)
2005
-
[35]
G. Dosi, G. Fagiolo, and A. Roventini, Schumpeter meeting Keynes: A policyfriendly model of endogenous growth and business cycles, Journal of Economic Dynamics and Control, 34, 1748- 1767 (2010); and [38]
2010
-
[36]
Mandel, S
A. Mandel, S. Furst, W. Lass, F. Meissner, C. Jaeger; Lagom generiC: an agent-based model of growing economies, ECF Working Paper 1 /2009
2009
-
[37]
Caiani, A., Godin, A., Caverzasi, E., Gallegati, M., Kinsella, S., & Stiglitz, J. E. (2016). Agent based-stock flow consistent macroeconomics: Towards a benchmark model. Journal of Economic Dynamics and Control, 69, 375-408
2016
-
[38]
Fagiolo and A
G. Fagiolo and A. Roventini, ’Macroeconomic policy in dsge and agent-based models’Revue de l’OFCE 5 (2012): 67-116
2012
-
[39]
Fagiolo, A
G. Fagiolo, A. Moneta and P. Windrum, A critical guide to empirical validation of agent-based models in economics: methodologies, procedures, and open problems. Computational Economics 30.3 (2007): 195-226
2007
-
[40]
Pyka and G
A. Pyka and G. Fagiolo, Agent-based modelling: a methodology for neo-Schumpeterian eco- nomics. Elgar companion to neo-schumpeterian economics 467 (2007)
2007
-
[41]
Kirman, The Economic Crisis is a Crisis for Economic Theory, CESifo Economic Studies, 56: 498-535 (2010)
A. Kirman, The Economic Crisis is a Crisis for Economic Theory, CESifo Economic Studies, 56: 498-535 (2010)
2010
-
[42]
R. J. Caballero, Macroeconomics after the Crisis: Time to Deal with the Pretense-of-Knowledge Syndrome, Journal of Economic Perspectives, 24: 85-102 (2010) 13
2010
-
[43]
M., Mealy, P., Pichler, A., Lafond, F., & Farmer, J
del Rio-Chanona, R. M., Mealy, P., Pichler, A., Lafond, F., & Farmer, J. D. (2020). Supply and demand shocks in the COVID-19 pandemic: An industry and occupation perspective. Oxford Review of Economic Policy, 36(Supplement 1), S94-S137
2020
-
[44]
Bak, How Nature Works: The Science of Self-Organized Criticality, Copernicus, Springer, New York, 1996
P. Bak, How Nature Works: The Science of Self-Organized Criticality, Copernicus, Springer, New York, 1996
1996
-
[45]
Bouchaud, J. P. (2024). The self-organized criticality paradigm in economics & finance. arXiv preprint arXiv:2407.10284
2024 arXiv
-
[46]
Kondor, I
on this point, see the interesting discussion in: I. Kondor, I. Csabai, G. Papp, E. Mones, G. Czimbalmos, M. Cs. Sandor, Strong random correlations in networks of heterogeneous agents, arXiv:1210.2224
-
[48]
Buchanan, This Economy does not compute, New York Times, October 1, 2008
M. Buchanan, This Economy does not compute, New York Times, October 1, 2008
2008
-
[49]
Blanchard, O. (2014). Where Danger Lurks. Finance & Development
2014
-
[50]
P., Cencetti, G., Tarzia, M., & Zamponi, F
Gualdi, S., Bouchaud, J. P., Cencetti, G., Tarzia, M., & Zamponi, F. (2015). Endogenous crisis waves: stochastic model with synchronized collective behavior. Physical review letters, 114(8), 088701
2015
-
[51]
Gaffeo, D
The Mark I family of models was elaborated in a series of papers and books, in particular: E. Gaffeo, D. Delli Gatti, S. Desiderio, and M. Gallegati, Adaptive Microfoundations for Emergent Macroeconomics, Eastern Economic Journal, 34: 441-463 (2008); D. Delli Gatti, E. Gaffeo,...
2008
-
[52]
Delli Gatti, S
D. Delli Gatti, S. Desiderio, E. Gaffeo, P. Cirillo, M. Gallegati,Macroeconomics from the Bottom- up, Springer: Berlin (2011)
2011
-
[53]
S., Naumann-Woleske, K., Bouchaud, J
Knicker, M. S., Naumann-Woleske, K., Bouchaud, J. P., & Zamponi, F. (2024). Post-COVID in- flation and the monetary policy dilemma: An agent-based scenario analysis. Journal of Economic Interaction and Coordination, 1-55
2024
-
[55]
Strogatz, S. H. (2012). Sync: How order emerges from chaos in the universe, nature, and daily life. Hachette Books
2012
-
[56]
J Econ Interac Coord 12(3):507–537
Gualdi S, Tarzia M, Zamponi F, Bouchaud JP (2017) Monetary policy and dark corners in a styl- ized agent-based model. J Econ Interac Coord 12(3):507–537. https:// doi. org/ 10. 1007/ s11403- 016- 0174-z
2017
-
[57]
P., Gualdi, S., Tarzia, M., & Zamponi, F
Bouchaud, J. P., Gualdi, S., Tarzia, M., & Zamponi, F. (2018). Optimal inflation target: insights from an agent-based model. Economics, 12(1), 20180015
2018
-
[58]
PLoS One 16(3):1–15 14
Sharma D, Bouchaud JP, Gualdi S, Tarzia M, Zamponi F (2020) V-, U-, L-, or W-shaped recovery after COVID? Insights from an agent based model. PLoS One 16(3):1–15 14
2020
-
[59]
Alvarez J, Bluedorn JC, Hansen NJ, Huang Y, Pugacheva E, Sollaci A (2022) Wage-price spirals: What is the historical evidence?
2022
-
[60]
Econ Inq 26(2):265–279
Holland AS (1988) The changing responsiveness of wages to price-level shocks: explicit and implicit indexation. Econ Inq 26(2):265–279
1988
-
[61]
N., Waterfall, J
Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P. (2007). Universally sloppy parameter sensitivities in systems biology models. PLoS computational biology, 3(10), e189
2007
-
[62]
N., Abbott, M
Quinn, K. N., Abbott, M. C., Transtrum, M. K., Machta, B. B., & Sethna, J. P. (2022). Information geometry for multiparameter models: New perspectives on the origin of simplicity. Reports on Progress in Physics, 86(3), 035901
2022
-
[63]
Naumann-Woleske, K. (2024). Scenario Discovery in a Complex Economy: Exploring the Param- eter space of Agent-based Models (Doctoral dissertation, Institut Polytechnique de Paris). 15
2024
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.