{"id":"51b0e726-7123-43e5-8aff-f27d058bc2a7","arxiv_id":"2501.09429","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ADAGE unifies four common agent-based modelling tasks, policy design, calibration, scenario generation, and robust behavioral learning, under a single Stackelberg game formulation with adaptive behavioral policies.","lead":"A research team from JPMorgan AI Research proposes ADAGE, a two-layer framework that frames adaptive agent-based models as a Stackelberg game, letting both agent behavior and environmental parameters adapt. The same framework is shown to handle policy design, calibration, scenario generation, and robust behavioral learning, which could make ABMs more flexible and less vulnerable to the Lucas critique.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (4) targets a Nash stationary point, not the Stackelberg equilibrium defined in §3.1; the cited single-agent gradient-domination argument does not bridge this gap.","rationale":"The reader's weakest assumption is essentially this, though I sharpen it: the problem is not only the missing proof of gradient-domination transfer but the definitional mismatch that Eq. (4) solves a Nash stationary point. The paper's experiments are useful and support that the framework can instantiate four tasks; code is provided; but no experiment checks whether the found policies correspond to the §3.1 Stackelberg equilibrium rather than to a Nash-like fixed point. In particular, the policy design experiment compares only against a free-market baseline, so it cannot distinguish a good heuristic outcome from a Stackelberg equilibrium. Therefore the theoretical claim should be either proved or qualified. I do not see grounds for REJECT: the advertised applications are demonstrated qualitatively and the framework has practical value regardless of equilibrium semantics. The appropriate disposition remains CONDITIONAL, as the reader concluded; hence verdict_should_be is UNCHANGED (the condition stands).","tokens_in":16954,"tokens_out":6002,"duration_ms":63228,"concrete_test":"Build the smallest possible ADAGE-style leader-follower game, e.g. a 2×2 normal-form game embedded as a one-step POMG: leader chooses θ ∈ {0,1}, follower chooses a ∈ {0,1}; set rewards so that the Stackelberg leader value differs from the Nash value (e.g. leader prefers commitment to θ=1, follower best-responds with a=0, while the joint Nash equilibrium is θ=0,a=1). Compute all Eq. (4) solutions by enumerating pure/mixed policies and checking ∇π_i R_i=0, then compute the true Stackelberg equilibrium from the §3.1 definition by exhaustive best-response. If any Eq. (4) solution is not a Stackelberg equilibrium—or if the leader's value at the Eq. (4) solution is below the commitment value—the sufficiency claim in §3.2 fails. This test is decisive because it isolates the equilibrium concept from learning dynamics and function approximation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central formal claim—that ADAGE solves a Stackelberg game by solving the coupled system (4)—breaks at a definitional mismatch in §3.2. §3.1 correctly defines a Stackelberg equilibrium as a leader commitment π_L* satisfying π_L* ∈ argmax_{π_L} [R_L | π_F ∼ ε(π_L)], with followers best-responding. §3.2 then re-characterizes the objective as finding a point at which no agent can improve its local objective function holding the behaviour of all other agents fixed—the definition of a Nash equilibrium, not a Stackelberg equilibrium. Equation (4) is exactly the first-order stationarity condition for that Nash-style objective: each agent's gradient is zero with other agents' policies held fixed. The cited result [1] is a single-agent gradient-domination theorem; even if one granted its full multi-agent analogue, it would only validate that a joint point satisfying Eq. (4) is a global Nash equilibrium. It does not address the leader's commitment problem, where R_L must be maximized against the follower best-response map ε(π_L), not against the particular follower policies found at the stationary point. The manuscript's own caveat—that it does not prove convergence and only learns approximate best responses—is honest but confirms that the equilibrium semantics of Eq. (4) are asserted, not derived. If this mismatch is real, ADAGE remains a useful bi-level optimization framework, but its headline claim of formalising the bi-level problem as a Stackelberg game and basing it on solving a coupled set of non-linear equations is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ADAGE, a two-layer framework for adaptive agent-based modelling in which a leader agent in an outer layer adjusts environment characteristics θ (e.g., tax parameters, calibration targets, scenario controls, or sampling distributions) while follower agents in an inner simulation layer learn behavioural policies conditioned on their observations of θ. The problem is formulated as a partially observable Markov game with n+1 agents and is claimed to be a Stackelberg game; the paper argues that solving a coupled system of first-order equations (Eq. 4) yields the desired equilibrium. Four tasks are presented as instantiations of the framework: policy design (TaxAI), calibration (cobweb market with bounded-rational agents), scenario generation (Tobin tax for market stabilisation), and robust behavioural learning (market maker across preferences). The experiments use PPO for most agents, with a Bayesian outer layer and an analytically derived outer layer as algorithm-independence demonstrations. The paper includes source code in the supplementary material.","tokens_in":17258,"tokens_out":4401,"duration_ms":47745,"significance":"If the formal claims held, ADAGE would provide a useful unification of several common ABM tasks under a single bi-level optimisation view, and the paper demonstrates genuine breadth: multiple simulators, mixed discrete/continuous actions, different learning algorithms in the outer layer, and comparisons against external baselines such as free-market, rational-expectations, and fixed-ω training. The manuscript also honestly reports that convergence to equilibrium is not proven and that best responses are approximate. However, the central theoretical step—identifying solutions of Eq. (4) with Stackelberg equilibria—is not established, and the experimental evaluation is mostly illustrative. The paper is therefore best read as a framework-and-demonstration contribution; with a precise restatement of what is proved and what is heuristic, it could be a valuable consolidation of adaptive-ABM methodology.","major_comments":[{"comment":"The Stackelberg equilibrium defined in §3.1 is a leader commitment π_L maximising R_L against the follower best-response oracle ε(π_L), but §3.2 re-characterises the objective as a point at which no agent can improve its local objective holding all other agents' behaviours fixed, and Eq. (4) is the first-order condition for that Nash-style problem. The cited gradient-domination result [1] is proven for single-agent policy gradients; no argument is given that it extends to the n+1-agent partially observable Markov game, and even a full multi-agent analogue would at most identify global Nash equilibria of the simultaneous-move game, not the leader's commitment optimum against followers' best responses. The manuscript's caveat that it only learns approximate best responses is honest, but it means the statement that the framework is 'formalised' as a Stackelberg game and 'based on solving a coupled set of non-linear equations' is asserted rather than derived. Please provide a proof or precise reference for the multi-agent gradient-domination condition in this leader-follower setting, or reframe the contribution as bi-level optimisation with Stackelberg-inspired semantics rather than as an exact Stackelberg equilibrium solver.","section":"§3.2, Eq. (4)"},{"comment":"Even if Eq. (4) were the correct stationarity condition, the proposed A-GD algorithm updates each policy against the latest iterates of all other agents and relies on the learning-rate separation α_L >> α_F to approximate nested-loop play. The cited convergence results [28, 57] are for two-time-scale stochastic approximation and bilevel optimisation under assumptions such as strong convexity/smoothness that are not verified for the POMG returns in Eq. (2). The paper therefore does not support the claim that running A-GD converges to a Stackelberg equilibrium; at best this is a heuristic. The text should state this explicitly wherever the optimisation method is described, rather than presenting Eq. (4) and A-GD as the solving mechanism for the formal Stackelberg problem.","section":"§3.2, alternating gradient descent/ascent"},{"comment":"The empirical claims would be easier to assess with uncertainty quantification. In Table 1, MAE and RMSE are reported without confidence intervals or a statement of the number of seeds/rollouts; in Table 2, ADAGE is worse than baseline for 2 of 9 capacities under mean σ and for 3 of 9 under mean absolute percentage change, yet the text says the approach 'successfully restricts market volatility in the vast majority of configurations'. Reporting standard errors, per-capacity bootstrap intervals, and the number of independent runs behind each number would make the conclusions commensurate with the evidence and would also clarify whether the observed improvements are consistent rather than noise.","section":"§4.2–4.3, experimental evaluation"}],"minor_comments":[{"comment":"The notation ∇_{π*_i} R_i = 0 is confusing because π*_i denotes the unknown solution; the intended meaning is ∇_{π_i} R_i evaluated at π_i = π*_i, which should be written explicitly.","section":"§3.2, Eq. (4)"},{"comment":"The text refers to 'the conventional case of c = 0.6 [4]', but the experiments vary capacity C over 2,...,18 and no capacity of 0.6 appears; the cited reference [4] also does not appear to define this convention.","section":"§4.3.2"},{"comment":"The sentence 'By optimising for eq. (18)' is incorrect: Eq. (18) defines market share, whereas the objective to be optimised is the MM reward in Eq. (16).","section":"§4.4.1"},{"comment":"Table 3 reports 'Training iterations 100, 1000'; please specify which environment uses which value, otherwise the configuration is ambiguous.","section":"Table 3"},{"comment":"Supplementary Fig. 9 omits the C=12 panel even though Table 2 reports results for C=12; please include the missing panel or explain the omission.","section":"Supplementary Fig. 9"},{"comment":"The policy-design experiment compares only against a free-market baseline; adding a standard tax schedule, such as a flat or proportional tax, would make the claim of improved social welfare more informative.","section":"§4.1.2"}],"recommendation":"major_revision","confidential_remarks":"The Stackelberg/Nash mismatch in §3.2 is the main substantive issue. It is serious but fixable within the manuscript's scope by reframing the contribution as a bi-level optimisation framework with approximate best responses, rather than claiming an exact Stackelberg-equilibrium solution. The paper would then be an honest consolidation of existing adaptive-ABM practice. The experiments are illustrative rather than decisive, so the revision should also tighten the statistical reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou'll want to know two things about this paper before deciding whether to spend time on it. First, the four-task unification is real and useful: policy design, calibration, scenario generation, and robust behavioral learning all fall out of the same two-layer setup by changing the leader's reward and action space. That's a nice synthesis, and the paper credits the AI Economist as the origin of the two-level idea while making it generic and simulator-independent. The experiments, though illustrative rather than exhaustive, do show the framework working across four distinct environments, with code released, and they include a Bayesian outer layer and an analytically derived policy to show the framework is not tied to PPO.\n\nSecond, the theory is the soft spot, and it's load-bearing. The paper defines a Stackelberg equilibrium correctly in §3.1: the leader commits to a policy and the followers best-respond. But §3.2 redefines the objective as a point where no agent can improve holding all other policies fixed—that's Nash, not Stackelberg. Equation (4) is the first-order stationarity condition for that Nash objective. The gradient-domination argument cited from Agarwal et al. is a single-agent MDP result; even under a multi-agent extension it would give a global Nash point, not a leader-commitment equilibrium. The manuscript is honestly hedged—it says explicitly that no convergence guarantees are provided and that only approximate best responses are learned—but the abstract and contributions still advertise a Stackelberg equilibrium 'based on solving a coupled set of non-linear equations.' That claim is not established by the text. The stress-test note is right on the money here.\n\nThe other issues are secondary. The policy-design experiment compares only to a free-market baseline, not to the AI Economist or any other tax-optimization method, so it doesn't show the framework improves on existing approaches. Calibration and scenario metrics lack error bars; only the robust-behavior plots give standard deviations.\n\nSo the bottom line is: this is a useful framework paper with an unproven, and probably overstated, equilibrium claim. The unification itself stands, and the empirical demonstration is enough to justify a serious referee. A revision should either prove the gradient-domination transfer or quietly replace 'Stackelberg equilibrium' with 'bi-level optimization target,' add a policy-design baseline, and report variances.\n\nI'd bring it to reading group and would cite it for the framework synthesis, not for the theory.","headline":"ADAGE is a genuinely useful framework synthesis, but its central claim to solve a Stackelberg equilibrium rests on an unproven—and likely overstated—equivalence.","tokens_in":17777,"tokens_out":3081,"would_cite":true,"duration_ms":30071,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ADAGE claims one Stackelberg formulation can handle four core agent-based modelling tasks.","keywords":["agent-based modelling","Stackelberg games","bi-level optimisation","multi-agent reinforcement learning","model calibration","policy design","scenario generation","meta-learning"],"falsifier":"Run a targeted policy search around the leader policy ADAGE converges to in any of the four environments, with the learned follower policies held fixed: if a leader deviation improves its own return while followers are allowed to best-respond, the stationary point found by Eq. (4) is not a Stackelberg equilibrium. Directly, one can test the gradient-domination premise in the cobweb calibration by sweeping many random follower and leader policies and checking whether every low-gradient point has return close to the best found; if not, the single-agent gradient-domination result does not transfer to this multi-agent game.","tokens_in":16728,"feed_emoji":"🧩","tokens_out":7576,"duration_ms":69412,"temperature":0.7,"pith_summary":"ADAGE formalizes adaptive agent-based modelling as a two-layer Stackelberg game: an outer leader chooses environment characteristics θ (tax rules, latent parameters, tax rates, or preference samples), and inner follower agents learn behavioural policies conditioned on those characteristics. The paper claims this one formulation subsumes four previously separate modelling tasks—policy design, calibration, scenario generation, and robust behavioural learning—because changing only the leader's reward and action space changes the task. If correct, modellers would not need bespoke algorithms for each task; the same coupled optimisation of leader and follower policies would adapt agents to environmental change while also adapting the environment to agent behaviour, directly addressing the Lucas critique. The paper demonstrates each task in a different economic or financial simulator, and swaps in different learning algorithms for the layers, to show that the framework is generic rather than tied to one simulator or one optimiser.","feed_headline":"Four agent-based modelling tasks collapse into one Stackelberg game","feed_subtitle":"Policy design, calibration, scenario generation, and robust learning all become one leader-follower optimisation.","key_machinery":"The load-bearing object is the conditional behavioural policy $\\pi_i(a \\mid o_i, \\hat{\\theta}_i)$: each follower's action distribution is a function of its local observation plus its view of the leader-chosen characteristics θ. The leader's action space and reward define the task, and θ parameterises the environment the followers perceive. The training problem is then reduced to solving the coupled non-linear system of first-order optimality conditions, Eq. (4), for all $n+1$ agents; the paper invokes the gradient-domination property of policy-gradient returns [1] to claim every stationary point is globally optimal, and uses alternating gradient descent/ascent with leader learning rates much larger than follower rates to approximate a nested-loop best-response oracle. In practice the follower policies are PPO networks, but the framework is presented as algorithm-agnostic: an analytic maximum-entropy distribution replaces the learned outer policy in the robust-behaviour task, and a Bayesian optimiser replaces the RL outer layer in one calibration variant.","core_discovery":"On its own terms, the paper's central claim is that a broad class of adaptive ABM problems can be written as a partially observable Markov game with $n+1$ agents: agent 0 is the leader and the remaining $n$ agents are followers, and the solution concept is a Stackelberg equilibrium—the leader maximises its expected return given that followers respond optimally to whatever the leader does. The leader's policy determines a characteristics variable θ, and each follower conditions its behavioural policy on an observation of θ, so follower behaviour adapts automatically when the environment changes. The paper argues that because the expected return in Eq. (2) satisfies the gradient-domination condition, every stationary point of the coupled first-order conditions ($\\nabla_{\\pi_i} R_i = 0$ for all $i$, and $\\nabla_{\\pi_L} R_L = 0$) is globally optimal; hence solving those non-linear equations by alternating gradient descent/ascent yields a Stackelberg equilibrium. Changing only the leader's reward and action space turns the same system into: policy design (maximise household welfare by setting tax parameters), calibration (match simulated prices to experimental data by tuning bounded-rationality costs), scenario generation (stabilise market volatility by setting a Tobin tax), and robust behavioural learning (maximise entropy over market-maker preferences to learn one policy that extrapolates across preferences). The experiments are demonstrations that these four tasks are special instances of the framework, not head-to-head comparisons of learning algorithms.","pith_inferences":["Because the paper's theoretical justification for Eq. (4) relies on a single-agent gradient-domination result, a natural next test is whether the learned stationary points in these environments are truly Stackelberg equilibria; if not, ADAGE still works as a bi-level heuristic but its 'solution of a coupled set of non-linear equations' claim is weaker than stated.","The same conditional-policy mechanism could be applied to tasks the paper does not run, such as inverse reinforcement learning of agent preferences from data, or robust policy design that optimises worst-case welfare across a distribution of θ rather than a single realisation.","The scenario-generation result suggests a cheap way to screen proposed market interventions: run ADAGE as an automated search over policy parameters and inspect which θ produce the desired emergent outcome, rather than hand-tuning those parameters.","Although the paper uses one leader, it notes multiple leaders are possible; a multi-leader extension would let several designers or regulators co-adapt, which would need a different equilibrium concept than the single-leader Stackelberg solution."],"forward_implications":["If ADAGE is correct, a modeller can switch between policy design, calibration, scenario generation, and robust behavioural learning by editing only the leader's reward function and action space, keeping the same POMG representation and Stackelberg solution concept.","Follower policies that condition on θ should react to new environment characteristics without retraining, which is the direct mechanism by which ADAGE claims to soften the Lucas critique.","Because the framework is simulator-independent, results from one ABM (e.g., Tobin tax stabilising a market) transfer as a template to other simulation platforms, and algorithm-independent, so the inner or outer layer can be swapped for analytic or Bayesian solvers.","The calibration experiment implies that bounded-rationality parameters can be fit to human experimental data through the same bi-level loop that also handles welfare maximisation, so calibration and policy design are not separate methodological islands.","The robust-behaviour experiment implies that a single conditional policy can extrapolate across a continuum of agent preferences, so the framework offers a route to meta-learning within an ABM without a separate meta-loop."],"supporting_citations":[{"why":"Supplies the single-agent gradient-domination theorem the paper uses to justify that stationary points of Eq. (4) are global optima.","marker":"[1]"},{"why":"Provides the Stackelberg POMDP formulation the leader-follower representation is built on.","marker":"[9]"},{"why":"Defines Stackelberg equilibria and best-response oracles in deep multi-agent reinforcement learning, the adopted solution concept.","marker":"[23]"},{"why":"TaxAI serves as the inner simulator for the policy-design experiment and gives the HSV tax function used by the leader.","marker":"[33]"},{"why":"Cobweb market experimental data is the calibration target and supplies the price and reward equations for the calibrated simulator.","marker":"[27]"},{"why":"The bounded-rationality information-processing-cost model being calibrated is taken from this prior MARL calibration study.","marker":"[16]"},{"why":"The market-maker reward and behaviour formulation in the robust behavioural learning experiment follows this over-the-counter market simulation framework.","marker":"[51]"},{"why":"PPO is the concrete reinforcement-learning algorithm used to learn the conditional behavioural policies in the experiments.","marker":"[44]"},{"why":"The AI Economist two-layer tax-policy design is the motivating precursor and appears again as a special case of ADAGE's policy-design configuration.","marker":"[61]"}],"fun_headline_variants":["Four ABM tasks merge into one Stackelberg game","Adaptive agent models unified via leader-follower game","One framework turns four agent-based tasks into a game","Stackelberg game unifies policy, calibration, scenarios, robust learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument works only if a joint stationary point of the coupled gradient conditions is actually a Stackelberg equilibrium—the paper assumes the single-agent gradient-domination property transfers to the $n+1$-agent leader-follower Markov game, but gives no proof for that setting.","fun_headline_variants_meta":{"raw":{"variants":["Four ABM tasks merge into one Stackelberg game","Adaptive agent models unified via leader-follower game","One framework turns four agent-based tasks into a game","Stackelberg game unifies policy, calibration, scenarios, robust learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000232,"raw_usage":{"total_tokens":1556,"prompt_tokens":1080,"completion_tokens":476,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":696,"completion_tokens_details":{"reasoning_tokens":408}},"tokens_in":696,"tokens_out":476,"duration_ms":6922,"temperature":1.0,"reasoning_tokens":408,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:01:49.143054+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a targeted policy search around the leader policy ADAGE converges to in any of the four environments, with the learned follower policies held fixed: if a leader deviation improves its own return while followers are allowed to best-respond, the stationary point found by Eq. (4) is not a Stackelberg equilibrium. Directly, one can test the gradient-domination premise in the cobweb calibration by sweeping many random follower and leader policies and checking whether every low-gradient point has return close to the best found; if not, the single-agent gradient-domination result does not transfer to this multi-agent game.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Stackelberg equilibria and best-response oracles in deep multi-agent reinforcement learning, the adopted solution concept."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TaxAI serves as the inner simulator for the policy-design experiment and gives the HSV tax function used by the leader."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cobweb market experimental data is the calibration target and supplies the price and reward equations for the calibrated simulator."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The bounded-rationality information-processing-cost model being calibrated is taken from this prior MARL calibration study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The market-maker reward and behaviour formulation in the robust behavioural learning experiment follows this over-the-counter market simulation framework."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The AI Economist two-layer tax-policy design is the motivating precursor and appears again as a special case of ADAGE's policy-design configuration."}],"review_version":1}