{"id":"a9ead179-0ee7-418c-82e5-24bff43f6afc","arxiv_id":"2502.12024","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A bisection-based adaptive algorithm converges to a mean field equilibrium for scalar-interaction dynamic games, and a model-free Q-learning variant learns it from simulation.","lead":"This paper designs algorithms that compute or learn a mean field equilibrium in dynamic games where agents interact through a single number, such as average capacity or the fraction of available drivers. It proves the algorithms converge without contraction assumptions and applies them to inventory, ridesharing, reputation, and social learning models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Assumption 1 fails at m=1 in the ridesharing model (no requests ever arrive), so Theorem 1's stated assumptions do not cover a flagship application.","rationale":"The reader's weakest-assumption analysis identifies exactly the same load-bearing issue: Assumption 1's irreducibility fails in the ridesharing model at m=1, and the paper asserts without verification that the assumption always holds. I agree with that assessment. The conditional theorem itself (if Assumption 1 holds, bisection on f converges to an MFE) appears mathematically sound; the problem is that a showcased application falls outside the theorem's hypotheses at an endpoint of the search domain, so the paper's claim that Assumption 1 is 'typically satisfied trivially' in all applications is inaccurate. This is a correctness gap in the applicability claim, not in the core root-finding argument. The reader's CONDITIONAL verdict is appropriate: the paper should either weaken Assumption 1 to the unique-invariant-distribution condition actually needed, or modify the ridesharing model (e.g., by ensuring f(M)<1 for all M in [0,1]) so that the stated irreducibility holds. No additional objections rise to the level of changing the verdict.","tokens_in":38806,"tokens_out":8555,"duration_ms":98847,"concrete_test":"Instantiate the ridesharing model with X1={0,1}, one request type, durations d_1=1, f(M)=M, and p_1=(1-M). Write out the transition matrix L_{1,g} at m=1 for the optimal policy. Verify that state (1,1) (unavailable, request type 1) has no incoming transitions from state (0,0), which is absorbing, so the chain is not irreducible. Then run Algorithm 1 on this instance with initial bracket [0,1] and a high long-trip payoff; check whether the returned m* approaches 1. If it does, the numerical example converges at a point where Assumption 1 is violated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central convergence argument rests on Assumption 1, which requires the Markov chain L_{m,g} to be irreducible and aperiodic for every m in [a,b] and every optimal policy g. Section 7 explicitly asserts that this assumption is satisfied in all of the applications. The ridesharing model of Section 7.1 contradicts this assertion at the upper endpoint of the bisection interval. In that model, the scalar interaction is M(s)=fraction of available drivers, and the probability of receiving no request is f(M), with request probabilities p_j. In the simulations f(M)=M and p_j=(1-M)/3. At m=1, f(1)=1 and p_j=0, so no requests ever arrive. Starting from the state (0,0) (available, no request), the only next state is (0,0); all states with positive remaining ride duration have no incoming transitions from the recurrent class. Thus L_{1,g} is not irreducible, violating Assumption 1. Moreover, M(s_{1,g})=1, so f(1)=1-M(s_{1,g})=0: the endpoint m=1 is itself a root of f. Consequently, the set of m where the theorem's hypotheses hold excludes a point in the algorithm's search interval, and the unconditional claim that Assumption 1 holds in all applications is false. The theorem may be salvageable by weakening irreducibility to 'unique invariant distribution with a single recurrent class,' but as written, the stated assumptions are not sufficient to cover the ridesharing example.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies stationary mean field equilibria in dynamic games in which payoffs and transitions depend on the population distribution only through a scalar M(s). It defines f(m)=m-M(s_{m,g}), where s_{m,g} is the invariant distribution induced by an optimal policy g at fixed scalar m, and observes that roots of f are MFEs. Algorithm 1 (Adaptive Value Function Iteration) evaluates f by dynamic programming plus an invariant-distribution computation and uses bisection; Theorem 1 asserts convergence to an MFE under Assumption 1 (ergodicity of L_{m,g}) and a single-valued optimal policy. Algorithm 2 replaces the dynamic-programming step with Q-learning and the invariant-distribution step with Monte Carlo sampling; Theorem 2 asserts asymptotic convergence. The paper also proves existence on non-compact Polish state spaces (Proposition 1), comparative statics (Proposition 2 and Theorem 3), and finite-time error bounds (Propositions 4 and 5), and reports simulations for inventory, capacity, ridesharing, social learning, and reputation models.","tokens_in":39003,"tokens_out":10099,"duration_ms":101564,"significance":"If the main theorem holds, the scalar-interaction formulation turns a nonconvex fixed-point problem into one-dimensional root-finding, and bisection gives global convergence without contraction or monotonicity assumptions. This is a genuinely useful and nontrivial contribution, especially the model-free Q-learning variant and the finite-time bounds. The existence result for non-compact state spaces and the comparative-statics results are also valuable. A further strength is that the convergence argument is not circular: the MFE is encoded as a root of f, and bisection does not presuppose the equilibrium. However, the paper's stated assumptions currently fail to cover at least one of its own applications (ridesharing at m=1), and parts of the comparative-statics theorem are not proved in the text. These issues are fixable but currently limit the manuscript's claims.","major_comments":[{"comment":"The opening of Section 7 states that Assumption 1 \"is satisfied in all of the applications below.\" This is contradicted by the ridesharing model of Section 7.1. There f(M)=M and p_j=(1-M)/3, so at m=1 no request arrives with probability one. Starting from the recurrent state (0,0), the next state is (0,0); states (0,j) with j>0 and (d,j) with d>0 are transient and have no incoming transitions from the recurrent class. Thus L_{1,g} is not irreducible, and Theorem 1 and Theorem 2, which assume Assumption 1 for every m in [a,b], do not apply at the endpoint m=1. Moreover M(s_{1,g})=1, so f(1)=0 and the endpoint is itself a root, so the endpoint cannot be discarded by an open-interval argument. The theorem is likely salvageable by weakening Assumption 1 to uniqueness of the invariant distribution (a single recurrent class) or by handling the endpoints separately, but as written the applications claim is false.","section":"Section 7.1; Section 7 opening"},{"comment":"The proof of Theorem 3 parts (iii) and (iv) is incomplete. For part (iii), the text says \"an argument similar to Theorem 3 in Light (2021) shows that the policy function g(x,m,z) is increasing in z,\" but no statement of the hypotheses of that result and no verification of those hypotheses in the present setting is provided. Part (iv) is dismissed with \"follows from the same argument and therefore omitted.\" Since parts (iii)-(iv) are stated as theorems and support the paper's analytical comparative-statics contribution, the proof should be written out or reduced to a precise reference whose assumptions are explicitly checked.","section":"Section 10.3, Theorem 3"},{"comment":"The proof of Theorem 2 has two gaps relative to the stated result. First, the asynchronous Q-learning convergence proposition (Proposition 3) is quoted with only step-size conditions sum(gamma)=infinity and sum(gamma^2)<infinity; for asynchronous Q-learning one also needs a condition guaranteeing that every state-action pair is visited infinitely often. Algorithm 2's Step 2 does not specify the behavior policy (e.g., epsilon-greedy with a suitable schedule), so the theorem's hypotheses are incomplete. Second, the proof says \"The proof of Proposition 1 shows that f is continuous,\" but Proposition 1 is proved under Assumption 2, whereas Theorem 2 assumes only Assumption 1. The continuity statement is true in the finite-state setting, but the argument should be supplied directly rather than by reference to a different assumption set.","section":"Section 3.2, Section 10.1"}],"minor_comments":[{"comment":"The update \"Set m_t = a+b / 2\" should read \"Set m_t = (a+b)/2\"; the initialization m_1=a, m_2=b and the subsequent use of m_t in Steps 2-4 also make the indexing of the sequence confusing.","section":"Algorithm 1, Step 1"},{"comment":"Theorem 1 is stated without a proof; the text says \"Theorem 2 and its proof immediately lead to the proof of Theorem 1,\" but no direct proof or corollary is given. Add a short proof for the full-information case.","section":"Section 3.1"},{"comment":"The statement contains the duplicated phrase \"with probability at least with probability\"; it should read \"with probability at least 1-(epsilon_H+epsilon_K).\"","section":"Proposition 5"},{"comment":"The function b(delta) introduced in Assumption 4(iv) conflicts notationally with the interval endpoint b in [a,b]; using a different symbol (e.g., rho(delta)) would avoid confusion.","section":"Assumption 4(iv)"},{"comment":"The symbol f is used both for the no-request probability f(M) and for the root function f(m)=m-M(s_{m,g}); different letters (e.g., q(M) and F(m)) would improve clarity.","section":"Section 7.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's algorithmic idea is solid and should be publishable after the assumption-coverage issue in ridesharing and the missing proof details are addressed. The heavy reliance on Light (2024a), Light (2024b), and Light (2021) is mostly for standard facts, which is acceptable, but Theorem 3(iii) should not lean on an unverified 'similar argument.' Given the simulation-heavy content, the paper may be a better fit for an operations-research or mathematical-economics journal than for a pure theory journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper sells an idea that deserves to catch on: when agents interact through a scalar m, compute MFE by bisecting f(m)=m-M(s_{m,g}) rather than iterating on the distribution. That is a real simplification, and I don't know of another paper in the MFE computation literature that states it this cleanly. The adaptive value iteration and Q-learning versions both follow from the same root-finding logic, and the numerical work on inventory, capacity, reputation, and ridesharing shows the method runs fast and gives sensible comparative statics. The existence theorem for non-compact state spaces is a bonus, and the proof via tightness of invariant distributions is fine.\n\nThe soft spots are proportionate. First, Assumption 1 is not the mild condition the text claims. It is used to make f well-defined and continuous, and Section 7 asserts it is satisfied in all applications. That assertion is false for the ridesharing model at m=1: with f(M)=M and p_j=(1-M)/3, no request ever arrives, the chain is not irreducible, and m=1 is itself a root of f. So Theorem 1's hypotheses don't cover a boundary point of the interval the algorithm actually searches. This is fixable by weakening irreducibility to unique invariant distribution plus continuity of s_{m,g}, or by checking the endpoint separately. As written, the text overclaims. Second, the comparative statics results in Theorem 3(iii) and (iv) are deferred or sketched, and the discrete-state arguments are not fully presented. Third, the numerical section has no code or data link and no quantitative comparison against fixed-point iteration, which makes the \"reliable\" claim one of principle rather than demonstrated practice.\n\nNone of this kills the paper. The bisection formulation is a useful tool, the Q-learning variant is a nontrivial extension, and the applications show reach. I'd send it to a serious referee. The right revision should fix Assumption 1, verify it (or replace it) in the applied models, and either include the comparative statics proofs or clearly mark them as deferred to a companion paper. A referee who checks the ridesharing model will catch the endpoint issue; the authors should close that gap before claiming global convergence.","headline":"Scalar bisection for MFE is a genuinely useful idea and the paper is largely defensible, but the ridesharing application violates Assumption 1 at the endpoint, so the global convergence claim is overstated as written.","tokens_in":39605,"tokens_out":1998,"would_cite":true,"duration_ms":20801,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91A16","91A50","90C40"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper reduces MFE computation in scalar-interaction dynamic games to a one-dimensional root-finding problem and gives globally convergent model-based and model-free algorithms.","keywords":["mean field equilibrium","scalar interaction","global convergence","Q-learning","adaptive value function iteration","comparative statics","dynamic oligopoly","ridesharing"],"falsifier":"Take the paper's ridesharing model and run Algorithm 1 with bisection interval $[0,1]$; at scalar value $m=1$ every driver is available, so no driver receives a request, states containing a request are unreachable, and the induced Markov chain is not irreducible on the full state space, meaning Assumption 1 fails and the invariant distribution used to define $f$ does not exist. A single evaluation at $m=1$ therefore breaks the guarantee, so the theorem is falsified unless Assumption 1 is checked at the boundaries.","tokens_in":38498,"feed_emoji":"🎯","tokens_out":7732,"duration_ms":75662,"temperature":0.7,"pith_summary":"This paper argues that for a large class of dynamic mean field games, namely those where agents influence each other only through a one-dimensional function of the population distribution, computing a stationary mean field equilibrium is a one-dimensional root-finding problem. It introduces an adaptive value-function iteration algorithm and a model-free Q-learning variant that both converge to an MFE under mild ergodicity and single-valuedness assumptions, without requiring contraction or monotonicity. If correct, this provides the first global convergence guarantees for MFE computation in this class, making counterfactual analysis and market-design exercises reliable where fixed-point iteration may fail. It also proves existence of MFE on non-compact state spaces and derives analytical comparative statics, all from the scalar interaction structure.","feed_headline":"When agents interact via one number, MFE reduces to root-finding","feed_subtitle":"New algorithms converge without contraction or monotonicity, enabling reliable market-design counterfactuals.","key_machinery":"The scalar interaction reduction is $f(m)=m-M(s_{m,g})$, where $M(s)$ is the scalar interaction function and $s_{m,g}$ is the unique invariant distribution of the Markov chain $L_{m,g}$ induced by the optimal policy $g$ at scalar value $m$. For each fixed $m$, the agent solves a standard discounted dynamic program; the optimal policy then defines a linear Markov chain whose invariant distribution is obtained by solving a linear system. Assumption 1 plus single-valuedness make $f$ continuous, so bracketing $f(a)\\leq 0\\leq f(b)$ and bisection converge to a root that certifies an MFE. In the learning version, Q-learning replaces the dynamic program and Monte Carlo simulation replaces the linear-system solve, preserving the outer bisection structure.","core_discovery":"The central claim is Theorem 1: under Assumption 1, meaning every induced Markov chain is irreducible and aperiodic, and under a single-valued optimal policy correspondence, the sequence $m_t$ produced by the bisection-driven Algorithm 1 converges to $m^*$ such that $f(m^*)=m^*-M(s_{m^*,g^*})=0$, and the policy $g^*(x,m^*)$ together with the invariant population state $s_{m^*,g^*}$ constitutes an MFE. The same root condition is reached asymptotically by Algorithm 2 when its inner Q-learning and Monte Carlo sampling are run with increasing accuracy. The paper further claims that because all interaction flows through the scalar $M(s)$, the equilibrium set is exactly the zero set of the continuous function $f$, so bisection replaces fixed-point iteration. A model-free version learns the MFE from simulated trajectories without knowing payoffs or transitions, and the same proof structure yields existence on Polish state spaces and monotone comparative statics of the equilibrium scalar under stochastic dominance.","pith_inferences":["The root-finding viewpoint suggests a practical screening test before applying the algorithm: compute $f$ on a coarse grid and check continuity by comparing $f(m+\\varepsilon)$ with $f(m)$; where jumps appear, either the policy has ties or the chain is near-non-ergodic, and the user knows the convergence guarantee does not apply.","Multi-dimensional interactions are the natural next target; the paper's Broyden extension has no global guarantee, but if the Jacobian of $f$ is diagonally dominant in an application, a globally convergent quasi-Newton method might be restored.","The platform-design reading of the inventory results is that a marketplace controlling both storage fees and commissions can substitute low storage fees for high commissions: even though the paper only sweeps a grid, the equilibrium mechanism of fewer stockouts and more transaction volume gives a testable prediction for platform pricing."],"forward_implications":["In any scalar-interaction model satisfying the assumptions, MFE can be computed by value iteration followed by a linear-system solve, with the number of outer iterations logarithmic in the desired scalar tolerance.","The same guarantee carries over to the data-driven setting: with enough Q-learning updates and enough simulated transitions, the learned policy and population state approach an MFE even though payoffs and transition probabilities were never estimated.","Existence no longer requires a compact state space; a moment-boundedness condition on invariant distributions is enough.","Comparative statics of the equilibrium scalar follow from stochastic dominance of policy-induced invariant distributions, so policy conclusions such as demand shifts raising equilibrium average capacity come with formal support."],"supporting_citations":[{"why":"Provides the discrete-time MFE definition and the fixed-point iteration baseline that the paper's bisection method replaces.","marker":"Adlakha and Johari (2013)"},{"why":"Supplies the oblivious equilibrium approximation for finite-agent industries that the model generalizes.","marker":"Weintraub et al. (2008)"},{"why":"Is the MFE framework the paper extends to scalar interactions, including uniqueness and comparative statics discussion.","marker":"Light and Weintraub (2022)"},{"why":"Is the convergence result for Q-learning used to turn Algorithm 2's inner loop into an optimal policy.","marker":"Bertsekas and Tsitsiklis (1996)"},{"why":"Provides the monotone comparative statics theorem applied to the equilibrium scalar via Proposition 2.","marker":"Milgrom and Roberts (1994)"},{"why":"Is used for perturbation bounds on invariant distributions in Assumption 4 and the finite-time error analysis.","marker":"Shardlow and Stuart (2000)"},{"why":"Supplies the maximum theorem used to establish continuity of the optimal policy and of the function $f$.","marker":"Aliprantis and Border (2006)"}],"fun_headline_variants":["Scalar interactions make MFE a simple root-finding problem","MFE algorithms converge without contraction or monotonicity","Model-free RL learns mean field equilibria from simulations","Global convergence for mean field equilibria in many market models","Bisection finds mean field equilibria when fixed-point fails"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof needs the map from each candidate scalar $m$ to an optimal policy to be single-valued and the induced Markov chain to be irreducible and aperiodic; where payoffs or transitions make best responses tied or make some states transient, $f$ may discontinue or fail to exist.","fun_headline_variants_meta":{"raw":{"variants":["Scalar interactions make MFE a simple root-finding problem","MFE algorithms converge without contraction or monotonicity","Model-free RL learns mean field equilibria from simulations","Global convergence for mean field equilibria in many market models","Bisection finds mean field equilibria when fixed-point fails"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1608,"prompt_tokens":1034,"completion_tokens":574,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":507}},"tokens_in":650,"tokens_out":574,"duration_ms":6530,"temperature":1.0,"reasoning_tokens":507,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T17:56:12.054210+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the paper's ridesharing model and run Algorithm 1 with bisection interval $[0,1]$; at scalar value $m=1$ every driver is available, so no driver receives a request, states containing a request are unreachable, and the induced Markov chain is not irreducible on the full state space, meaning Assumption 1 fails and the invariant distribution used to define $f$ does not exist. A single evaluation at $m=1$ therefore breaks the guarantee, so the theorem is falsified unless Assumption 1 is checked at the boundaries.","supporting_citations":[{"cited_title":"Markov perfect industry dynamics with many firms,","cited_arxiv_id":null,"evidence_quote":"Supplies the oblivious equilibrium approximation for finite-agent industries that the model generalizes."}],"review_version":1}