pith. sign in

arxiv: 2604.27380 · v1 · submitted 2026-04-30 · 🧮 math.OC

Mean-Field Systems with Heterogeneous Subteams: Optimality of Cluster-Symmetric Independent Policies and Equivalence with Decentralized McKean-Vlasov Control of Cluster-Representative Agents

Connor S. Braun , Sina Sanjari , Naci Saldi , Gunnar Blohm , Serdar Y\"uksel This is my paper

Pith reviewed 2026-05-07 08:16 UTC · model grok-4.3

classification 🧮 math.OC
keywords mean-field controlheterogeneous agentsclustersdecentralized policiesMcKean-Vlasov controlexchangeable coststeam optimalitymean-field limit
0
0 comments X

The pith

In mean-field teams with finite heterogeneity, optimal policies are symmetric within clusters and depend solely on each cluster's state distribution.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines large systems of interacting agents divided into a finite number of distinct clusters, where agents are identical within each cluster but clusters may differ. For discounted costs that are exchangeable within clusters, it proves that there exist optimal joint policies that are symmetric within clusters and depend on the overall system state only through the empirical distribution of states in each cluster separately. As the total number of agents grows large, these policies converge to fully decentralized policies that each agent can implement using only its own state and the distribution in its cluster. This convergence justifies modeling the infinite population limit as a control problem with a single representative agent per cluster, where the representatives are coupled through their state distributions, and provides a verification theorem for solving this limit problem. Such results enable the analysis of cooperative multi-agent systems that have realistic group differences without forcing all agents to be identical.

Core claim

For discounted partially exchangeable cost criteria in discrete-time teams with agents grouped into finitely many symmetric clusters, the optimal centralized joint policies are exchangeable within each cluster and depend on the agent ensemble only up to the state empirical distribution over each cluster. A generalization of De Finetti's theorem shows subsequential convergence of these policies to decentralized cluster-symmetric policies as population size tends to infinity. These limiting policies are asymptotically optimal for finite populations and correspond to the solution of a decentralized McKean-Vlasov team control problem with coupled representative agents, one for each cluster, for

What carries the argument

Cluster-symmetric independent policies, which are exchangeable within clusters and depend only on per-cluster empirical state distributions; these carry the argument by preserving optimality in finite systems and enabling subsequential convergence to a mean-field limit with one representative agent per cluster.

Load-bearing premise

The costs must be discounted and partially exchangeable, agents must be symmetric within each of finitely many clusters, and the proof relies on subsequential convergence of policies as the total population size tends to infinity under standard regularity conditions.

What would settle it

Construct a two-cluster finite-population example with a partially exchangeable discounted cost where, for large numbers of agents, the optimal joint policy requires knowledge of the full configuration of states across agents rather than just the per-cluster empirical distributions or breaks symmetry within a cluster; if such a policy exists and outperforms the cluster-symmetric ones, the optimality claim fails.

read the original abstract

Across science and engineering, mean-field methods have been a powerful and versatile approach for the analysis of systems of many interacting elements. However, common arguments used to characterize an infinite population limit can be quite restrictive from a modeling perspective by requiring that all agents be identical (i.e. symmetric, or homogeneous). In this paper, we consider large interactive particle systems under agent heterogeneity for a class of discrete time teams composed of finitely many species of agents, grouped into symmetric subteams, called clusters. In particular, for the class of discounted, partially exchangeable cost criteria considered, we establish the optimality of centralized joint policies which are exchangeable within each cluster and depend on the agent ensemble only up to the state empirical distribution over each cluster. Following this, a generalization of De Finetti's theorem is used to demonstrate the subsequential convergence of these optimal policies to one which is decentralized (depending on only the local state and distribution over each cluster) and symmetric within each subteam as the population size approaches infinity. This solution is shown to induce a sequence of asymptotically optimal policies for the finite population problems which retain their structure and decentralization. Furthermore, our analysis justifies the optimality of a decentralized McKean-Vlasov team representation involving coupled representative agents for each of the clusters, and establishes a verification theorem/value iterations for the mean-field limit. In this way, we provide an avenue for analyzing complex, cooperative systems with finite heterogeneity and set the stage for further research on learning algorithms.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 3 minor

Summary. The paper considers discrete-time cooperative team problems with finitely many heterogeneous clusters of agents under discounted, partially exchangeable costs. It establishes that optimal centralized policies are exchangeable within each cluster and depend on the finite-N system only through the vector of cluster empirical measures. A generalized De Finetti theorem is invoked to extract subsequential limits that are decentralized (functions of local state and the vector of limiting measures) and symmetric within clusters; these limits are shown to be asymptotically optimal for the original finite-N problems and to solve an equivalent decentralized McKean-Vlasov team problem with one representative agent per cluster, for which a verification theorem and value-iteration scheme are derived.

Significance. If the convergence arguments are completed, the results provide a rigorous justification for reducing finite-heterogeneity team problems to a tractable infinite-population McKean-Vlasov control problem with coupled representatives. This extends classical mean-field team theory beyond full homogeneity while preserving decentralization and asymptotic optimality, and supplies a foundation for scalable learning algorithms in multi-type cooperative systems. The explicit use of partial exchangeability and subsequential limits is a methodological strength.

major comments (3)
  1. [§5, Theorem 5.3] §5 (Mean-field limit), Theorem 5.3 and the surrounding tightness argument: the subsequential convergence of controlled empirical-measure processes to a deterministic McKean-Vlasov flow is asserted under “standard regularity conditions” enabling the generalized De Finetti representation and propagation of chaos. For heterogeneous clusters the interaction occurs through the product of Wasserstein spaces; uniform Lipschitz and linear-growth conditions with respect to the product metric are required but are not explicitly stated or verified for the class of partially exchangeable costs. Without a dedicated lemma confirming these conditions (and hence uniqueness of the limit flow), the claimed vanishing of the value gap between finite-N and mean-field problems cannot be guaranteed.
  2. [§6] §6 (Asymptotic optimality), the passage from subsequential policy convergence to asymptotic optimality: the argument shows that the cost of the limiting decentralized policy equals the mean-field value along the same subsequence used for De Finetti extraction. If the regularity conditions do not guarantee that every subsequence yields the same deterministic limit, the optimality statement may hold only along a further subsequence; the paper should clarify whether additional uniform integrability or uniqueness arguments close this gap for the full sequence of finite-N problems.
  3. [§7] §7 (Verification theorem), the coupled HJB system for the representative agents: existence of a classical solution is invoked to obtain the verification result, yet the paper provides no existence proof under the stated assumptions on the dynamics and partially exchangeable costs. If existence is merely assumed, the equivalence between the finite-N optimality and the McKean-Vlasov problem is conditional rather than unconditional; a remark or appendix establishing existence (or relaxing to viscosity solutions) is needed.
minor comments (3)
  1. Notation: the symbol μ is used both for finite-N empirical measures and for the limiting McKean-Vlasov measures; a consistent distinction (e.g., μ^N vs. μ) would improve readability.
  2. [Introduction] References: the introduction should cite recent works on multi-population mean-field games and teams (e.g., the literature on heterogeneous MFGs with finite types) to better situate the novelty of the cluster-symmetric decentralized limit.
  3. [§4, Lemma 4.1] The statement of the generalized De Finetti theorem (Lemma 4.1) should explicitly list the measurability and integrability conditions on the cost functional that are needed for the representation to apply to controlled processes.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments on our manuscript. We address each major comment below and commit to revisions that strengthen the rigor of the convergence, optimality, and verification arguments.

read point-by-point responses

  1. Referee: [§5, Theorem 5.3] The subsequential convergence of controlled empirical-measure processes to a deterministic McKean-Vlasov flow is asserted under “standard regularity conditions” enabling the generalized De Finetti representation and propagation of chaos. For heterogeneous clusters the interaction occurs through the product of Wasserstein spaces; uniform Lipschitz and linear-growth conditions with respect to the product metric are required but are not explicitly stated or verified for the class of partially exchangeable costs. Without a dedicated lemma confirming these conditions (and hence uniqueness of the limit flow), the claimed vanishing of the value gap cannot be guaranteed.

    Authors: We agree that the regularity conditions must be made explicit for the heterogeneous case. In the revised manuscript we will insert a new Lemma 5.1 immediately preceding Theorem 5.3. The lemma verifies that the partially exchangeable costs satisfy uniform Lipschitz continuity and linear growth with respect to the product Wasserstein metric on the finite collection of cluster measures, under the paper’s standing assumptions on the dynamics and discount factor. The same lemma establishes uniqueness of the McKean-Vlasov flow, thereby completing the tightness argument and guaranteeing that every subsequential limit is the unique deterministic flow. revision: yes

  2. Referee: [§6] The argument shows that the cost of the limiting decentralized policy equals the mean-field value along the same subsequence used for De Finetti extraction. If the regularity conditions do not guarantee that every subsequence yields the same deterministic limit, the optimality statement may hold only along a further subsequence; the paper should clarify whether additional uniform integrability or uniqueness arguments close this gap for the full sequence of finite-N problems.

    Authors: We acknowledge the potential gap. The revised Section 6 will first invoke the uniqueness of the McKean-Vlasov limit established by the new Lemma 5.1. We will then add a uniform-integrability argument for the sequence of finite-N costs, which follows directly from the linear-growth bound on the partially exchangeable costs and the discounting. Together these steps show that the value gap vanishes along the entire sequence, not merely along further subsequences, thereby making the asymptotic-optimality claim unconditional. revision: yes

  3. Referee: [§7] Existence of a classical solution is invoked to obtain the verification result, yet the paper provides no existence proof under the stated assumptions on the dynamics and partially exchangeable costs. If existence is merely assumed, the equivalence between the finite-N optimality and the McKean-Vlasov problem is conditional rather than unconditional; a remark or appendix establishing existence (or relaxing to viscosity solutions) is needed.

    Authors: The referee correctly identifies that existence is currently assumed. In the revision we will add Appendix D containing a self-contained existence proof for a classical solution of the coupled HJB system. The proof uses a contraction-mapping argument on the Banach space of bounded continuous functions, exploiting the uniform Lipschitz conditions on the dynamics and costs together with the positive discount factor. This renders the verification theorem and the finite-N / mean-field equivalence unconditional. revision: yes

Circularity Check

0 steps flagged

No circularity; optimality and mean-field equivalence derived from external De Finetti theorem and standard convergence results

full rationale

The derivation begins from symmetry properties of the partially exchangeable discounted costs to obtain optimality of cluster-symmetric centralized policies for finite N. It then applies a generalization of De Finetti's theorem (an external classical result) to extract a subsequential decentralized limit as N tends to infinity, and shows this limit solves the coupled McKean-Vlasov team problem under stated regularity conditions that enable tightness and convergence of empirical measures. A verification theorem is established for the mean-field limit. None of these steps reduces the claimed optimality or equivalence to a quantity defined internally by the paper's own fitting, self-definition, or unverified self-citation; the load-bearing convergence and representation arguments rest on independent external theorems rather than on quantities constructed from the target result itself. The paper is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on domain assumptions about the cost structure and population scaling together with standard mathematical tools; no new entities are postulated and no parameters are explicitly fitted to data in the abstract description.

axioms (2)
  • domain assumption Cost criteria are discounted and partially exchangeable
    Invoked to guarantee that optimal policies can be chosen exchangeable within clusters and dependent only on cluster-wise empirical distributions.
  • standard math Generalized De Finetti theorem applies to the partially exchangeable setting
    Used to obtain subsequential convergence of optimal policies to decentralized cluster-symmetric policies as population size tends to infinity.

pith-pipeline@v0.9.0 · 5602 in / 1655 out tokens · 117275 ms · 2026-05-07T08:16:30.541048+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages

  1. [1]

    Aldous, I

    D. Aldous, I. Ibragimov, and J. Jacod.Ecole d’Ete de Probabilites de Saint-Flour XIII, 1983. Springer, 1985

    work page 1983

  2. [2]

    M. A. Armstrong.Basic Topology. Undergraduate Texts in Mathematics. Springer, New York, NY, 1983

    work page 1983

  3. [3]

    Aurell, R

    A. Aurell, R. Carmona, and M. Laurière. Stochastic graphon games: Ii. the linear-quadratic case.Applied Mathematics & Optimization, 85(3):39, May 2022. 33

    work page 2022

  4. [4]

    Bardi and M

    M. Bardi and M. Fischer. On non-uniqueness and uniqueness of solutions in finite-horizon mean field games.ESAIM: Control, Optimisation and Calculus of Variations, 25, 2019

    work page 2019

  5. [5]

    Bayraktar, S

    E. Bayraktar, S. Chakraborty, and R. Wu. Graphon mean field systems. (arXiv:2003.13180), Oct. 2022

    work page arXiv 2003

  6. [6]

    Bayraktar and D

    E. Bayraktar and D. Kim. Concentration of measure for graphon particle system.Advances in Applied Probability, 56(4):1279–1306, Dec. 2024

    work page 2024

  7. [7]

    Bayraktar and R

    E. Bayraktar and R. Wu. Graphon particle system: Uniform-in-time concentration bounds. Stochastic Processes and their Applications, 156:196–225, Feb. 2023

    work page 2023

  8. [8]

    Bensoussan, T

    A. Bensoussan, T. Huang, and M. Laurière. Mean field control and mean field game models with several populations.Minimax Theory and its Applications, 3(2):173–209, Dec. 2018

    work page 2018

  9. [9]

    G. Bet, F. Coppini, and F. R. Nardi. Weakly interacting oscillators on dense random graphs. Journal of Applied Probability, 61(1):255–278, Mar. 2024

    work page 2024

  10. [10]

    Bhamidi, A

    S. Bhamidi, A. Budhiraja, and R. Wu. Weakly interacting particle systems on inhomogeneous random graphs.Stochastic Processes and their Applications, 129(6):2174–2206, June 2019

    work page 2019

  11. [11]

    Blackwell

    D. Blackwell. Memoryless strategies in finite-stage dynamic programming.The Annals of Mathematical Statistics, 35(2):863–865, 1964

    work page 1964

  12. [12]

    Budhiraja and R

    A. Budhiraja and R. Wu. Some fluctuation results for weakly interacting multi-type particle systems.Stochastic Processes and their Applications, 126(8):2253–2296, Aug. 2016

    work page 2016

  13. [13]

    N. Bäuerle. Mean field markov decision processes.Applied Mathematics & Optimization, 88(1):12, Apr. 2023

    work page 2023

  14. [14]

    P. E. Caines and M. Huang. Graphon mean field games and the gmfg equations:ε-nash equilibria. In2019 IEEE 58th Conference on Decision and Control (CDC), page 286–292, Dec. 2019

    work page 2019

  15. [15]

    P. E. Caines and M. Huang. Graphon mean field games and their equations.SIAM Journal on Control and Optimization, 59(6):4373–4399, Jan. 2021

    work page 2021

  16. [16]

    Carmona, F

    R. Carmona, F. Delarue, and D. Lacker. Mean field games with common noise.The Annals of Probability, 44(6), Nov. 2016

    work page 2016

  17. [17]

    Cui and H

    K. Cui and H. Koeppl. Learning graphon mean field games and approximate nash equilibria. ArXiv, Nov. 2021

    work page 2021

  18. [18]

    Diaconis and D

    P. Diaconis and D. Freedman. Finite exchangeable sequences.The Annals of Probability, 8(4):745–764, 1980

    work page 1980

  19. [19]

    E. Feleqi. The derivation of ergodic mean field game equations for several populations of players. Dynamic Games and Applications, 3(4):523–536, Dec. 2013

    work page 2013

  20. [20]

    M. Fujii. Probabilistic approach to mean field games and mean field type control problems with multiple populations. (arXiv:1911.11501), Nov. 2020

    work page arXiv 1911

  21. [21]

    S. Gao, R. F. Tchuendom, and P. E. Caines. Linear quadratic graphon field games.Communi- cations in Information and Systems, 21(3):341–369, 2021. 34

    work page 2021

  22. [22]

    Hernández-Lerma and J

    O. Hernández-Lerma and J. B. Lasserre.Discrete-Time Markov Control Processes. Springer, New York, NY, 1996

    work page 1996

  23. [23]

    Huang, R

    M. Huang, R. P. Malhamé, and P. E. Caines. Large population stochastic dynamic games: closed- loop Mckean-Vlasov systems and the nash certainty equivalence principle.Communications in Information & Systems, 6(3):221–252, 2006

    work page 2006

  24. [24]

    Kallenberg.Foundations of Modern Probability

    O. Kallenberg.Foundations of Modern Probability. Springer Science & Business Media, 2002

    work page 2002

  25. [25]

    Kallenberg.Probabilistic Symmetries and Invariance Principles

    O. Kallenberg.Probabilistic Symmetries and Invariance Principles. Springer Science & Business Media, 2005

    work page 2005

  26. [26]

    Lacker and A

    D. Lacker and A. Soret. A label-state formulation of stochastic graphon games and approximate equilibria on large networks.Mathematics of Operations Research, page moor.2022.1329, Nov. 2022

    work page arXiv 2022

  27. [27]

    Lambrecht and M

    G. Lambrecht and M. Laurière. Discrete-time mean field type games: Probabilistic setup. arXiv.org, Dec. 2025

    work page 2025

  28. [28]

    Lasry and P.-L

    J.-M. Lasry and P.-L. Lions. Mean field games.Japanese Journal of Mathematics, 2(1):229–260, Mar. 2007

    work page 2007

  29. [29]

    Light and G

    B. Light and G. Y. Weintraub. Mean field equilibrium: Uniqueness, existence, and comparative statics.Operations Research, 70(1):585–605, Jan. 2022

    work page 2022

  30. [30]

    Lovász.Large Networks and Graph Limits, volume 60

    L. Lovász.Large Networks and Graph Limits, volume 60. American Mathematical Society, 2012

    work page 2012

  31. [31]

    Parise and A

    F. Parise and A. Ozdaglar. Graphon games. InProceedings of the 2019 ACM Conference on Economics and Computation, EC ’19, page 457–458, New York, NY, USA, June 2019. Association for Computing Machinery

    work page 2019

  32. [32]

    Parthasarathy.Probability Measures on Metric Spaces

    K. Parthasarathy.Probability Measures on Metric Spaces. Academic Press, New York, NY, 1967

    work page 1967

  33. [33]

    Sanjari, N

    S. Sanjari, N. Saldi, and S. Yüksel. Optimality of independently randomized symmetric policies for exchangeable stochastic teams with infinitely many decision makers.Mathematics of Operations Research, 48(3):1254–1285, Aug. 2023

    work page 2023

  34. [34]

    Sanjari, N

    S. Sanjari, N. Saldi, and S. Yüksel. Optimality of symmetric independent policies under decentralized mean-field information sharing for stochastic teams and equivalence with Mckean- Vlasov control of a representative agent. (arXiv:2404.04957), Aug. 2025. (To appear in Mathematics of Operations Research)

    work page arXiv 2025

  35. [35]

    R. Serfozo. Convergence of lebesgue integrals with varying measures.Sankhy¯ a: The Indian Journal of Statistics, Series A (1961-2002), 44(3):380–402, 1982

    work page 1961

  36. [36]

    Vasal, R

    D. Vasal, R. Mishra, and S. Vishwanath. Sequential decomposition of graphon mean field games. In2021 American Control Conference (ACC), page 730–736, May 2021

    work page 2021

  37. [37]

    D.-x. Xu, Z. Gou, and N.-j. Huan. Social optima in linear quadratic graphon field control: Analysis via infinite dimensional approach.Journal of Nonlinear and Variational Analysis, 9(5), Oct. 2025. 35

    work page 2025