pith. sign in

arxiv: 2605.00588 · v1 · submitted 2026-05-01 · 🧮 math.OC

Learning-Based Stackelberg Equilibrium Seeking with Application to Demand-Side Energy Management

Silvia Cianchi , Reza Rahimi Baghbadorani , Anibal Sanjab , Sergio Grammatico This is my paper

Pith reviewed 2026-05-09 19:08 UTC · model grok-4.3

classification 🧮 math.OC
keywords demand-side managementStackelberg gamezeroth-order optimizationlearning algorithmequilibrium seekingprivacy preservationenergy pricing
0
0 comments X

The pith

A zeroth-order learning algorithm enables distribution system operators to design incentive signals that converge to a Stackelberg equilibrium in demand-side management while preserving end-user privacy.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes a learning-based method for setting dynamic electricity prices to manage demand. The method uses online estimation of how users respond to prices, drawn only from their observed actions. It proves convergence to an equilibrium where the operator's prices and users' responses are stable. This matters because it lets operators achieve grid services without full knowledge of user models or compromising privacy, and it cuts down on the number of price adjustments needed.

Core claim

The authors present a zeroth-order algorithm that iteratively updates incentive signals using data-driven estimation of end-users' responses. They prove that this process converges to an equilibrium tariff in the Stackelberg game formulation of demand-side management. The approach also enables the operator to estimate users' decision-making problems and relies solely on communicated user actions, thereby preserving privacy.

What carries the argument

The zeroth-order algorithm for incentive design assisted by data-driven online estimation of user responses.

Load-bearing premise

That the users' responses to price signals can be accurately estimated from their observed actions without additional model information.

What would settle it

Observing divergence in the price updates or failure to reduce interactions in simulations using real energy consumption data would disprove the convergence claim.

Figures

Figures reproduced from arXiv: 2605.00588 by Anibal Sanjab, Reza Rahimi Baghbadorani, Sergio Grammatico, Silvia Cianchi.

Figure 1
Figure 1. Figure 1: Convergence of the DSO objective, J0, for different values of K, with the optimal t ∗ max, and the true feasible set, C. At the bottom, a comparison with Algorithm 1, when it is not assisted by the estimation of the user response. (16) can be accurately approximated by J0(yˆ) and J0(y0) [21, Proposition 2.1]. IV. NUMERICAL RESULTS As a numerical test case, we consider the setup in [2], consisting of a EC w… view at source ↗
Figure 3
Figure 3. Figure 3: Convergence of the DSO objective, J0, for different values of tmax, with the true feasible set, C, and K = 10. We first run the simulations under the assumption that the DSO knows the feasible set of the EC, C, and the optimal number of iterations of Algorithm 2, t ∗ max, computed as in (15), which allows to maximally exploit the learned surrogate model within the trust region. In view at source ↗
read the original abstract

Demand-side management (DSM) enables distribution system operators (DSOs) to steer electricity consumption through dynamic price signals or incentive mechanisms, thereby leveraging end-users' flexibility potential for delivering grid services. The resulting hierarchical interaction between the DSO and the end-users can be formulated as a Stackelberg game, where the operator dynamically sets the prices and the end-users optimally respond to them. Efficiently designing these price signals is challenging, as the users' response models are unknown or difficult to estimate. In this paper, we propose a learning-based zeroth-order algorithm for incentive design, in which the iterative update of the incentive signals is efficiently assisted by a data-driven online estimation of the users' responses. The proposed method is then proven to converge to an equilibrium tariff while allowing the DSO to estimate the decision-making problems at the user level. Moreover, the method preserves users' privacy, as the update rule of the DSO is solely based on observations of communicated end-user actions. Numerical simulations employing real-world data illustrate the efficient convergence of our learning-based proposed method, while significantly reducing the number of required interactions between the DSO and the end-users with respect to the state-of-the-art approach.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a learning-based zeroth-order algorithm for Stackelberg equilibrium seeking in demand-side management, where the DSO iteratively designs incentive tariffs based on online data-driven estimation of end-user response maps from observed actions alone. It claims a convergence proof to the equilibrium tariff, privacy preservation (no direct access to user models or objectives), and numerical validation on real-world data showing reduced interactions versus state-of-the-art methods.

Significance. If the convergence holds under realistic DSM conditions, the result would be significant for practical incentive design in energy systems by enabling model-free, privacy-preserving tariff updates with fewer iterations. The manuscript earns credit for providing a convergence analysis and for grounding validation in real-world data simulations rather than purely synthetic cases.

major comments (2)
  1. [Convergence Analysis] Convergence theorem (likely §4 or Theorem 1): the claim that zeroth-order updates converge to the true Stackelberg equilibrium from action observations alone requires explicit regularity conditions on the user response map (e.g., Lipschitz continuity, strong monotonicity of the effective pseudo-gradient, or uniqueness of the recovered model). The inverse problem of recovering user objectives from argmax mappings is underdetermined; without these conditions stated and shown to hold for the DSM setting, the proof does not rule out bias or non-convergence.
  2. [Algorithm and Estimation] Online estimation procedure (likely §3): the manuscript states that the DSO estimates 'decision-making problems at the user level' from actions, yet multiple distinct user optimization problems can produce identical price-to-action maps. The paper must clarify whether it estimates the full problem or only the response map, and prove that the estimated map suffices for unbiased zeroth-order gradient estimates.
minor comments (2)
  1. [Abstract] Abstract: the phrase 'proven to converge' should be qualified by a one-sentence reference to the key assumptions (e.g., response-map regularity) to avoid overstatement for readers who do not reach the technical sections.
  2. [Notation] Notation and figures: ensure that the symbols for the leader's tariff variable and the followers' response functions are used consistently between the problem formulation and the algorithm pseudocode; a small table of notation would improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify key aspects of our convergence analysis and estimation procedure. We address each major comment below and will incorporate revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Convergence Analysis] Convergence theorem (likely §4 or Theorem 1): the claim that zeroth-order updates converge to the true Stackelberg equilibrium from action observations alone requires explicit regularity conditions on the user response map (e.g., Lipschitz continuity, strong monotonicity of the effective pseudo-gradient, or uniqueness of the recovered model). The inverse problem of recovering user objectives from argmax mappings is underdetermined; without these conditions stated and shown to hold for the DSM setting, the proof does not rule out bias or non-convergence.

    Authors: We agree that the convergence result benefits from explicit regularity conditions. Theorem 1 in §4 relies on the user response map being Lipschitz continuous and the effective pseudo-gradient being strongly monotone, which are standard assumptions for convex user optimization problems in DSM and ensure uniqueness of the Stackelberg equilibrium as well as unbiased zeroth-order estimates from actions. To address the comment directly, we will add a new subsection in §4 that states these conditions explicitly, verifies their validity for typical DSM settings (e.g., quadratic user costs), and shows how they rule out bias or non-convergence in the inverse mapping from actions. revision: yes

  2. Referee: [Algorithm and Estimation] Online estimation procedure (likely §3): the manuscript states that the DSO estimates 'decision-making problems at the user level' from actions, yet multiple distinct user optimization problems can produce identical price-to-action maps. The paper must clarify whether it estimates the full problem or only the response map, and prove that the estimated map suffices for unbiased zeroth-order gradient estimates.

    Authors: The online estimation in §3 recovers the response map (tariff-to-action mapping) from observed actions, not the full user optimization problem or objectives. This map is sufficient for the zeroth-order algorithm because the gradient estimates are formed via finite differences on the observed actions alone. We will revise §3 to clarify this distinction, remove any ambiguity around 'decision-making problems,' and add a supporting lemma proving that the estimated response map yields unbiased zeroth-order updates without needing the underlying objectives. This also reinforces the privacy guarantee. revision: yes

Circularity Check

0 steps flagged

No circularity: standard zeroth-order Stackelberg update with online estimation

full rationale

The derivation applies known zeroth-order optimization to a Stackelberg leader-follower game for DSM incentive design. The DSO update uses observed user actions for data-driven response estimation, then performs gradient-free steps toward equilibrium. No equation reduces by construction to a fitted input or self-definition (e.g., no parameter estimated from data then relabeled as a prediction of the same quantity). Convergence claims rest on standard assumptions from zeroth-order and game-theoretic literature rather than a self-citation chain or imported uniqueness theorem. The paper is self-contained against external benchmarks and does not rename known empirical patterns as new results.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Based solely on the abstract, the work relies on standard assumptions from game theory and zeroth-order optimization; no new entities are introduced and free parameters are not specified.

axioms (2)
  • domain assumption The underlying Stackelberg game admits convergence of the proposed iterative updates under typical conditions such as convexity of user-level problems.
    Common premise for equilibrium-seeking algorithms in hierarchical DSM models.
  • standard math Zeroth-order methods can approximate necessary information for updates using only function evaluations or observed outcomes.
    Standard technique in derivative-free optimization literature.

pith-pipeline@v0.9.0 · 5522 in / 1345 out tokens · 47915 ms · 2026-05-09T19:08:58.969075+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages

  1. [1]

    A summary of demand response in electricity markets,

    M. H. Albadi and E. F. El-Saadany, “A summary of demand response in electricity markets,”Electric power systems research, vol. 78, no. 11, pp. 1989–1996, 2008

    work page 1989

  2. [2]

    A two-part pricing mechanism for demand side management,

    S. Cianchi, A. Sanjab, and S. Grammatico, “A two-part pricing mechanism for demand side management,” in11th IEEE Interna- tional Conference on Control, Decision and Information Technologies (CoDIT), 2025, pp. 2245–2250

    work page 2025

  3. [3]

    V on Stackelberg,Market structure and equilibrium

    H. V on Stackelberg,Market structure and equilibrium. Springer, 1934

    work page 1934

  4. [4]

    A Stackelberg game for incentive-based demand response in energy markets,

    M. Fochesato, C. Cenedese, and J. Lygeros, “A Stackelberg game for incentive-based demand response in energy markets,” in61st IEEE Conference on Decision and Control (CDC), 2022, pp. 2487–2492

    work page 2022

  5. [5]

    A stochastic MPEC approach for grid tariff design with demand-side flexibility,

    M. Askeland, T. Burandt, and S. A. Gabriel, “A stochastic MPEC approach for grid tariff design with demand-side flexibility,”Energy systems, vol. 14, no. 3, pp. 707–729, 2023

    work page 2023

  6. [6]

    Machine learning for identifying demand patterns of home energy management systems with dynamic electricity pricing,

    D. Koolen, N. Sadat-Razavi, and W. Ketter, “Machine learning for identifying demand patterns of home energy management systems with dynamic electricity pricing,”Applied Sciences, vol. 7, no. 11, p. 1160, 2017

    work page 2017

  7. [7]

    Drivers of domestic electricity users’ price responsiveness: A novel machine learning approach,

    P. Guo, J. C. Lam, and V . O. Li, “Drivers of domestic electricity users’ price responsiveness: A novel machine learning approach,”Applied energy, vol. 235, pp. 900–913, 2019

    work page 2019

  8. [8]

    Luo, J.-S

    Z.-Q. Luo, J.-S. Pang, and D. Ralph,Mathematical programs with equilibrium constraints. Cambridge University Press, 1996

    work page 1996

  9. [9]

    Robust energy management for microgrids with high-penetration renewables,

    Y . Zhang, N. Gatsis, and G. B. Giannakis, “Robust energy management for microgrids with high-penetration renewables,”IEEE Transactions on Sustainable Energy, vol. 4, no. 4, pp. 944–953, 2013

    work page 2013

  10. [10]

    Optimal demand response and real-time pricing by a sequential distributed consensus- based admm approach,

    D. H. Nguyen, T. Narikiyo, and M. Kawanishi, “Optimal demand response and real-time pricing by a sequential distributed consensus- based admm approach,”IEEE Transactions on Smart Grid, vol. 9, no. 5, pp. 4964–4974, 2017

    work page 2017

  11. [11]

    Learning with adaptive conservativeness for distributionally robust optimization: Incentive design for voltage regulation,

    Z. Liang, Q. Li, J. Comden, A. Bernstein, and Y . Dvorkin, “Learning with adaptive conservativeness for distributionally robust optimization: Incentive design for voltage regulation,” in63rd IEEE Conference on Decision and Control (CDC), 2024, pp. 866–873

    work page 2024

  12. [12]

    Network- aware and welfare-maximizing dynamic pricing for energy sharing,

    A. S. Alahmed, G. Cavraro, A. Bernstein, and L. Tong, “Network- aware and welfare-maximizing dynamic pricing for energy sharing,” in63rd IEEE Conference on Decision and Control (CDC). IEEE, 2024, pp. 859–865

    work page 2024

  13. [13]

    Big hype: Best intervention in games via distributed hypergradient descent,

    P. D. Grontas, G. Belgioioso, C. Cenedese, M. Fochesato, J. Lygeros, and F. D ¨orfler, “Big hype: Best intervention in games via distributed hypergradient descent,”IEEE Transactions on Automatic Control, vol. 69, no. 12, pp. 8338–8353, 2024

    work page 2024

  14. [14]

    Follower agnostic learning in Stackelberg games,

    C. Maheshwari, J. Cheng, S. Sastry, L. Ratliff, and E. Mazumdar, “Follower agnostic learning in Stackelberg games,” in63rd IEEE Conference on Decision and Control (CDC). IEEE, 2024, pp. 222– 228

    work page 2024

  15. [15]

    Safe pricing mechanisms for distributed resource allocation with bandit feedback,

    S. Hutchinson, B. Turan, and M. Alizadeh, “Safe pricing mechanisms for distributed resource allocation with bandit feedback,”IEEE Trans- actions on Control of Network Systems, vol. 11, no. 4, pp. 2010–2021, 2024

    work page 2010

  16. [16]

    A dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach,

    R. Lu, S. H. Hong, and X. Zhang, “A dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach,” Applied Energy, vol. 220, pp. 220–230, 2018

    work page 2018

  17. [17]

    TSO-DSO-customer coordination for purchasing flexibility system services: Challenges and lessons learned from a demonstration in Sweden,

    Y . Ruwaida, J. P. Chaves-Avila, N. Etherden, I. Gomez-Arriola, G. G¨urses-Tran, K. Kessels, C. Madina, A. Sanjab, M. Santos-Mugica, D. N. Trakas, and M. Troncia, “TSO-DSO-customer coordination for purchasing flexibility system services: Challenges and lessons learned from a demonstration in Sweden,”IEEE Transactions on Power Systems, vol. 38, no. 2, pp. ...

    work page 2023

  18. [18]

    Finite-dimensional variational inequali- ties and complementarity problems,

    F. Facchinei and J. S. Pang, “Finite-dimensional variational inequali- ties and complementarity problems,” 2007

    work page 2007

  19. [19]

    Dempe,Foundations of bilevel programming

    S. Dempe,Foundations of bilevel programming. Springer, 2002

    work page 2002

  20. [20]

    Smooth minimization of non-smooth functions,

    Y . Nesterov, “Smooth minimization of non-smooth functions,”Math- ematical programming, vol. 103, no. 1, pp. 127–152, 2005

    work page 2005

  21. [21]

    New penalized stochastic gradient methods for linearly constrained strongly convex optimization,

    M. Li, P. Grigas, and A. Atamt ¨urk, “New penalized stochastic gradient methods for linearly constrained strongly convex optimization,”Jour- nal of Optimization Theory and Applications, vol. 205, no. 2, p. 29, 2025

    work page 2025

  22. [22]

    F. H. Clarke,Optimization and nonsmooth analysis. SIAM, 1990

    work page 1990

  23. [23]

    Random gradient-free minimization of convex functions,

    Y . Nesterov and V . Spokoiny, “Random gradient-free minimization of convex functions,”Foundations of Computational Mathematics, vol. 17, no. 2, pp. 527–566, 2017

    work page 2017

  24. [24]

    Convex analysis approach to DC pro- gramming: theory, algorithms and applications,

    P. D. Tao and L. H. An, “Convex analysis approach to DC pro- gramming: theory, algorithms and applications,”Acta mathematica vietnamica, vol. 22, no. 1, pp. 289–355, 1997

    work page 1997