pith. sign in

arxiv: 2509.03727 · v2 · submitted 2025-09-03 · 🧮 math.OC

Adversarial Decision-Making in Partially Observable Multi-Agent Systems: A Sequential Hypothesis Testing Approach

Haosheng Zhou , Daniel Ralston , Xu Yang , Ruimeng Hu This is my paper

Pith reviewed 2026-05-18 18:45 UTC · model grok-4.3

classification 🧮 math.OC
keywords adversarial decision-makingsequential hypothesis testingpartially observable systemsStackelberg gamedeception strategiesoptimal controllinear-quadratic dynamicsmulti-agent equilibrium
0
0 comments X

The pith

A sequential hypothesis testing framework models strategic deception as a partially observable Stackelberg game with a semi-explicit solution for the misleading follower.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets up adversarial decision-making in partially observable multi-agent systems as an ongoing exchange where one agent completes its task while trying to mislead the other through its actions. It treats the setup as a Stackelberg game in which the blue team follower uses controls that both advance its goal and create false inferences for the red team leader, who counters by exploiting leaked information to steer the blue team's choices. Sequential hypothesis testing supplies the mechanism that lets each side update beliefs about the other's intent and adapt accordingly. In the linear-quadratic case the authors obtain a semi-explicit optimal control for the blue team and supply iterative plus machine-learning procedures to recover the red team's best reply. A reader would care because the approach turns deception from an external uncertainty into an explicit, optimizable part of the strategy design.

Core claim

The authors formulate the blue team's task-completion-plus-misdirection problem and the red team's counter-misdirection problem as a partially observable Stackelberg game driven by sequential hypothesis testing. Under linear-quadratic dynamics this yields a semi-explicit optimal control law for the blue team; the red team's optimal response is then characterized by iterative and machine-learning methods. Numerical experiments show that the resulting deception-driven policies alter equilibrium behavior and that leaked information shapes the strength of the misdirection effect.

What carries the argument

The partially observable Stackelberg game driven by sequential hypothesis testing, which turns deception into a dynamic optimization problem coupling each agent's control policy to the other's inference process.

If this is right

  • The blue team's optimal policy can be obtained in semi-explicit form once the system is known to be linear-quadratic.
  • Iterative or machine-learning procedures suffice to approximate the red team's best counter-strategy against any fixed blue policy.
  • Deception alters how each agent updates its belief and therefore changes the equilibrium policies that emerge.
  • The amount and timing of leaked information directly affect the equilibrium payoff gap between the two teams.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If similar decompositions exist outside the linear-quadratic case, the same game structure could supply approximate solutions for broader classes of dynamics.
  • The explicit modeling of inference and counter-inference suggests that standard robust-control designs may underperform once agents begin to treat uncertainty as deliberate misdirection rather than noise.
  • Varying the leakage parameter in the numerical setup would give a direct, testable map from information exposure to equilibrium shift.

Load-bearing premise

The blue-red interaction can be captured and solved by casting it as a partially observable Stackelberg game whose linear-quadratic structure permits a semi-explicit optimal control for the follower.

What would settle it

A simulation in which the derived blue-team control, when paired with the iterative red-team response, produces no measurable reduction in the red team's inference accuracy or manipulation success relative to a non-deceptive baseline.

Figures

Figures reproduced from arXiv: 2509.03727 by Daniel Ralston, Haosheng Zhou, Ruimeng Hu, Xu Yang.

Figure 1
Figure 1. Figure 1: Comparisons of the optimal trajectories (1)–(2) and controls (17) with λ = 0.075 across different choices of fc: baseline fc ≡ 0, positive fc ≡ 0.5, negative fc ≡ −0.25, and periodic fc(t) = 0.5 sin(10πt) [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparisons of the optimal trajectories (1)–(2) and controls (17) with fc(t) = sin(10πt) across different values of λ [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparisons of fc, J primary( ˆα, βˆ), E[log LˆT ], optimal state trajectories (2) and controls (17) across multiple rounds of red-blue interaction within the Stackelberg game [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
read the original abstract

Adversarial decision-making in partially observable multi-agent systems requires sophisticated strategies for both deception and counter-deception. This paper presents a sequential hypothesis testing (SHT)-driven framework that captures the interplay between strategic misdirection and inference in adversarial environments. We formulate this interaction as a partially observable Stackelberg game, where a follower agent (blue team) seeks to fulfill its primary task while actively misleading an adversarial leader (red team). In opposition, the red team, leveraging leaked information, instills carefully designed patterns to manipulate the blue team's behavior, mitigating the misdirection effect. Unlike conventional approaches that focus on robust control under adversarial uncertainty, our framework explicitly models deception as a dynamic optimization problem, where both agents strategically adapt their policies in response to inference and counter-inference. We derive a semi-explicit optimal control solution for the blue team within a linear-quadratic setting and develop iterative and machine learning-based methods to characterize the red team's optimal response. Numerical experiments demonstrate how deception-driven strategies influence adversarial interactions and reveal the impact of leaked information in shaping equilibrium behaviors. These results provide new insights into strategic deception in multi-agent systems, with potential applications in cybersecurity, autonomous decision-making, and financial markets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript develops a sequential hypothesis testing (SHT)-driven framework for adversarial decision-making in partially observable multi-agent systems. It formulates the problem as a partially observable Stackelberg game in which the blue team (follower) pursues a primary task while attempting to mislead the red team (leader), who in turn exploits leaked information to design counter-strategies that manipulate the blue team's behavior. The authors derive a semi-explicit optimal control solution for the blue team under linear dynamics and quadratic costs, introduce iterative and machine-learning methods to characterize the red team's best response, and present numerical experiments illustrating the effects of deception and information leakage on equilibrium outcomes.

Significance. If the separation principle holds and the semi-explicit solution is rigorously derived, the work would offer a useful bridge between sequential hypothesis testing and dynamic game theory for modeling deception in POMDP settings. The numerical experiments, if accompanied by clear baselines and error metrics, could provide concrete evidence of how leaked information alters equilibrium strategies, with potential relevance to cybersecurity and autonomous systems.

major comments (2)
  1. [§4, Theorem 3.1] §4 (Linear-Quadratic Formulation), Theorem 3.1: The semi-explicit Riccati-based solution for the blue team's optimal control is derived under the assumption that belief-state dynamics remain independent of the red team's policy. Because the red team adaptively shapes its actions using leaked information to influence the blue team's observations and inference, the information structure becomes endogenous; this coupling generally precludes the standard separation principle and closed-form solution unless additional structural restrictions (not stated in the manuscript) are imposed on the observation model or the red team's strategy space.
  2. [§5] §5 (Numerical Experiments): The experiments claim to demonstrate the impact of deception-driven strategies and leaked information, yet no quantitative comparison to non-adaptive or non-deceptive baselines is reported, nor is an error analysis or sensitivity study with respect to the leakage parameter provided. This weakens the support for the claim that the framework reveals new equilibrium behaviors.
minor comments (2)
  1. [§2] The notation for the blue and red teams' information sets (e.g., filtration definitions) could be made more explicit in §2 to avoid ambiguity when the red team conditions on leaked signals.
  2. [§3] A few typographical inconsistencies appear in the indexing of time steps within the belief-update recursion; these do not affect the main argument but should be corrected for clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major comment point by point below, indicating the revisions we plan to incorporate to strengthen the manuscript.

read point-by-point responses

  1. Referee: [§4, Theorem 3.1] §4 (Linear-Quadratic Formulation), Theorem 3.1: The semi-explicit Riccati-based solution for the blue team's optimal control is derived under the assumption that belief-state dynamics remain independent of the red team's policy. Because the red team adaptively shapes its actions using leaked information to influence the blue team's observations and inference, the information structure becomes endogenous; this coupling generally precludes the standard separation principle and closed-form solution unless additional structural restrictions (not stated in the manuscript) are imposed on the observation model or the red team's strategy space.

    Authors: We appreciate the referee's observation on the potential endogeneity of the information structure. In the derivation of Theorem 3.1, the semi-explicit solution relies on an observation model in which the blue team's local measurements and the sequential hypothesis testing procedure are structured such that the belief-state evolution depends only on the blue team's own actions and observations, independent of the red team's policy. This is achieved through the specific linear-Gaussian observation model and the SHT framework that decouples inference from the leader's adaptive strategy. Nevertheless, to make this explicit and address the concern, we will revise §4 to clearly state these structural restrictions on the observation model and red team's strategy space that preserve the separation principle and enable the Riccati-based solution. revision: yes

  2. Referee: [§5] §5 (Numerical Experiments): The experiments claim to demonstrate the impact of deception-driven strategies and leaked information, yet no quantitative comparison to non-adaptive or non-deceptive baselines is reported, nor is an error analysis or sensitivity study with respect to the leakage parameter provided. This weakens the support for the claim that the framework reveals new equilibrium behaviors.

    Authors: We agree that the numerical section would benefit from additional quantitative support. In the revised manuscript, we will augment §5 with direct comparisons against non-adaptive and non-deceptive baseline strategies, include error metrics (e.g., mean squared deviation from equilibrium costs), and add a sensitivity analysis with respect to the leakage parameter. These additions will provide clearer quantitative evidence of the effects of deception and information leakage on equilibrium outcomes. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation remains self-contained.

full rationale

The abstract and described framework formulate the interaction as a partially observable Stackelberg game driven by sequential hypothesis testing, then derive a semi-explicit LQ optimal control for the blue team plus iterative/ML methods for the red team. No quoted equations reduce a claimed prediction or uniqueness result to a fitted parameter or self-citation by construction. Standard separation and Riccati structures are invoked under LQ assumptions without evidence that the central semi-explicit form is tautological or that load-bearing steps collapse to prior self-referential results. The derivation therefore retains independent mathematical content relative to its inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Review is based on abstract only; specific free parameters, axioms, and entities are not detailed. The framework appears to rest on standard domain assumptions from optimal control and game theory.

axioms (2)
  • domain assumption The system dynamics and costs admit a linear-quadratic structure that permits a semi-explicit optimal control solution for the blue team.
    Invoked when the abstract states derivation of the semi-explicit solution within a linear-quadratic setting.
  • domain assumption Sequential hypothesis testing can be used to model the blue team's inference and misdirection against the red team's counter-inference.
    Central to the SHT-driven framework described in the abstract.

pith-pipeline@v0.9.0 · 5750 in / 1460 out tokens · 46611 ms · 2026-05-18T18:45:11.475085+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 38 canonical work pages · 1 internal anchor

  1. [1]

    Sun Zi, The Art of War: Sun Zi’s Military Methods, Columbia University Press, 2007

    work page 2007

  2. [2]

    Aggarwal, C

    P. Aggarwal, C. Gonzalez and V . Dutt, Cyber-Security: Role of Decep- tion in Cyber-Attack Detection, Adv. Hum. Factors Cybersecurity, Proc. AHFE Int. Conf. Hum. Factors Cybersecurity, July 27-31, 2016, Florida, USA, pp 85-96

    work page 2016

  3. [3]

    R. C. Arkin, P. Ulam and A. R. Wagner, Moral Decision Making in Autonomous Systems: Enforcement, Moral Emotions, Dignity, Trust, and Deception, Proc. IEEE, vol. 100, 2011, pp 571-589

    work page 2011

  4. [4]

    Gerschlager, Deception in Markets: An Economic Analysis , Springer, 2005

    C. Gerschlager, Deception in Markets: An Economic Analysis , Springer, 2005

    work page 2005

  5. [5]

    K. Back, C. Cao and G. Willard, Imperfect Competition among Informed Traders, J. Finance, vol. 55, 2000, pp 2117-2155

    work page 2000

  6. [6]

    R. R. Yager, A Knowledge-Based Approach to Adversarial Decision Making, Int. J. Intell. Syst. , vol. 23, 2008, pp 1-21

    work page 2008

  7. [7]

    Rajendran, V

    J. Rajendran, V . Jyothi and R. Karri, Blue Team Red Team Approach to Hardware Trust Assessment, Proc. IEEE Int. Conf. Comput. Des. (ICCD), 2011, pp 285-288

    work page 2011

  8. [8]

    R. S. Liptser and A. N. Shiryaev, Statistics of Random Processes: I. General Theory, Springer Science & Business Media, 2013

    work page 2013

  9. [9]

    Tartakovsky, I

    A. Tartakovsky, I. Nikiforov and M. Basseville, Sequential Analysis: Hypothesis Testing and Changepoint Detection , CRC Press, 2014

    work page 2014

  10. [10]

    N. A. Goodman, P. R. Venkata and M. A. Neifeld, Adaptive Waveform Design and Sequential Hypothesis Testing for Target Recognition with Active Sensors, IEEE J. Sel. Top. Signal Process., vol. 1, 2007, pp 105- 113

    work page 2007

  11. [11]

    Sch ¨onbrodt, E

    F. Sch ¨onbrodt, E. Wagenmakers, M. Zehetleitner and M. Perugini, Sequential Hypothesis Testing with Bayes Factors: Efficiently Testing Mean Differences, Psychol. Methods, vol. 22, 2017, pp 322

    work page 2017

  12. [12]

    Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Springer Science & Business Media, 2009

    H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Springer Science & Business Media, 2009

    work page 2009

  13. [13]

    Wald and J

    A. Wald and J. Wolfowitz, Optimum Character of the Sequential Probability Ratio Test, Ann. Math. Stat. , 1948, pp 326-339

    work page 1948

  14. [14]

    Bain and D

    A. Bain and D. Crisan, Fundamentals of Stochastic Filtering , Springer, 2009

    work page 2009

  15. [15]

    Davis, Linear Estimation and Stochastic Control , Chapman, 1977

    M. Davis, Linear Estimation and Stochastic Control , Chapman, 1977

    work page 1977

  16. [16]

    Hor ´ak and B

    K. Hor ´ak and B. Bo ˇsansk`y, Solving Partially Observable Stochastic Games with Public Observations, Proc. AAAI Conf. Artif. Intell. , vol. 33, 2019, pp 2029-2036

    work page 2019

  17. [17]

    O. Ma, Y . Pu, L. Du, Y . Dai, R. Wang, X. Liu, Y . Wu and S. Ji, SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems, Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2024, pp 645-659

    work page 2024

  18. [18]

    Q. Liu, C. Szepesv ´ari and C. Jin, Sample-Efficient Reinforcement Learning of Partially Observable Markov Games, Adv. Neural Inf. Process. Syst., vol. 35, 2022, pp 18296-18308

    work page 2022

  19. [19]

    Kurniawati, D

    H. Kurniawati, D. Hsu and W. Lee, SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces, 2009

    work page 2009

  20. [20]

    N. Roy, G. Gordon and S. Thrun, Finding Approximate POMDP Solutions through Belief Compression, J. Artif. Intell. Res. , vol. 23, 2005, pp 1-40

    work page 2005

  21. [21]

    S. K. Kim, O. Salzman and M. Likhachev, POMHDP: Search-Based Belief Space Planning using Multiple Heuristics, Proc. Int. Conf. Autom. Plan. Sched., vol. 29, 2019, pp 734-744

    work page 2019

  22. [22]

    Lipp and S

    T. Lipp and S. Boyd, Antagonistic Control, Syst. Control Lett. , vol. 98, 2016, pp 44-48

    work page 2016

  23. [23]

    Taskesen, D

    B. Taskesen, D. Iancu, C. Koc ¸yi˘git and D. Kuhn, Distributionally Robust Linear Quadratic Control, Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 36, 2024

    work page 2024

  24. [24]

    Hakobyan and I

    A. Hakobyan and I. Yang, Wasserstein Distributionally Robust Control of Partially Observable Linear Stochastic Systems, IEEE Trans. Autom. Control, 2024

    work page 2024

  25. [25]

    Moon and T

    J. Moon and T. Bas ¸ar, Linear Quadratic Risk-Sensitive and Robust Mean Field Games, IEEE Trans. Autom. Control, vol. 62, 2016, pp 1062-1077

    work page 2016

  26. [26]

    Bauso, H

    D. Bauso, H. Tembine and T. Bas ¸ar, Robust Mean Field Games. Dyn. Games Appl., vol. 6, 2016, pp 277-303

    work page 2016

  27. [27]

    Lenhart and J

    S. Lenhart and J. Workman, Optimal Control Applied to Biological Models, Chapman, 2007

    work page 2007

  28. [28]

    McAsey, L

    M. McAsey, L. Moua and W. Han, Convergence of the Forward- Backward Sweep Method in Optimal Control, Comput. Optim. Appl. , vol. 53, 2012, pp 207-226

    work page 2012

  29. [29]

    A. Y . Ng and S. Russell, Algorithms for Inverse Reinforcement Learning, ICML, vol. 1, 2000, pp 2

    work page 2000

  30. [30]

    J. A. Sharp, K. Burrage and M. J. Simpson, Implementation and Ac- celeration of Optimal Control for Systems Biology, J. R. Soc. Interface , vol. 18, 2021

    work page 2021

  31. [31]

    G. R. Rose, Numerical Methods for Solving Optimal Control Problems, M.Sc. Thesis, University of Tennessee, Knoxville, 2015

    work page 2015

  32. [32]

    Deep Learning Approximation for Stochastic Control Problems

    J. Han and W. E, Deep Learning Approximation for Stochastic Control Problems, arXiv Preprint arXiv:1611.07422 , 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  33. [33]

    H. Zhou, D. Ralston, X. Yang, and R. Hu, Integrating Sequential Hy- pothesis Testing into Adversarial Games: A Sun Zi-Inspired Framework, arXiv preprint arXiv:2502.13462, 2025. Accepted for publication in the Proceedings of the 64th IEEE Conference on Decision and Control

    work page arXiv 2025

  34. [34]

    W. Ward, Y . Yu, J. Levy, N. Mehr, D. Fridovich-Keil, and U. Topcu, Active Inverse Learning in Stackelberg Trajectory Games,arXiv preprint arXiv:2308.08017, 2023

    work page arXiv 2023

  35. [35]

    Y . Kim, A. Benvenuti, B. Chen, M. Karabag, A. Kulkarni, N. D. Bastian, U. Topcu, and M. Hale, Deceptive Sequential Decision-Making via Regularized Policy Optimization, arXiv preprint arXiv:2501.18803 , 2025

    work page arXiv 2025

  36. [36]

    O. L. Mangasarian, Sufficient Conditions for the Optimal Control of Nonlinear Systems, SIAM Journal on control, vol. 4, 1966, pp 139-152

    work page 1966

  37. [37]

    Wanner, and E

    G. Wanner, and E. Hairer, Solving Ordinary Differential Equations II , Springer Berlin Heidelberg, vol. 375, 1996. HAOSHENG ZHOU et al.: ADVERSARIAL DECISION-MAKING IN P ARTIALL Y OBSERVABLE MUL TI-AGENT SYSTEMS 11 APPENDIX I PROOFS OF PROPOSITIONS 1–2 Proof of Proposition 1. Using the notations of Lemma 1, identify m = n = 2 , ξt as (Vt, Yt) under H0, an...

    work page 1996

  38. [38]

    + √ ∆i 4 λ rβ σ2 W − 1 2 hi 02 , ∆i := 1 r2 β (ηhi 11 + ρih02)2 + 8λreg λ rβ σ2 W − 1 2 h02. B. Detailed Derivations of FBS In the case of a quadratic penalty P[fc] = R T 0 (fc(t)−1)2 dt, the red team hopes to optimize its control fc that minimizes the expected cost (27), subject to the state dynamics (16) and (20). Therefore, the red team’s optimization ...

    work page 2025