Pith. sign in

REVIEW 5 minor 57 references

Recovery Control in Replicated Systems through Autonomous Multiagent Rollout

T0 review · 0 major / 5 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Autonomous multiagent rollout decides when to recover replicas at scale and cuts cost versus periodic policies.

desk verdict Solid first application of multiagent rollout (with a useful autonomous/signaling variant) to recovery control under correlated Byzantine failures; theory is standard DP, experiments include a real testbed, and the scaling claim holds up. read the letter →

arxiv 2607.11187 v1 pith:52YIGQF3 submitted 2026-07-13 eess.SY cs.SY

Kim Hammar , Yuchao Li This is my paper
classification eess.SYcs.SY
keywords recoverycontrolreplicatedsystemsPOMDPmultiagentrolloutsignalingByzantinefaulttolerancereinforcementlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Replicated services stay correct only if failed replicas are recovered faster than new failures arrive, but deciding which replicas to recover and when is hard: the true failure state is only partially observed, failures are correlated, and recovering a healthy replica wastes resources while leaving a failed one unrecovered risks service disruption. This paper formulates that decision as a POMDP whose control vector has one binary component per replica, then applies multiagent rollout so each replica’s agent chooses its own recovery action by one-step lookahead against a base policy. With precomputed signaling, the agents can act in parallel; the resulting autonomous multiagent rollout scales to seventy replicas, approximately improves any base policy under stated conditions, and, on both simulation and a physical testbed, lowers cost relative to the periodic and simple threshold policies used in practice.

What carries the argument

Autonomous multiagent rollout with signaling: each agent independently minimizes a one-step lookahead cost that substitutes a precomputed signaling policy for the still-unselected control components of the other agents, reducing coordination and enabling parallel computation while retaining an approximate cost-improvement property relative to the base policy.

What would settle it

Re-run the N=50 testbed experiment with a substantially different alert distribution or failure-correlation matrix; if the autonomous multiagent rollout policy no longer undercuts both the base threshold policy and the periodic policies on cost and time-to-recovery, the claimed transfer fails.

Watch

Extended reading notes

Core claim

The recovery-control problem for Byzantine-faulty service replicas under partial observability is a multiagent POMDP that can be solved approximately by autonomous multiagent rollout with signaling; the method scales to systems with up to seventy replicas and reduces operational cost compared with the recovery policies currently used in practice.

Load-bearing premise

The observation statistics measured on the authors’ testbed, together with the hybrid failure model (privileged domain can only crash; application domain can be Byzantine), are assumed faithful enough that policies optimized inside the POMDP transfer to the physical system and to other deployments.

Editorial extensions

If this is right

  • Operators of replicated services can replace fixed-period recovery schedules with a belief-driven multiagent policy that recovers only when the expected cost of inaction exceeds the cost of recovery.
  • Systems with tens of replicas become computationally tractable for online recovery control because the control-space explosion of ordinary rollout is replaced by a linear number of binary decisions that can be evaluated in parallel.
  • Any existing base policy (periodic, threshold, or heuristic) can be improved, approximately, by wrapping it in one-step multiagent rollout, provided a usable signaling policy is available.
  • The same multiagent-rollout-plus-signaling pattern applies to other POMDPs whose control is a high-dimensional binary vector and whose agents share a common belief.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The performance gap between coordinated multiagent rollout and the autonomous version is small when the neural-network signaling policy is accurate, suggesting that cheap offline imitation may be enough for many coordination-light recovery problems.
  • Because the method already uses particle filtering for the belief, it can be combined with more sophisticated base policies or longer lookahead without changing the multiagent decomposition.
  • The hybrid failure model (crash-only privileged domain) is what makes local recovery agents trustworthy; if the control plane itself can be Byzantine, an additional outer recovery layer would be required.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper formulates recovery control for replicated computing systems (with Byzantine application-domain failures, crash-only privileged agents, partial observations, and failure dependencies) as an infinite-horizon discounted POMDP. It exploits the multiagent structure of the control vector to apply sequential multiagent rollout, reducing the per-step minimization from exponential in N to linear, and introduces an autonomous variant that uses a precomputed signaling policy so that agents can act in parallel. Approximate cost-improvement guarantees (Props. 1 and 3) and complexity bounds (Props. 2 and 4) are derived under standard Bellman-operator arguments. Simulations (up to N=70) and physical testbed experiments (N=10 and N=50) show that the methods improve upon a simple belief-threshold base policy and upon the periodic recovery policies used in practice, while autonomous multiagent rollout keeps compute time nearly constant in N.

Significance. The work supplies a concrete, scalable decision-theoretic alternative to the heuristic and periodic recovery schemes that dominate Byzantine fault-tolerant systems literature. The multiagent reformulation and signaling construction are natural extensions of Bertsekas’ framework to a systems problem that has both combinatorial control spaces and real partial-observability constraints; the approximate-improvement properties rest on standard DP arguments and are therefore reliable. Closed-loop evaluation on a physical testbed whose observation statistics were collected independently of policy optimization, together with publicly released measurement data, strengthens external credibility. If the reported cost and time-to-recovery reductions transfer, the method is of immediate practical interest for cloud and consensus-based services.

minor comments (5)
  1. Abstract and §I: a few residual line-break artifacts remain (“occur .”, “faultysi- multaneously”). A final copy-edit pass would remove them.
  2. Fig. 4 and Prop. 2: the observation-space cardinality is written |Z|=1000^N in the experimental text but the general complexity statement uses |Z|; a short clarifying sentence that the experimental |Z| is (w+1)^N with w=999 would avoid momentary confusion.
  3. Appendix E: the neural-network training curves are shown only for N=10. A one-sentence statement of how (or whether) the same architecture and data-generation procedure were used for the N=70 autonomous-rollout curves in Figs. 9–10 would improve reproducibility.
  4. Eq. (1) and Fig. 3: the failure-dependency matrix A is described both as a free design parameter and as being set from software-version co-location on the testbed. Explicitly stating which construction is used in each experiment block would help readers reproduce the dependency graphs.
  5. §VI-B Testbed scenario 2: the recovery-frequency metric is reported without units or a precise definition of “per time step.” Adding the definition already used for time-to-recovery would make the three metrics fully self-contained.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: cost-improvement guarantees rest on standard Bellman-operator arguments; experimental costs are measured on held-out trajectories and a physical testbed.

full rationale

The paper formulates recovery control as a POMDP (Section III), then applies single-agent and multiagent rollout (Sections IV–V) whose approximate cost-improvement properties (Props. 1 and 3) are proved from the classical monotonicity and constant-shift properties of the Bellman operators T and T_μ (Appendices A–C). Those proofs do not embed the target recovery costs or the experimental outcomes; they hold for any base policy μ and any bounded cost-function approximation. The observation model p(z|y,u) is estimated once from independent testbed measurements (Fig. 6, §VI-A) and is thereafter treated as a fixed input; policies are evaluated on held-out simulated trajectories and on the same physical testbed under real recoveries (Figs. 12–13). The neural signaling network is ordinary imitation learning of the multiagent rollout policy and is not used to justify the theoretical claims. No self-definitional loop, fitted-input-called-prediction, or load-bearing self-citation chain is present. The derivation is therefore self-contained against its own inputs.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on a standard discounted POMDP model plus a handful of domain modeling choices (hybrid failures, partially synchronous network, empirically measured alert distributions) and a collection of free hyperparameters that define the cost weights, failure process, and rollout configuration. No new physical entities are postulated; the signaling policy is an ordinary function approximator.

free parameters (7)
  • p_F (base failure propensity) = 0.05
    Set to 0.05 in all experiments; directly scales the failure probability in Eq. (1) and therefore the absolute cost numbers.
  • Erdős–Rényi edge probability for A = 0.5
    p=0.5 used to generate the failure-dependency matrix A; changes the correlation structure of failures.
  • cost weights η, λ = η=0.2, λ∈{1.5,20}
    η=0.2, λ=1.5 (scenario 1) or λ=20 (scenario 2); trade off failure, recovery and disruption costs in Eq. (2).
  • discount factor α = 0.95
    α=0.95; standard but free choice that affects the relative value of long-horizon recovery.
  • base-policy belief threshold = 0.9
    μ recovers when b^i > 0.9; defines the starting policy that rollout improves upon.
  • rollout / particle hyperparameters (L,m,ℓ,M) = L=10,m=5,ℓ=1,M=50
    L=10, m=5, ℓ=1, M=50 particles; control the quality–compute trade-off of the approximate cost-to-go.
  • neural signaling network architecture and training = 4 hidden layers × 64, γ=1e-4
    4×64 ReLU, batch 512, lr 10^{-4}, 10^5 imitation samples; free design choices that determine signaling accuracy.
assumptions (5)
  • standard math Standard discounted infinite-horizon POMDP theory (existence of optimal cost, Bellman equation, monotonicity of T and T_μ).
    Invoked throughout §§III–V and Appendices A–C; taken from Bertsekas textbooks.
  • domain assumption Hybrid failure model: privileged domain fails only by crash; application domain may be Byzantine.
    Stated in §II and used to justify that only replica recovery (not agent recovery) needs sophisticated control.
  • domain assumption Partially synchronous network model (eventual delivery with unknown bound).
    §II; required for the underlying consensus protocol to be solvable.
  • ad hoc to paper Failure probability of replica i is min{p_F(1+∑_j A_ji x_j),1} (Eq. 1).
    Specific parametric form chosen by the authors; not derived from first principles.
  • domain assumption Observation likelihoods p(z|y,u) equal the empirical frequencies measured on the authors’ testbed.
    §VI-A and Fig. 6; the POMDP is identified from those measurements.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recovery Control in Replicated Systems through Autonomous Multiagent Rollout." pith.science (2026). https://pith.science/paper/52YIGQF3

@misc{pith2026260711187,
  author       = {Pith},
  title        = {Pith review of: Recovery Control in Replicated Systems through Autonomous Multiagent Rollout},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/52YIGQF3}},
  note         = {Machine review of arXiv:2607.11187}
}
read the original abstract

We study recovery control in replicated computing systems. Such systems consist of replicas that collectively provide a service to a client population. This redundancy enables the system to withstand failures provided that failed replicas are recovered faster than new failures occur. We show that the problem of deciding when to initiate recovery of selected replicas can be formulated as a partially observable Markov decision problem (POMDP) with a multiagent structure. We exploit this structure to apply a multiagent rollout method for approximating optimal control policies. Our method uses precomputed signaling information that reduces the need for replica coordination and facilitates parallel computations. Experiments show that our method scales to systems with up to 70 replicas and reduces costs compared to the recovery policies currently used in practice.

Figures

Figures reproduced from arXiv: 2607.11187 by the authors.

Figure 1
Figure 1. Multiagent recovery control in a system with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Probability that replica 3 has failed by time k in a system with N = 5 replicas, assuming no recoveries occur. The two plots relate to different configurations of the failure-dependency matrix A, as illustrated by the graphs to the right of each plot. An edge from node j to node i in the graph means that Aji = 1. Curves relate to different values of the parameter pF; cf. Eq. (1). Due to limited monitoring capabiliti… view at source ↗
Figure 2
Figure 2. Architecture of a node in our system model. The node is segmented [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Size of the control, state, and observation spaces as a function of the [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Conceptually, each control component u i k is selected by a separate “agent”. It is evident that this reformulated problem is equivalent to the original, since any control that is possible in one problem is also possible in the other problem, while the cost structure o…
Figure 6
Figure 6. Figure 6: Empirical observation distributions based on our testbed measurements. The measurements are available at [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: The testbed that we use for system identification and evaluation [28]. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Compute time of different rollout methods as a function of the number [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Simulation scenario 1: Comparison between the cost of the base [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 12
Figure 12. Figure 12: Testbed scenario 1: a system with N = 10 service replicas. The figure shows a performance comparison between multiagent rollout policies [cf. Eqs. (14)–(15)], the base policy [cf. Eq. (18)], the neural network signaling policy µbθ, and periodic recovery policies with …
Figure 11
Figure 11. Figure 11: Simulation scenario 2: Comparison between the cost of the base pol [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 13
Figure 13. Figure 13: Testbed scenario 2: a system with N = 50 service replicas. The figure shows a performance comparison between the autonomous multiagent rollout policy [cf. Eq. (15)], the base policy [cf. Eq. (18)], the neural network signaling policy µbθ, and periodic recovery policie…
Figure 14
Figure 14. Figure 14: Loss of the signaling policy µbθ on the training and validation datasets with N = 10; cf. Eq. (21). The dataset in Eq. (20) is divided into two parts: 80% is used for training and 20% is reserved for validation. A training epoch refers to one complete pass of the incr…
Figure 15
Figure 15. Figure 15: Compute time of rollout with different configurations of the [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 16
Figure 16. Figure 16: Simulation scenario 1: cost of the base policy µ and rollout policies with different belief estimators. Bar groups indicate the number of replicas N. Numbers and error bars indicate the mean and the standard deviation from 100 simulations with different random seeds. …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 1 canonical work pages

  1. [1]

    Probabilistic logics and synthesis of reliable organ- isms from unreliable components,

    J. von Neumann, “Probabilistic logics and synthesis of reliable organ- isms from unreliable components,” inAutomata Studies, C. Shannon and J. McCarthy, Eds. Princeton University Press, 1956, pp. 43–98

    1956

  2. [2]

    Reliable circuits using less reliable relays,

    E. F. Moore and C. E. Shannon, “Reliable circuits using less reliable relays,”Journal of the Franklin Institute, vol. 262, no. 3, pp. 191–208, 1956

    1956

  3. [3]

    The Byzantine generals problem,

    L. Lamport, R. Shostak, and M. Pease, “The Byzantine generals problem,”ACM Transactions on Programming Languages and Systems, vol. 4, no. 3, p. 382–401, jul 1982

    1982

  4. [4]

    Design of fault-tolerant computers,

    A. Avi ˇzienis, “Design of fault-tolerant computers,” inProceedings of the November 14-16, 1967, Fall joint computer conference, 1967, pp. 733–743

    1967

  5. [5]

    Consensus and coop- eration in networked multi-agent systems,

    R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and coop- eration in networked multi-agent systems,”Proceedings of the IEEE, vol. 95, no. 1, pp. 215–233, 2007

    2007

  6. [6]

    The design of a generic intrusion-tolerant architecture for web servers,

    A. Saidane, V . Nicomette, and Y . Deswarte, “The design of a generic intrusion-tolerant architecture for web servers,”IEEE Transactions on Dependable and Secure Computing, vol. 6, no. 1, pp. 45–58, 2009. 14

    2009

  7. [7]

    Practical Byzantine fault tolerance and proactive recovery,

    M. Castro and B. Liskov, “Practical Byzantine fault tolerance and proactive recovery,”ACM Transactions on Computer Systems, vol. 20, no. 4, p. 398–461, Nov. 2002

    2002

  8. [8]

    Zyzzyva: Speculative Byzantine fault tolerance,

    R. Kotla, L. Alvisi, M. Dahlin, A. Clement, and E. Wong, “Zyzzyva: Speculative Byzantine fault tolerance,”ACM Transactions on Computer Systems, vol. 27, no. 4, Jan. 2010

    2010

Show all 57 references
  1. [9]

    Bertsekas,Rollout, Policy Iteration, and Distributed Reinforcement Learning

    D. Bertsekas,Rollout, Policy Iteration, and Distributed Reinforcement Learning. Athena Scientific, 2021

    2021

  2. [10]

    Athena Scientific, 2025, 2nd edition

    ——,A Course in Reinforcement Learning. Athena Scientific, 2025, 2nd edition

    2025

  3. [11]

    Multiagent reinforcement learning: Rollout and policy iteration,

    ——, “Multiagent reinforcement learning: Rollout and policy iteration,” IEEE/CAA Journal of Automatica Sinica, vol. 8, no. 2, pp. 249–272, 2021

    2021

  4. [12]

    D. P. Bertsekas and J. N. Tsitsiklis,Neuro-Dynamic Programming. Athena Scientific, 1996

    1996

  5. [13]

    Multiagent rollout with reshuffling for warehouse robots path planning,

    W. Emanuelsson, A. P. Riveiros, Y . Li, K. H. Johansson, and J. M˚artensson, “Multiagent rollout with reshuffling for warehouse robots path planning,”IFAC-PapersOnLine, vol. 56, no. 2, pp. 3027–3032, 2023, 22nd IFAC World Congress

    2023

  6. [14]

    Multi- agent rollout and policy iteration for POMDP with application to multi- robot repair problems,

    S. Bhattacharya, S. Kailas, S. Badyal, S. Gil, and D. Bertsekas, “Multi- agent rollout and policy iteration for POMDP with application to multi- robot repair problems,” inProceedings of the 2020 Conference on Robot Learning, ser. Proceedings of Machine Learning Research, J. Ko...

    2020

  7. [15]

    Approximate mul- tiagent reinforcement learning for on-demand urban mobility problem on a large map,

    D. Garces, S. Bhattacharya, D. Bertsekas, and S. Gil, “Approximate mul- tiagent reinforcement learning for on-demand urban mobility problem on a large map,” in2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 6843–6849

    2024

  8. [16]

    Online policy adaptation for networked systems using rollout,

    F. S. Samani, K. Hammar, and R. Stadler, “Online policy adaptation for networked systems using rollout,” inNOMS 2024-2024 IEEE Network Operations and Management Symposium, 2024, pp. 1–9

    2024

  9. [17]

    Optimizing consensus- based multi-target tracking with multiagent rollout control policies,

    T. Li, L. W. Krakow, and S. Gopalswamy, “Optimizing consensus- based multi-target tracking with multiagent rollout control policies,” in 2021 IEEE Conference on Control Technology and Applications (CCTA), 2021, pp. 131–137

    2021

  10. [18]

    Rollout-based Shapley values for explainable cooperative multi-agent reinforcement learning,

    F. Ruggeri, W. Emanuelsson, A. Terra, R. Inam, and K. H. Johansson, “Rollout-based Shapley values for explainable cooperative multi-agent reinforcement learning,” in2024 IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN), 2024, pp. 227–233

    2024

  11. [19]

    Distributed online rollout for multivehicle routing in unmapped environments,

    J. W. Weber, D. R. Giriyan, D. R. Parkar, D. P. Bertsekas, and A. W. Richa, “Distributed online rollout for multivehicle routing in unmapped environments,” inProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, ser. AAMAS ’24. Richland,...

    2024

  12. [20]

    On-line pol- icy iteration with trajectory-driven policy generation,

    Y . Li, F. Chen, Y . Li, C. Fan, and D. Bertsekas, “On-line pol- icy iteration with trajectory-driven policy generation,”arXiv preprint arXiv:2604.15004, 2026

    2026 arXiv

  13. [21]

    Elements for a theory of teams,

    J. Marschak, “Elements for a theory of teams,”Management Science, vol. 1, no. 2, pp. 127–137, 1955

    1955

  14. [22]

    Survey of decen- tralized control methods for large scale systems,

    N. Sandell, P. Varaiya, M. Athans, and M. Safonov, “Survey of decen- tralized control methods for large scale systems,”IEEE Transactions on Automatic Control, vol. 23, no. 2, pp. 108–128, 1978

    1978

  15. [23]

    Generalized person-by-person optimization in team problems with binary decisions,

    D. Bauso and R. Pesenti, “Generalized person-by-person optimization in team problems with binary decisions,” in2008 American Control Conference, 2008, pp. 717–722

    2008

  16. [24]

    Can decentralized control outperform centralized? the role of communication latency,

    L. Ballotta, M. R. Jovanovi ´c, and L. Schenato, “Can decentralized control outperform centralized? the role of communication latency,” IEEE Transactions on Control of Network Systems, vol. 10, no. 3, pp. 1629–1640, 2023

    2023

  17. [25]

    Decentralized control of large collaborative swarms using random finite set theory,

    B. Doerr and R. Linares, “Decentralized control of large collaborative swarms using random finite set theory,”IEEE Transactions on Control of Network Systems, vol. 8, no. 2, pp. 587–597, 2021

    2021

  18. [26]

    Decentralized protection strategies against SIS epidemics in networks,

    S. Trajanovski, Y . Hayel, E. Altman, H. Wang, and P. Van Mieghem, “Decentralized protection strategies against SIS epidemics in networks,” IEEE Transactions on Control of Network Systems, vol. 2, no. 4, pp. 406–419, 2015

    2015

  19. [27]

    Team decision problems with convex quadratic con- straints,

    A. Gattami, “Team decision problems with convex quadratic con- straints,”IEEE Transactions on Control of Network Systems, vol. 4, no. 3, pp. 555–563, 2017

    2017

  20. [28]

    Intrusion tolerance for networked systems through two-level feedback control,

    K. Hammar and R. Stadler, “Intrusion tolerance for networked systems through two-level feedback control,” in2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2024, pp. 338–352

    2024

  21. [29]

    Intrusion tolerance as a two-level game,

    ——, “Intrusion tolerance as a two-level game,” inDecision and Game Theory for Security, A. Sinha, J. Fu, Q. Zhu, and T. Zhang, Eds. Cham: Springer Nature Switzerland, 2025, pp. 3–23

    2025

  22. [30]

    Feedback control applied to survivability: a host-based autonomic defense system,

    O. Kreidl and T. Frazier, “Feedback control applied to survivability: a host-based autonomic defense system,”IEEE Transactions on Reliabil- ity, vol. 53, no. 1, pp. 148–166, 2004

    2004

  23. [31]

    On decentralized detection with partial information sharing among sensors,

    O. P. Kreidl, J. N. Tsitsiklis, and S. I. Zoumpoulis, “On decentralized detection with partial information sharing among sensors,”IEEE Trans- actions on Signal Processing, vol. 59, no. 4, pp. 1759–1765, 2011

    2011

  24. [32]

    Dis- tributed sensor and actuator reconfiguration for fault-tolerant networked control systems,

    A. M. H. Teixeira, J. Ara ´ujo, H. Sandberg, and K. H. Johansson, “Dis- tributed sensor and actuator reconfiguration for fault-tolerant networked control systems,”IEEE Transactions on Control of Network Systems, vol. 5, no. 4, pp. 1517–1528, 2018

    2018

  25. [33]

    Online attack recovery in cyberphysical systems,

    L. Burbano, K. Garg, S. J. Leudo, A. A. Cardenas, and R. G. Sanfelice, “Online attack recovery in cyberphysical systems,”IEEE Security & Privacy, vol. 21, no. 4, pp. 20–28, 2023

    2023

  26. [34]

    Performance degradation monitoring and recovery of vision-based control systems,

    Y . Xu, S. Yin, S. X. Ding, H. Luo, and Z. Zhao, “Performance degradation monitoring and recovery of vision-based control systems,” IEEE Transactions on Control Systems Technology, vol. 29, no. 6, pp. 2712–2719, 2021

    2021

  27. [35]

    Survey of model-based failure detection and isolation in complex plants,

    J. Gertler, “Survey of model-based failure detection and isolation in complex plants,”IEEE Control Systems Magazine, vol. 8, no. 6, pp. 3–11, 1988

    1988

  28. [36]

    Cachin, R

    C. Cachin, R. Guerraoui, and L. Rodrigues,Introduction to Reliable and Secure Distributed Programming, 2nd ed. Springer Publishing Company, Incorporated, 2011

    2011

  29. [37]

    Consensus in the presence of partial synchrony,

    C. Dwork, N. Lynch, and L. Stockmeyer, “Consensus in the presence of partial synchrony,”Journal of the ACM, vol. 35, no. 2, p. 288–323, apr 1988

    1988

  30. [38]

    Impossibility of distributed consensus with one faulty process,

    M. J. Fischer, N. A. Lynch, and M. S. Paterson, “Impossibility of distributed consensus with one faulty process,”Journal of the ACM, vol. 32, no. 2, p. 374–382, apr 1985

    1985

  31. [39]

    Byzantine fault-tolerant state-machine replication from a systems perspective,

    T. Distler, “Byzantine fault-tolerant state-machine replication from a systems perspective,”ACM Computing Surveys, vol. 54, no. 1, feb 2021

    2021

  32. [40]

    State transfer for hypervisor- based proactive recovery of heterogeneous replicated services,

    T. Distler, R. Kapitza, and H. Reiser, “State transfer for hypervisor- based proactive recovery of heterogeneous replicated services,”FERS- Mitteilungen, vol. 29, no. 1, 2011

    2011

  33. [41]

    Efficient Byzantine-resilient reliable multicast on a hybrid failure model,

    M. Correia, L. C. Lung, N. Neves, and P. Verissimo, “Efficient Byzantine-resilient reliable multicast on a hybrid failure model,” in21st IEEE Symposium on Reliable Distributed Systems, 2002. Proceedings., 2002, pp. 2–11

    2002

  34. [42]

    Optimal control of Markov processes with incomplete state information,

    K. J. ˚Astr¨om, “Optimal control of Markov processes with incomplete state information,”Journal of Mathematical Analysis and Applications, vol. 10, no. 1, pp. 174–205, 1965

    1965

  35. [43]

    Krishnamurthy,Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing

    V . Krishnamurthy,Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing. Cambridge University Press, 2016

    2016

  36. [44]

    Bertsekas,Dynamic Programming and Optimal Control: Vol

    D. Bertsekas,Dynamic Programming and Optimal Control: Vol. II, 4th ed. Athena Scientific, 2012

    2012

  37. [45]

    Adaptive network security policies via belief aggregation and rollout,

    K. Hammar, Y . Li, T. Alpcan, E. C. Lupu, and D. Bertsekas, “Adaptive network security policies via belief aggregation and rollout,”IEEE Transactions on Information Forensics and Security, pp. 1–1, 2026, 10.1109/TIFS.2026.3709131

    2026 doi

  38. [46]

    Multi- agent reinforcement learning: Rollout and policy iteration for POMDP with application to multirobot problems,

    S. Bhattacharya, S. Kailas, S. Badyal, S. Gil, and D. Bertsekas, “Multi- agent reinforcement learning: Rollout and policy iteration for POMDP with application to multirobot problems,”Transactions on Robotics, vol. 40, p. 2003–2023, Jan. 2024

    2003

  39. [47]

    Feature-based belief aggregation for partially observable Markov decision problems,

    Y . Li, K. Hammar, and D. Bertsekas, “Feature-based belief aggregation for partially observable Markov decision problems,”arXiv preprint arXiv:2507.04646, 2025

    2025 arXiv

  40. [48]

    Scalable solutions to zero-sum partially observable stochastic games through belief aggregation with approxi- mation guarantees,

    K. Hammar and T. Alpcan, “Scalable solutions to zero-sum partially observable stochastic games through belief aggregation with approxi- mation guarantees,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 20, p. 17023–17031, Mar. 2026

    2026

  41. [49]

    Bertsekas,Reinforcement Learning and Optimal Control

    D. Bertsekas,Reinforcement Learning and Optimal Control. Athena Scientific, 2019

    2019

  42. [50]

    Rollout approach to sensor scheduling for remote state estimation under integrity attack,

    H. Liu, Y . Li, K. H. Johansson, J. M ˚artensson, and L. Xie, “Rollout approach to sensor scheduling for remote state estimation under integrity attack,”Automatica, vol. 144, p. 110473, 2022

    2022

  43. [51]

    Optimal security response to network intrusions in IT systems,

    K. Hammar, “Optimal security response to network intrusions in IT systems,” Ph.D. dissertation, KTH Royal Institute of Technology, 2024

    2024

  44. [52]

    Bertsekas,Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control

    D. Bertsekas,Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control. Athena Scientific, 2022

    2022

  45. [53]

    Monotone mappings with application in dynamic programming,

    D. P. Bertsekas, “Monotone mappings with application in dynamic programming,”SIAM Journal on Control and Optimization, vol. 15, no. 3, pp. 438–464, 1977

    1977

  46. [54]

    Docker: lightweight Linux containers for consistent devel- opment and deployment,

    D. Merkel, “Docker: lightweight Linux containers for consistent devel- opment and deployment,”Linux journal, vol. 2014, no. 239, p. 2, 2014. 15

    2014

  47. [55]

    Snort - lightweight intrusion detection for networks,

    M. Roesch, “Snort - lightweight intrusion detection for networks,” in Proceedings of the 13th USENIX Conference on System Administration, ser. LISA ’99. USA: USENIX Association, 1999, p. 229–238

    1999

  48. [56]

    Implementing fault-tolerant services using the state machine approach: A tutorial,

    F. B. Schneider, “Implementing fault-tolerant services using the state machine approach: A tutorial,”ACM Computing Surveys, vol. 22, no. 4, p. 299–319, dec 1990

    1990

  49. [57]

    Efficient Byzantine fault-tolerance,

    G. S. Veronese, M. Correia, A. N. Bessani, L. C. Lung, and P. Verissimo, “Efficient Byzantine fault-tolerance,”IEEE Transactions on Computers, vol. 62, no. 1, pp. 16–30, 2013

    2013

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.