Pith. sign in

REVIEW 3 major objections 4 minor 46 references

STAIR: Effective Incident Response Using an End-to-End Agentic Planning Framework

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A graph-state agent framework routes incident response and lifts defense scores to 0.94.

desk verdict Solid systems integration with a genuinely held-out benchmark, but the headline 9.5% gain is carried entirely by the experience database, whose effect labels come from the same evaluation profiles used to compute the final score; that structural advantage needs a harder look before the number is taken at face value. read the letter →

arxiv 2608.09524 v1 pith:4WWMJCEF submitted 2026-08-10 cs.CR cs.AI

Hanlin Jiang , Jionghao Huang , Shaofei Li , Bojia Yu , Peng Jiang , Yuxin Ren , Ning Jia , Yao Guo
show 1 more author
Ding Li
This is my paper · ORCID
classification cs.CRcs.AI
keywords incidentresponseagenticplanningGraph-as-StateLLMagentsexperiencereusecyberrangerecoverystageSOAR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that automated incident response gets substantially better when planning is anchored in a live, evolving picture of the incident rather than in open-ended text or tool traces. It presents STAIR, a framework that keeps the incident as a graph, decides which of six recovery stages to advance, hands planning to a stage-specialized agent, and executes actions through a harness that records their real effects. On 100 Docker-based cyber ranges, STAIR reports a normalized overall defense score of 0.94, against 0.86 for the strongest baseline, a relative improvement of 9.5%. If this holds, it would mean long-horizon response failures that plague LLM agents, such as state loss, stage mixing, and weak experience reuse, are addressable by structural workflow design rather than by a bigger model.

What carries the argument

The load-bearing mechanism is the coupling of three objects: Graph-as-State (GAS), the Stage-Specialized Agent Planner, and an effect-validated Experience Database. GAS is an attributed graph holding machines, artifacts, defender nodes, attack edges, and response edges, so the incident's past and present live in one updatable structure rather than in a prompt or a trace. The Stage Router turns that graph into a dispatch signal $(s_t, \rho_t)$, selecting a stage and recovery focus, and the corresponding agent plans with stage-scoped tools and memory. The Execution Harness closes the loop: it runs the action in the live environment, writes feedback into GAS, and labels the action effective, high-risk, or ineffective using an incident-specific profile, which is exactly the signal that builds reusable experience for later states.

What would settle it

Re-run the 30 held-out ranges twice: once with the experience database built from effect labels generated by one rubric, and once with labels generated by a second, independently designed rubric, keeping the final score computed from the original hidden profiles. If the ODS advantage over the no-experience variant largely disappears or changes sign across the two label rubrics, the reported gain depends on information leaked from the scoring profiles rather than on transferable response knowledge.

Watch

Extended reading notes

Core claim

STAIR's central claim is that the decision substrate matters more than model power: response planning should be a stateful, stage-aware, experience-supported loop. The framework represents the incident at each step as Graph-as-State (GAS), a graph whose nodes are machines, artifacts, and defenders and whose edges are attack and response behaviors, updated continuously from alert data and execution feedback. A Stage Router reads GAS and picks one of six recovery stages, containment, assessment, preservation, eviction, hardening, restoration, and dispatches a stage-specialized agent whose tools, memory, and experience skill are scoped to that stage. After each action, the Execution Harness validates the effect against an incident-specific profile and writes the tuple of pre-action state, action, effect label, and feedback into an Experience Database, so future planning retrieves proven actions and warnings. In evaluation on 100 Docker-based ranges (70 training, 30 held-out), STAIR reaches an overall defense score of 0.9409 versus 0.8589 for the best baseline, and the ablations show each component contributes: removing experience drops ODS to 0.8250, removing graph state further to 0.6889, and removing stage planning to 0.6031.

Load-bearing premise

The load-bearing assumption is that the effect labels used to build the experience database are an independent measure of response quality and not just the same evaluation profiles used to compute the final score; if they are the same signal, the system is partly being scored on what it was trained to see.

Editorial extensions

If this is right

  • If the central claim is right, incident-response agents should be organized as stateful workflows rather than raw tool users; open-ended agents will keep losing track of attack paths in long runs.
  • A persistent graph state plus stage-scoped tools and memory should keep recovery actions aligned with the current objective, avoiding the stage-objective mixing seen in baselines.
  • Effect-validated experience reuse should transfer across incident variants, including previously unseen families, without replaying exact trajectories.
  • The component ablations imply that the 9.5% gain is a joint effect of graph state, stage planning, and experience, not a single trick, so removing any one component measurably degrades response.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit but testable consequence is that the framework's value depends on the independence of the effect-labeling profiles from the final scoring profiles; if they share signal, part of the reported gain is by construction.
  • The authors' prototype assumes a fixed six-stage recovery lifecycle and a closed set of 33 response capabilities; extending it to adaptive stage definitions or ad-hoc tools would test how much of the benefit is structural versus task-specific.
  • The same graph-state/stage-dispatch/experience-validation loop could generalize to other long-horizon operational workflows, such as vulnerability remediation or cloud outage recovery, where state, stage, and validated actions are the natural units.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents STAIR, an end-to-end agentic planning framework for incident response that combines Graph-as-State (GAS), a stage-specialized multi-agent planner, an Experience Database, and an Execution Harness with bounded tool execution. The authors evaluate the framework on 100 Docker-based cyber ranges with a 70/30 train/test split, reporting an Overall Defense Score (ODS) of 0.9409 against 0.8589 for the strongest baseline (CC-DeepSeek V4 Pro), a 9.5% relative improvement. Additional experiments address robustness across incident complexity tiers, cumulative ablations of the three main components, and transfer of experience to held-out variants.

Significance. If the evaluation concern below is resolved, the work is practically significant: it demonstrates that a stateful, stage-aware architecture with validated experience reuse improves closed-loop incident response over open-ended LLM agents, text planners, and RL-style baselines on a nontrivial executable benchmark. The paper's strengths include a clearly described system design, a held-out test split, cumulative ablations that show coherent degradation, and a separate analysis of experience transfer. The code is linked, which supports reproducibility, although the benchmark itself is not released. The main empirical claim, however, depends on the Experience Database component, whose effect labels are generated from the same incident-specific evaluation profiles used for final scoring; the ablations show this component is responsible for the entire headline gain, so the claimed advantage is not yet established as independent of the scoring function.

major comments (3)
  1. [§V(d), §VI-A, Eq. (4), Table III] The Experience Database is constructed from effect labels y_i produced by the Execution Harness using the same incident-specific evaluation profiles that later compute ODS. The paper states that the profile is not exposed to the planner or written into GAS, but it is used to label experiences, and retrieval returns stage-compatible records ranked by state similarity. For held-out variants of training families, the retrieved positive experiences therefore encode what the hidden scoring profile rewards. Table III shows that removing only the Experience Database reduces ODS from 0.9409 to 0.8250, below the strongest baseline (0.8589), so the entire headline improvement is attributable to this channel. To support the claimed 9.5% improvement, please either build the Experience Database from profiles independent of the test scoring profiles, hold out entire families from both training and retrieval, or otherwise demonstrate that retrieved experiences do not encode the test reward function; the current ablation does not separate 'useful historical experience' from 'oracle access to the scoring function.'
  2. [Table II and Table III] All metrics are macro-averaged point estimates from a single run over 30 test ranges, with no variance, confidence intervals, or significance tests. The central 9.5% margin and the ablations in Table III should be accompanied by per-range distributions and paired statistical tests (e.g., bootstrap confidence intervals or a Wilcoxon signed-rank test across ranges). This is especially important because several differences are driven by a small number of cases: for example, AttackSup 0.9333 versus 1.0000 corresponds to one range out of 30.
  3. [§VI-A and §VI-F] The claim that experience reuse transfers to held-out incidents is weakened by the composition of the test set: 14 of the 20 held-out cases are seen-family variants, while only 6 come from previously unseen families. For seen-family variants, training experiences from the same family are retrieved under similar GAS states and can carry family-specific information about the scoring profile. RQ4 reports a 14.7% ODS gain on seen-family variants versus only 8.2% on unseen families; please report the seen/unseen split with confidence intervals and treat the unseen-family result as the primary evidence for generalization, since it is the only part of the transfer claim not confounded by profile-specific experience.
minor comments (4)
  1. [Cross-references] The manuscript mixes notations such as '§4.1' in the figure caption and '§IV-A' in the text; please unify the cross-reference style throughout.
  2. [Table III] In the last row, '7.400' appears with inconsistent spacing; also, TTR is only meaningful as a comparison metric when response lengths are commensurable, so please clarify whether the shorter TTR in the ablated settings reflects truncated failures rather than efficiency.
  3. [Figure 4] The figure legend uses 'CC+DeepSeek' and 'CC+Opus 4.7' while Table II uses 'CC-DeepSeek V4 Pro' and 'CC-Opus 4.7'; please align the names across the paper.
  4. [Reference [30] and artifact statement] The code repository is given as an anonymous link only; please add a formal availability statement with version, license, and expected runtime/configuration requirements, and state whether the benchmark ranges will be released.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Experience Database is built from training-range labels, and held-out test scores are neither used to label retrieved experiences nor written back to the database.

full rationale

The paper's central claim is an empirical evaluation on held-out ranges, not a derivation that reduces to its inputs. The Experience Database is constructed offline from training-range trajectories, with effect labels y_i produced by the Execution Harness using incident-specific evaluation profiles (Eq. (4), Sec. IV-B, Sec. V(b)). Test-range ODS is computed from hidden evaluation profiles (Sec. VI-A), and the paper explicitly states that test trajectories are not written back to the Experience Database. The same profile is not used both to label a retrieved record and to score the test run that retrieves it: training profiles label the database, while held-out test profiles (including seen-family variants and unseen families) determine the final score. Retrieved experiences therefore carry training-time reward information, which is a legitimate inductive signal rather than a circular re-statement of the test output. The w/o-E ablation (Table III) attributes most of the gain to the Experience Database, but this is an empirical attribution finding, not a definitional equivalence: no equation equates y_i with ODS, and the paper does not feed test profiles into the planner or the database. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation. The benchmark is self-constructed but properly split into training and held-out test ranges, so the derivation chain is self-contained.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central empirical claim rests on several domain assumptions about state representation, stage ontology, experience transfer, and benchmark representativeness. No numeric parameters are fitted except the hand-chosen ODS metric weights and the unspecified step budget; no new physical entities are introduced.

free parameters (2)
  • ODS metric weights = 0.40 Recovery, 0.30 AttackSup, 0.15 AESR, 0.15 SCS
    Equation (6) defines the primary metric with hand-chosen weights; the headline 0.94 depends on this weighting, and no sensitivity analysis is given.
  • Response step budget
    All methods stop at a step budget that is never specified; it bounds TTR and may favor some methods, but it is shared across baselines.
assumptions (6)
  • domain assumption The incident state can be faithfully represented as an attributed graph G_t = (V_t, E_t, A_t) over machines, artifacts, and defender nodes.
    Invoked in Section IV-A; if important incident information does not fit into this schema, state maintenance and stage routing lose fidelity.
  • domain assumption The six-stage space S (containment, assessment, preservation, eviction, hardening, restoration) is a complete and mutually exclusive partition of incident response progress.
    Defined in Section IV-C; the Stage Router's dispatch and the stage-specific agents rely on this stage ontology.
  • domain assumption Historical response experiences transfer across incident variants through embedding similarity of serialized graph states.
    Used in Section IV-B and V(b); retrieval quality depends on stella_en_1.5B_v5 embeddings capturing state similarity across new incidents.
  • domain assumption The Docker-based cyber ranges and their hidden evaluation profiles are representative of real incident response and accurately measure defense quality.
    Invoked in Section VI-A; all conclusions generalize only as far as this synthetic benchmark represents real operations.
  • domain assumption The LLM backend (deepseek-v4-pro) and the profile-generation skill produce reliable reasoning and correct effect labels.
    Used throughout Section V; no independent verification of LLM output correctness is provided.
  • domain assumption The effect label y_i from the execution profile correctly separates effective, high-risk, and ineffective actions.
    Required by Eq. (4) and Section IV-B; incorrect labels would poison the Experience Database.

how reviews work

0 comments
Cite this review

Pith. "Pith review of STAIR: Effective Incident Response Using an End-to-End Agentic Planning Framework." pith.science (2026). https://pith.science/paper/4WWMJCEF

@misc{pith2026260809524,
  author       = {Pith},
  title        = {Pith review of: STAIR: Effective Incident Response Using an End-to-End Agentic Planning Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4WWMJCEF}},
  note         = {Machine review of arXiv:2608.09524}
}
read the original abstract

Incident response planning is critical for restoring compromised software systems after cyberattacks. Common practice relies on expert-driven playbooks that encode fixed response procedures, but these static workflows struggle to adapt to evolving incident states, changing recovery objectives, and execution feedback. Recent LLM-based planners and tool-using agents improve automation, yet they remain unstable in long-horizon response because they lack a unified basis for maintaining incident state, aligning actions with the current recovery stage, and reusing historical experience. We present STAIR, an end-to-end agentic planning framework for incident response. The framework maintains the current incident as Graph-as-State, uses a Stage Router to dispatch planning to stage-specialized agents, and retrieves historical experiences to guide action selection. An Execution Harness executes actions, returns feedback to update the incident state, and validates action effects for future experience reuse. Across 100 Docker-based cyber ranges, our framework achieves a normalized defense score of 0.94 and improves over the strongest baseline by 9.5%.

Figures

Figures reproduced from arXiv: 2608.09524 by the authors.

Figure 1
Figure 1. The workflow of the proposed agentic incident response framework. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. GAS snapshot and hardening update in the running example. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Compressed action-generation prompt for the hardening step in the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Attack suppression and overall defense score across attack scenario [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Retrieved experience record that helps complete restoration-stage [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 38 canonical work pages

  1. [1]

    Sp 800-61 rev. 1. computer security incident handling guide,

    K. A. Scarfone, T. Grance, and K. Masone, “Sp 800-61 rev. 1. computer security incident handling guide,” 2008

    work page 2008

  2. [2]

    Lessons lost: Incident response in the age of cyber insurance and breach attorneys,

    D. W. Woods, R. Böhme, J. Wolff, and D. Schwarcz, “Lessons lost: Incident response in the age of cyber insurance and breach attorneys,” in32nd USENIX Security Symposium (USENIX Security 23), 2023, pp. 2259–2273

    work page 2023

  3. [3]

    Do you play it by the books? a study on incident response playbooks and influencing factors,

    D. Schlette, P. Empl, M. Caselli, T. Schreck, and G. Pernul, “Do you play it by the books? a study on incident response playbooks and influencing factors,” in2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024, pp. 3625–3643

    work page 2024

  4. [4]

    How ready is your ready? assessing the usability of incident response playbook frameworks,

    R. Stevens, D. V otipka, J. Dykstra, F. Tomlinson, E. Quartararo, C. Ah- ern, and M. L. Mazurek, “How ready is your ready? assessing the usability of incident response playbook frameworks,” inProceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1–18

    work page 2022

  5. [5]

    Security operations center: A systematic study and open challenges,

    M. Vielberth, F. Böhm, I. Fichtinger, and G. Pernul, “Security operations center: A systematic study and open challenges,”Ieee Access, vol. 8, pp. 227 756–227 779, 2020

    work page 2020

  6. [6]

    Evolution of endpoint detection and response (edr) in cyber security: A comprehensive review,

    H. Kaur, D. Sanjaiy SL, T. Paul, R. Kumar Thakur, K. V . Kumar Reddy, J. Mahato, and K. Naveen, “Evolution of endpoint detection and response (edr) in cyber security: A comprehensive review,” inE3S Web of Conferences, vol. 556. EDP Sciences, 2024, p. 01006

    work page 2024

  7. [7]

    Security information and event management (siem): analysis, trends, and usage in critical infrastructures,

    G. González-Granadillo, S. González-Zarzosa, and R. Diaz, “Security information and event management (siem): analysis, trends, and usage in critical infrastructures,” 2021

    work page 2021

  8. [8]

    A comprehensive analysis of features, benefits, challenges, and best practices of security information and event management (siem) solutions,

    M. Vardalachakis, M. Vasilakis, and M. Tampouratzis, “A comprehensive analysis of features, benefits, challenges, and best practices of security information and event management (siem) solutions,” inComputer Sciences & Mathematics Forum, vol. 12, no. 1. MDPI, 2026, p. 18

    work page 2026

Show all 46 references
  1. [9]

    Requirements for playbook-assisted cyber incident response, reporting and automation,

    M. Akbari Gurabi, L. Nitz, A. Bregar, J. Popanda, C. Siemers, R. Matzutt, and A. Mandal, “Requirements for playbook-assisted cyber incident response, reporting and automation,”Digital Threats: Research and Practice, vol. 5, no. 3, pp. 1–11, 2024

    2024

  2. [10]

    Ircopilot: Automated incident response with large language models,

    X. Lin, J. Zhang, G. Deng, T. Liu, T. Zhang, Q. Guo, and R. Chen, “Ircopilot: Automated incident response with large language models,” arXiv preprint arXiv:2505.20945, 2025

    2025

  3. [11]

    Incident response planning using a lightweight large language model with reduced hallucination,

    K. Hammar, T. Alpcan, and E. C. Lupu, “Incident response planning using a lightweight large language model with reduced hallucination,” arXiv preprint arXiv:2508.05188, 2025

    2025 arXiv

  4. [12]

    Cost of a data breach report 2024,

    I. Secur, “Cost of a data breach report 2024,”Accessed: Jan, vol. 27, p. 2025, 2024

    2024

  5. [13]

    In-context autonomous network incident response: An end-to-end large language model agent approach,

    Y . Gao, K. Hammar, and T. Li, “In-context autonomous network incident response: An end-to-end large language model agent approach,”arXiv preprint arXiv:2602.13156, 2026

    2026 arXiv

  6. [14]

    Adaptive security response strategies through conjectural online learning,

    K. Hammar, T. Li, R. Stadler, and Q. Zhu, “Adaptive security response strategies through conjectural online learning,”IEEE Transactions on Information Forensics and Security, 2025

    2025

  7. [15]

    Learning near-optimal intrusion responses against dynamic attackers,

    K. Hammar and R. Stadler, “Learning near-optimal intrusion responses against dynamic attackers,”IEEE Transactions on Network and Service Management, vol. 21, no. 1, pp. 1158–1177, 2023

    2023

  8. [16]

    Operations- informed incident response playbooks,

    A. Shaked, Y . Cherdantseva, P. Burnap, and P. Maynard, “Operations- informed incident response playbooks,”Computers & Security, vol. 134, p. 103454, 2023

    2023

  9. [17]

    Hallucination-resistant se- curity planning with a large language model,

    K. Hammar, T. Alpcan, and E. Lupu, “Hallucination-resistant se- curity planning with a large language model,”arXiv preprint arXiv:2602.05279, 2026

    2026

  10. [18]

    Employing llms for incident response planning and review,

    S. Hays and J. White, “Employing llms for incident response planning and review,”arXiv preprint arXiv:2403.01271, 2024

    2024 arXiv

  11. [19]

    Ai-driven guided response for security operation centers with microsoft copilot for security,

    S. Freitas, J. Kalajdjieski, A. Gharib, and R. McCann, “Ai-driven guided response for security operation centers with microsoft copilot for security,” inCompanion Proceedings of the ACM on Web Conference 2025, 2025

    2025

  12. [20]

    Ai incident response playbook generator,

    M. Kumar, S. Sengar, S. Mondal, A. Duaet al., “Ai incident response playbook generator,” in2026 IEEE International Conference on In- terdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI), vol. 4. IEEE, 2026, pp. 1–4

    2026

  13. [21]

    Large language models are autonomous cyber defenders,

    S. R. Castro, R. Campbell, N. Lau, O. Villalobos, J. Duan, and A. A. Cardenas, “Large language models are autonomous cyber defenders,” in 2025 IEEE Conference on Artificial Intelligence (CAI). IEEE, 2025, pp. 1125–1132

    2025

  14. [22]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,”arXiv preprint arXiv:2210.03629, 2022

    2022 arXiv

  15. [23]

    Structural generalization in autonomous cyber incident response with message-passing neural networks and reinforcement learning,

    J. Nyberg and P. Johnson, “Structural generalization in autonomous cyber incident response with message-passing neural networks and reinforcement learning,” in2024 IEEE International Conference on Cyber Security and Resilience (CSR). IEEE, 2024, pp. 282–289

    2024

  16. [24]

    Reward shaping for happier autonomous cyber security agents,

    E. Bates, V . Mavroudis, and C. Hicks, “Reward shaping for happier autonomous cyber security agents,” inProceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023, pp. 221–232

    2023

  17. [25]

    Deep reinforcement learning for cyber security,

    T. T. Nguyen and V . J. Reddi, “Deep reinforcement learning for cyber security,”IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 8, pp. 3779–3795, 2021

    2021

  18. [26]

    Advancing llm-based security automation with customized group relative policy optimization for zero-touch networks,

    X. Cao, Y . Lin, G. Nan, Q. Zhou, Y . Luo, Y . Gao, Z. Zhang, H. Lu, Q. Cui, Y . Houet al., “Advancing llm-based security automation with customized group relative policy optimization for zero-touch networks,” IEEE Journal on Selected Areas in Communications, 2025

    2025

  19. [27]

    Webthinker: Empowering large reasoning models with deep research capability,

    X. Li, J. Jin, G. Dong, H. Qian, Y . Wu, J.-R. Wen, Y . Zhu, and Z. Dou, “Webthinker: Empowering large reasoning models with deep research capability,”Advances in Neural Information Processing Systems, vol. 38, pp. 120 091–120 131, 2026

    2026

  20. [28]

    Symbolic learning enables self-evolving agents,

    Y . Ou, W. Zhou, S. Ding, L. Li, J. Wu, T. Wang, J. Chen, S. Wang, X. Xu, N. Zhanget al., “Symbolic learning enables self-evolving agents,”AI Open, 2025

    2025

  21. [29]

    Dealing with sparse rewards in reinforcement learning,

    J. Hare, “Dealing with sparse rewards in reinforcement learning,”arXiv preprint arXiv:1910.09281, 2019

    1910 arXiv

  22. [30]

    Code repository,

    “Code repository,” https://anonymous.4open.science/r/stair

  23. [31]

    A survey on large language model based autonomous agents,

    L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y . Linet al., “A survey on large language model based autonomous agents,”Frontiers of Computer Science, vol. 18, no. 6, p. 186345, 2024

    2024

  24. [32]

    Understanding the planning of llm agents: A survey,

    X. Huang, W. Liu, X. Chen, X. Wang, H. Wang, D. Lian, Y . Wang, R. Tang, and E. Chen, “Understanding the planning of llm agents: A survey,”arXiv preprint arXiv:2402.02716, 2024

    2024 arXiv

  25. [33]

    A survey on the memory mechanism of large language model- based agents,

    Z. Zhang, Q. Dai, X. Bo, C. Ma, R. Li, X. Chen, J. Zhu, Z. Dong, and J.- R. Wen, “A survey on the memory mechanism of large language model- based agents,”ACM Transactions on Information Systems, vol. 43, no. 6, pp. 1–47, 2025

    2025

  26. [34]

    Swe-agent: Agent-computer interfaces enable automated soft- ware engineering,

    J. Yang, C. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “Swe-agent: Agent-computer interfaces enable automated soft- ware engineering,”Advances in Neural Information Processing Systems, 2024

    2024

  27. [35]

    Repairagent: An autonomous, llm-based agent for program repair,

    I. Bouzenia, P. Devanbu, and M. Pradel, “Repairagent: An autonomous, llm-based agent for program repair,” in2025 IEEE/ACM 47th Interna- tional Conference on Software Engineering (ICSE), 2025

    2025

  28. [36]

    Evaluating agent-based program repair at google,

    P. Rondon, R. Wei, J. Cambronero, J. Cito, A. Sun, S. Sanyam, M. Tufano, and S. Chandra, “Evaluating agent-based program repair at google,” in2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2025

    2025

  29. [37]

    Llm-based multi-agent systems for software engineering: Literature review, vision, and the road ahead,

    J. He, C. Treude, and D. Lo, “Llm-based multi-agent systems for software engineering: Literature review, vision, and the road ahead,” ACM Transactions on Software Engineering and Methodology, 2025

    2025

  30. [38]

    Chatdev: Communicative agents for software development,

    C. Qian, W. Liu, H. Liu, N. Chen, Y . Dang, J. Li, C. Yang, W. Chen, Y . Su, X. Conget al., “Chatdev: Communicative agents for software development,” inProceedings of the 62nd annual meeting of the asso- ciation for computational linguistics (volume 1: Long papers), 2024

    2024

  31. [39]

    Contextpilot: Code context engineering with memory-augmented exploration agents,

    S. Gao, C. Wang, S. Li, Y . Peng, and M. R. Lyu, “Contextpilot: Code context engineering with memory-augmented exploration agents,” 2026

    2026

  32. [40]

    What are playbooks?

    P. A. Networks, “What are playbooks?” https://docs-cortex.paloaltone tworks.com/r/Cortex-XSOAR/6.x/Cortex-XSOAR-Playbook-Design-G uide/What-Are-Playbooks

  33. [41]

    Recommended playbook use cases, templates, and exam- ples,

    Microsoft, “Recommended playbook use cases, templates, and exam- ples,” https://learn.microsoft.com/en-us/azure/sentinel/automation/play book-recommendations

  34. [42]

    Playbook: Ransomware investigate and contain,

    Splunk, “Playbook: Ransomware investigate and contain,” https://resear ch.splunk.com/playbooks/ransomware_investigate_and_contain/

  35. [43]

    Claude code docs,

    Anthropic, “Claude code docs,” https://code.claude.com/docs/en/overvi ew

  36. [44]

    Metagpt: Meta programming for a multi-agent collaborative framework,

    S. Hong, M. Zhuge, J. Chen, X. Zheng, Y . Cheng, J. Wang, C. Zhang, S. Yau, Z. Lin, L. Zhouet al., “Metagpt: Meta programming for a multi-agent collaborative framework,” inInternational Conference on Learning Representations, 2024

    2024

  37. [45]

    What is docker?

    “What is docker?” https://docs.docker.com/get-started/docker-overvie w/

  38. [46]

    Containerlab,

    “Containerlab,” https://containerlab.dev/

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.