Pith. sign in

REVIEW 5 major objections 5 minor 38 references

Flow-of-Action: SOP Enhanced LLM-Based Multi-Agent System for Root Cause Analysis

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Constraining ReAct-style LLM agents with standard operating procedures lifts root-cause analysis accuracy from 35.5% to 64.0%.

desk verdict The architecture is thoughtfully designed and the ablation is useful, but the headline accuracy claim is undermined by a plausible leakage path from the historical incident store into the test queries. read the letter →

arxiv 2502.08224 v1 pith:VMHZHMSW submitted 2025-02-12 cs.SE

Changhua Pei , Zexin Wang , Fengrui Liu , Zeyan Li , Yang Liu , Xiao He , Rong Kang , Tieying Zhang
show 4 more authors
Jianjun Chen Jianhui Li Gaogang Xie Dan Pei
This is my paper · ORCID
classification cs.SE
keywords RootCauseAnalysisMulti-AgentSystemLargeLanguageModelStandardOperatingProcedureLLMAgentMicroservicesIncidentDiagnosisActionSet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that giving a ReAct-style LLM agent a structured diagnostic procedure—encoded as Standard Operating Procedures (SOPs) that can be matched, automatically generated, and compiled into executable code—plus a small team of auxiliary agents (an objectivity agent, a judge agent, an action agent, and a code agent) raises root-cause analysis accuracy on a 90-incident microservice benchmark from 35.50% to 64.01%. The authors argue that the gain comes from curbing the LLM's hallucinated or irrelevant tool calls at key decision points and from replacing free-form thought-action loops with a thought-actionset-action paradigm that delays the final action choice. If correct, the result would mean that agentic RCA can be made reliable enough for practical use by encoding expert diagnostic knowledge as soft constraints rather than by fine-tuning the model.

What carries the argument

The central mechanism is the SOP flow: a structured sequence of tools (match_sop, generate_sop, generate_sop_code, run_sop, match_observation) that mirrors how SREs diagnose incidents. Around it the paper builds an action set—the MainAgent first asks ActionAgent and JudgeAgent for candidate actions, then deliberates before choosing—and a multi-agent division of labor in which ObAgent distills noisy observations, JudgeAgent decides when root cause is found, and CodeAgent turns SOP text into executable code so that atomic diagnostic steps run together. The SOP flow is presented to the LLM as prompt text (soft constraints), not a hard workflow, preserving flexibility.

What would settle it

Re-run the comparison with all baseline agents given identical multimodal tools and with the 90 test incidents provably excluded from the historical-incident knowledge base; if the accuracy gap between ReAct and Flow-of-Action shrinks below the paper's reported 28-point difference—or if the test incidents are found to overlap the retrieval pool—the central claim would be an artifact of evaluation asymmetry.

Watch

Extended reading notes

Core claim

Flow-of-Action is an SOP-centered multi-agent system whose main agent orchestrates diagnosis by following a soft 'SOP flow' prompt: first match the incident to a relevant SOP, generate a new one if none matches, convert the SOP into code and run it, then match the resulting observations against historical incidents to infer fault type and location. The paper's central empirical claim is that this constrained workflow outperforms the unconstrained ReAct agent, with a Root Cause Location Accuracy of 70.89 and Type Accuracy of 57.12 (average 64.01) versus ReAct's 47.67 and 23.33 (average 35.50), on 90 fault-injection incidents over GoogleOnlineBoutique. The authors further claim, via ablations, that every component—SOP knowledge, SOP flow, the action set, and each of the four auxiliary agents—contributes to the accuracy, and that the approach generalizes to unseen fault types through automatic SOP generation.

Load-bearing premise

The baseline ReAct, CoT, and Reflexion agents must have had the same tool access, preprocessing, and knowledge base as Flow-of-Action, and the 90 test incidents must not be present in the historical-incident pool that match_observation retrieves from; the paper does not state either condition explicitly.

Editorial extensions

If this is right

  • Flow-of-Action more than doubles the average accuracy attained by the ReAct baseline on the 90-incident benchmark (64.01% vs 35.50%).
  • Removing any single component—SOP knowledge, SOP flow, action set, or any auxiliary agent—degrades accuracy, confirming each module's role.
  • The tool that generates new SOPs for unmatched incidents means the system can adapt to novel fault types without manual SOP authoring.
  • Converting SOPs to executable code lets multi-step diagnostic checks run as one atomic operation, cutting token and resource use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable transfer: applying the same SOP-flow and action-set recipe to other agentic troubleshooting domains (e.g., network configuration errors or database incidents) should reproduce a similar accuracy lift over ReAct, provided a comparable set of standard procedures exists.
  • The historical-incident retrieval step (match_observation) is the likeliest source of evaluation leakage; an external benchmark should verify that test incidents are excluded from the retrieval pool, since the paper does not document this exclusion.
  • The action-set size experiment suggests an optimal middle ground between stochastic exploration and deterministic execution; measuring this trade-off more precisely on a larger sample would sharpen the design guidance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. Flow-of-Action is an LLM multi-agent system for root-cause analysis in microservices. The paper contributes three interlocking mechanisms: an SOP flow that retrieves, generates, and compiles standard diagnostic procedures into executable code; an action-set mechanism in which the main agent first proposes a set of candidate actions before selecting one; and auxiliary agents (ActionAgent, ObAgent, JudgeAgent, CodeAgent) that offload action proposing, observation interpretation, and stopping decisions. Evaluation is performed on a self-built benchmark of 90 ChaosMesh-injected incidents in GoogleOnlineBoutique. The reported headline result is that Flow-of-Action with GPT-4-Turbo reaches 64.01% average accuracy versus 35.50% for ReAct, with Table 4 attributing the gain to each component.

Significance. The direction is valuable: encoding SRE experience as SOPs and using a flow-plus-action-set to constrain an otherwise free-form ReAct agent is a plausible and well-motivated way to reduce hallucinated tool calls. The multi-agent architecture is clearly described, and the ablation covers the main modules rather than only the whole system. The manuscript is also honest about some limitations, such as the APL trade-off and the inability to fully separate SOP knowledge from SOP flow in one ablation. The central problem is not the design but the evidence: the reported 28-point gain is vulnerable to evaluation asymmetries that are not addressed in the current text. If the authors can clarify the benchmark construction, baseline tool access, and run-to-run variability, the result would be a useful contribution to the AIOps/LLM-agent literature.

major comments (5)
  1. [§2.1.2, §2.3.4, §3.1.1] The benchmark section never states that the 90 evaluation incidents are excluded from the historical-incident knowledge base described in §2.1.2. In §2.3.4, match_observation retrieves similar historical incidents and ObAgent uses their 'type' attribute to suggest fault types; if an evaluation incident is present in that store, the system can retrieve an entry whose type encodes the ground-truth answer. This would inflate both TA and LA and could explain a substantial part of the 35.50-to-64.01 gap. Please specify the provenance of the historical-incident store and confirm that the 90 test incidents are disjoint from it, or redesign the retrieval so that no ground-truth attribute is returned.
  2. [§3.1.2] The baseline descriptions are too coarse to support a fair comparison. The paper does not state which tools ReAct, CoT, and Reflexion could invoke, whether they had access to the multimodal collection/analysis tools of §2.2.1, or whether they received the same knowledge base and preprocessing. Flow-of-Action uses the full tool suite plus SOP flow and agents; if the baselines operated with fewer or weaker tools, the accuracy difference is a tool-access artifact rather than evidence for the SOP/action-set/multi-agent design. Please provide per-baseline tool lists, prompts, and context, and ideally run all baselines with the same non-SOP tool suite.
  3. [§3.3, Table 3] The action-set size of 5 was selected by validation on a subset of the same 90-incident dataset, and every number in Tables 3 and 4 is a single-run point estimate with no seeds, confidence intervals, or significance tests. Because LLM-agent trajectories are stochastic, the reported differences between configurations cannot be distinguished from noise. Please report multiple runs with variance, and use a development split that is disjoint from the final test set for hyperparameter choices.
  4. [§3.4, Table 4] The 'w/o SOP Knowledge' ablation removes both the SOP knowledge and the SOP flow, so it cannot attribute the observed drop to either component. The text acknowledges this equivalence but then uses the row to argue for the crucial role of SOP. Moreover, the row's average accuracy (15.39) is below the ReAct GPT-3.5 baseline (19.17) in Table 3, which is surprising if the condition 'essentially reverts to ReAct'; this discrepancy needs an explanation, and the ablation should be split into separate removals of SOP knowledge and of flow prompts.
  5. [§3.1.1] The entire evaluation rests on 90 incidents from one e-commerce demo system with 9 injected fault types. This is acceptable as a first demonstration, but the abstract's phrasing that Flow-of-Action 'meets the accuracy requirements for RCA in real-world systems' is stronger than the evidence supports. Please temper the claim or add a second workload or external dataset.
minor comments (5)
  1. [§3.1.2] The APL formula defines N as the number of samples completed within the maximum path length, so incomplete trajectories are silently excluded; please report the completion rate and give the maximum path length in the setup, since the value 20 first appears in §3.4.
  2. [§2.2.1] The inline function names (e.g., '𝑤ℎ𝑒𝑡ℎ𝑒𝑟 _𝑖𝑠_𝑎𝑏𝑛𝑜𝑟𝑚𝑎𝑙 _𝑚𝑒𝑡𝑟𝑖𝑐') contain odd whitespace and should be typeset in a monospace font.
  3. [Eq. (1)] The scoring of 'correctly identified' root-cause locations and types is not defined; in particular, it is unclear how partial matches and the cap of three predicted root causes are counted. Please give the exact scoring rule.
  4. [§3.2] The phrase 'surpasses the SOTA' should be replaced by 'surpasses the best baseline in our comparison', since no external state-of-the-art system is defined.
  5. [Table 3] The fixed 11.11 accuracy for K8SGPT and HolmesGPT is attributed to handling only one fault type; please make the mapping from the 9 fault types to the 11.11% value explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

Empirical evaluation paper with no derivation chain that reduces to its own inputs; no circularity identified.

full rationale

The paper's central claim is an empirical accuracy comparison (64.01% vs. 35.50%) between Flow-of-Action and ReAct on a fault-injection benchmark. No equation in the paper defines the reported accuracy in terms of the method's own parameters or the benchmark labels. Section 3.3 tunes the action-set size on a subset of the dataset, but this is a hyperparameter choice, not a prediction derived from the evaluation, and the paper reports the resulting performance rather than presenting the tuned value as an independent confirmation. The abstractions SOP, SOP flow, action set, and multi-agent design are presented as the method under test, not as outputs of the evaluation. The cited prior work [3, 36] supplies motivation and metric inspiration, not a uniqueness theorem or an ansatz that carries the load; the authors' self-citations are incidental. Potential concerns about test-incident leakage into the historical knowledge base or asymmetric tool access for baselines are validity threats to the empirical comparison, not circularity by construction, because no stated equation or citation forces the result to equal its inputs. Under the hard rule requiring a quotable reduction, no such reduction exists.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The method's only quantitative support is the 90-incident self-built benchmark, so the ledger is dominated by evaluation-side assumptions: representativeness of the fault-injection platform, stability of single-run LLM evaluations, disjointness of test incidents from the historical-incident pool, accuracy of LLM-generated SOPs, and the validity of the mABC-derived metric. The one visible fitted constant is the action-set size (5); several others (top-k, similarity threshold, sigma) are either inherited from [36] or unreported.

free parameters (5)
  • action_set_size = 5
    Chosen in RQ2 (Figure 7) by sweeping sizes on a subset of the same 90-incident benchmark; the headline 64.01% result uses this tuned value (section 3.3).
  • sigma (penalty in LA/TA metric) = 0.1
    Penalty term in Eq. (1), adopted from mABC [36]; a free constant that shapes all accuracy numbers, applied uniformly to every method.
  • max_root_causes = 3
    Predictions capped at three root causes for all LLM methods (section 3.1.2); hand-chosen constraint that interacts with the LA/TA formulas.
  • max_path_length (APL cap) = 20
    RCA steps capped at 20 to avoid unbounded loops (section 3.4); affects APL and could cut off longer diagnoses.
  • match_sop top-k and filtering threshold = not specified
    Retrieval returns top-k SOPs above a similarity threshold (section 2.3.1); neither k nor the threshold is reported, so a free parameter is unobservable.
assumptions (5)
  • domain assumption The 90 fault-injection incidents on GoogleOnlineBoutique with nine ChaosMesh anomaly types are representative of real-world microservice incidents
    The entire evaluation rests on this simulation platform (section 3.1.1); the paper gives no evidence of representativeness beyond the 90 cases.
  • domain assumption Single-run LLM evaluations are meaningful despite acknowledged stochasticity
    Table 3 reports one run per configuration with no seeds or intervals, while section 1 names LLM randomness as a core challenge.
  • domain assumption Test incidents are excluded from the historical-incident knowledge base retrieved by match_observation
    Section 2.1.2 adds historical incidents to the knowledge base and section 3.1.1 creates the 90-incident dataset; the paper never states these pools are disjoint.
  • domain assumption LLM-generated SOPs are accurate enough to guide RCA
    The system depends on generated SOPs when no match exists; section 2.3.1 concedes the IO-error example is 'not highly precise', yet no quantitative quality check is applied.
  • domain assumption The mABC metric (Eq. 1) with sigma=0.1 and a 3-cause cap measures RCA quality faithfully
    Adopted from [36] without validation; the formula rewards conservative, low-prediction behavior, which may favor the proposed method's design.
invented entities (2)
  • JudgeAgent independent evidence
    purpose: Determines whether the root cause has been found and adds Speak to the action set otherwise
    Its contribution is supported only by the internal ablation (Table 4: w/o JudgeAgent drops average accuracy to 35.00); there is no external evidence about how well the stopping criterion generalizes.
  • ObAgent independent evidence
    purpose: Extracts fault types and key information from noisy multimodal data after match_observation
    Ablation shows its removal drops accuracy to 34.39 (Table 4), an internal falsifiable handle; external generalization to non-synthetic logs is untested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Flow-of-Action: SOP Enhanced LLM-Based Multi-Agent System for Root Cause Analysis." pith.science (2026). https://pith.science/paper/VMHZHMSW

@misc{pith2026250208224,
  author       = {Pith},
  title        = {Pith review of: Flow-of-Action: SOP Enhanced LLM-Based Multi-Agent System for Root Cause Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VMHZHMSW}},
  note         = {Machine review of arXiv:2502.08224}
}
read the original abstract

In the realm of microservices architecture, the occurrence of frequent incidents necessitates the employment of Root Cause Analysis (RCA) for swift issue resolution. It is common that a serious incident can take several domain experts hours to identify the root cause. Consequently, a contemporary trend involves harnessing Large Language Models (LLMs) as automated agents for RCA. Though the recent ReAct framework aligns well with the Site Reliability Engineers (SREs) for its thought-action-observation paradigm, its hallucinations often lead to irrelevant actions and directly affect subsequent results. Additionally, the complex and variable clues of the incident can overwhelm the model one step further. To confront these challenges, we propose Flow-of-Action, a pioneering Standard Operation Procedure (SOP) enhanced LLM-based multi-agent system. By explicitly summarizing the diagnosis steps of SREs, SOP imposes constraints on LLMs at crucial junctures, guiding the RCA process towards the correct trajectory. To facilitate the rational and effective utilization of SOPs, we design an SOP-centric framework called SOP flow. SOP flow contains a series of tools, including one for finding relevant SOPs for incidents, another for automatically generating SOPs for incidents without relevant ones, and a tool for converting SOPs into code. This significantly alleviates the hallucination issues of ReAct in RCA tasks. We also design multiple auxiliary agents to assist the main agent by removing useless noise, narrowing the search space, and informing the main agent whether the RCA procedure can stop. Compared to the ReAct method's 35.50% accuracy, our Flow-of-Action method achieves 64.01%, meeting the accuracy requirements for RCA in real-world systems.

Figures

Figures reproduced from arXiv: 2502.08224 by the authors.

Figure 1
Figure 1. Illustration example of challenge 2. deployment processes and limited scalability [1, 8, 24, 26, 37], attrib￾uted to the proliferation of services and frequent service iterations. In response to this context, Microservices Architecture (MSA) has surfaced and continually evolved [2]. By disassembling monolithic applications into small, self-sufficient service units, each dedicated to specific business functionalities… view at source ↗
Figure 2
Figure 2. Multimodal data collection and analysis. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of ReAct and Flow-of-Action. RC means root cause. Dashed lines represent paths triggered under specific [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Example of Flow-of-Action. relevant to the incident (corresponding to𝑚𝑎𝑡𝑐ℎ_𝑠𝑜𝑝). Additionally, if a particular incident does not align with any existing SOP, the au￾tomation of SOP generation should be considered (corresponding to 𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑒_𝑠𝑜𝑝). While the comprehensive…
Figure 6
Figure 6. Figure 6: Generated SOP for IO error. Within the entirety of the flow, the generation of SOPs stands as a pivotal phase as it directly influences the subsequent RCA process. To enhance the precision of RCA, we have devised hierarchical SOPs. Our objective is for the RCA process …
Figure 5
Figure 5. Figure 5: SOP flow prompt information of LLMs. 2.3.1 Fault Type/Information→SOP. In our flow, we initially utilize 𝑚𝑎𝑡𝑐ℎ_𝑠𝑜𝑝 to associate the fault information with the relevant SOP. This matching process involves computing the similarity between the current query and all SOP na…
Figure 7
Figure 7. Figure 7: Accuracy of different action set sizes. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 11
Figure 11. Figure 11: Elastic Dashboard. B Tools [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 8
Figure 8. Figure 8: Prometheus Dashboard [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Deepflow Dashboard [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Jaeger Dashboard [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 23 canonical work pages

  1. [1]

    Sarthak Chakraborty, Shaddy Garg, Shubham Agarwal, Ayush Chauhan, and Shiv Kumar Saini. 2023. Causil: Causal graph for instance level microservice data. In Proceedings of the ACM Web Conference 2023 . 2905–2915

    work page 2023

  2. [2]

    Hongyang Chen, Pengfei Chen, Guangba Yu, Xiaoyun Li, Zilong He, and Huxing Zhang. 2024. MicroFI: Non-Intrusive and Prioritized Request-Level Fault Injection for Microservice Applications. IEEE Transactions on Dependable and Secure Computing (2024)

    work page 2024

  3. [3]

    Yinfang Chen, Huaibing Xie, Minghua Ma, Yu Kang, Xin Gao, Liu Shi, Yunjie Cao, Xuedong Gao, Hao Fan, Ming Wen, et al. 2024. Automatic root cause analysis via large language models for cloud incidents. In Proceedings of the Nineteenth European Conference on Computer Systems . 674–688

    work page 2024

  4. [4]

    Ruomeng Ding, Chaoyun Zhang, Lu Wang, Yong Xu, Minghua Ma, Xiaomin Wu, Meng Zhang, Qingjun Chen, Xin Gao, Xuedong Gao, et al . 2023. Trace- Diag: Adaptive, Interpretable, and Efficient Root Cause Analysis on Large-Scale Microservice Systems. In Proceedings of the 31st ACM Joint European Software En- gineering Conference and Symposium on the Foundations of...

    work page 2023

  5. [5]

    Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al . 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352 (2023)

    arXiv 2023

  6. [6]

    Azam Ikram, Sarthak Chakraborty, Subrata Mitra, Shiv Saini, Saurabh Bagchi, and Murat Kocaoglu. 2022. Root cause analysis of failures in microservices through causal discovery. Advances in Neural Information Processing Systems 35 (2022), 31158–31170

    work page 2022

  7. [7]

    Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong C Park

  8. [8]

    Xinrui Jiang, Yicheng Pan, Meng Ma, and Ping Wang. 2023. Look Deep into the Microservice System Anomaly through Very Sparse Logs. In Proceedings of the ACM Web Conference 2023. 2970–2978

    work page 2023

Show all 38 references
  1. [9]

    k8sgpt ai. 2023. k8sgpt. https://github.com/k8sgpt-ai/k8sgpt

    2023

  2. [10]

    Labring. 2023. FastGPT. https://github.com/labring/FastGPT

    2023

  3. [11]

    Chengshu Li, Jacky Liang, Andy Zeng, Xinyun Chen, Karol Hausman, Dorsa Sadigh, Sergey Levine, Li Fei-Fei, Fei Xia, and Brian Ichter. 2023. Chain of code: Reasoning with a language model-augmented code emulator. arXiv preprint arXiv:2312.04474 (2023)

    2023 arXiv

  4. [12]

    Zeyan Li, Nengwen Zhao, Mingjie Li, Xianglin Lu, Lixin Wang, Dongdong Chang, Xiaohui Nie, Li Cao, Wenchi Zhang, Kaixin Sui, et al . 2022. Actionable and interpretable fault localization for recurring failures in online service systems. In Proceedings of the 30th ACM Joint Euro...

    2022

  5. [13]

    Cheng-Ming Lin, Ching Chang, Wei-Yao Wang, Kuang-Da Wang, and Wen-Chih Peng. 2024. Root Cause Analysis in Microservice Using Neural Granger Causal Discovery. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 206–213

    2024

  6. [14]

    Panagiotis Misiakos, Chris Wendler, and Markus Püschel. 2024. Learning DAGs from data with few root causes.Advances in Neural Information Processing Systems 36 (2024)

    2024

  7. [15]

    Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. Logic- lm: Empowering large language models with symbolic solvers for faithful logical reasoning. arXiv preprint arXiv:2305.12295 (2023)

    2023 arXiv

  8. [16]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789 (2023)

    2023 arXiv

  9. [17]

    Alec Radford. 2018. Improving language understanding by generative pre- training. (2018)

    2018

  10. [18]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9

    2019

  11. [19]

    robusta dev. 2024. holmesgpt. https://github.com/robusta-dev/holmesgpt

    2024

  12. [20]

    Carl Martin Rosenberg and Leon Moonen. 2020. Spectrum-based log diagnosis. In Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) . 1–12

    2020

  13. [21]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2024. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems 36 (2024)

    2024

  14. [22]

    Junxian Shen, Han Zhang, Yang Xiang, Xingang Shi, Xinrui Li, Yunxi Shen, Zijian Zhang, Yongxiang Wu, Xia Yin, Jilong Wang, et al. 2023. Network-centric distributed tracing with deepflow: Troubleshooting your microservices in zero code. In Proceedings of the ACM SIGCOMM 2023 Co...

    2023

  15. [23]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36 (2024)

    2024

  16. [24]

    Gagan Somashekar, Anurag Dutt, Mainak Adak, Tania Lorido Botran, and An- shul Gandhi. 2024. GAMMA: Graph Neural Network-Based Multi-Bottleneck Localization for Microservices Applications. In Proceedings of the ACM on Web Conference 2024. 3085–3095

    2024

  17. [25]

    Shreshth Tuli, Giuliano Casale, and Nicholas R Jennings. 2022. Tranad: Deep transformer networks for anomaly detection in multivariate time series data. arXiv preprint arXiv:2201.07284 (2022)

    2022 arXiv

  18. [26]

    Lu Wang, Chaoyun Zhang, Ruomeng Ding, Yong Xu, Qihang Chen, Wentao Zou, Qingjun Chen, Meng Zhang, Xuedong Gao, Hao Fan, et al . 2023. Root cause analysis for microservice systems via hierarchical reinforcement learning from human feedback. In Proceedings of the 29th ACM SIGKDD...

    2023

  19. [27]

    Zexin Wang, Jianhui Li, Minghua Ma, Ze Li, Yu Kang, Chaoyun Zhang, Chetan Bansal, Murali Chintalapati, Saravan Rajmohan, Qingwei Lin, et al. 2024. Large Language Models Can Provide Accurate and Interpretable Incident Triage. In2024 IEEE 35th International Symposium on Software...

    2024

  20. [28]

    Zexin Wang, Changhua Pei, Minghua Ma, Xin Wang, Zhihan Li, Dan Pei, Saravan Rajmohan, Dongmei Zhang, Qingwei Lin, Haiming Zhang, et al. 2024. Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective. In Proceedings of the ACM on Web Conference 202...

    2024

  21. [29]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  22. [30]

    Li Wu, Johan Tordsson, Erik Elmroth, and Odej Kao. 2020. Microrca: Root cause localization of performance issues in microservices. In NOMS 2020-2020 IEEE/IFIP Network Operations and Management Symposium . IEEE, 1–9

    2020

  23. [31]

    Haowen Xu, Wenxiao Chen, Nengwen Zhao, Zeyan Li, Jiahao Bu, Zhihan Li, Ying Liu, Youjian Zhao, Dan Pei, Yang Feng, et al. 2018. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. In Proceedings of the 2018 world wide web conferen...

    2018

  24. [32]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022)

    2022 arXiv

  25. [33]

    Zhenhe Yao, Changhua Pei, Wenxiao Chen, Hanzhang Wang, Liangfei Su, Huai Jiang, Zhe Xie, Xiaohui Nie, and Dan Pei. 2024. Chain-of-Event: Interpretable Root Cause Analysis for Microservices through Automatically Learning Weighted Event Causal Graph. In Companion Proceedings of ...

    2024

  26. [34]

    Zhenhe Yao, Haowei Ye, Changhua Pei, Guang Cheng, Guangpei Wang, Zhiwei Liu, Hongwei Chen, Hang Cui, Zeyan Li, Jianhui Li, et al . 2024. SparseRCA: Unsupervised Root Cause Analysis in Sparse Microservice Testing Traces. In2024 IEEE 35st International Symposium on Software Reli...

    2024

  27. [35]

    Guangba Yu, Pengfei Chen, Yufeng Li, Hongyang Chen, Xiaoyun Li, and Zibin Zheng. 2023. Nezha: Interpretable fine-grained root causes analysis for microser- vices on multi-modal observability data. In Proceedings of the 31st ACM Joint European Software Engineering Conference an...

    2023

  28. [36]

    Wei Zhang, Hongcheng Guo, Jian Yang, Yi Zhang, Chaoran Yan, Zhoujin Tian, Hangyuan Ji, Zhoujun Li, Tongliang Li, Tieqiao Zheng, et al. 2024. mABC: multi- Agent Blockchain-Inspired Collaboration for root cause analysis in micro-services architecture. arXiv preprint arXiv:2404.1...

    2024 arXiv

  29. [37]

    Lecheng Zheng, Zhengzhang Chen, Jingrui He, and Haifeng Chen. 2024. MULAN: Multi-modal Causal Structure Learning and Root Cause Analysis for Microservice Systems. In Proceedings of the ACM on Web Conference 2024 . 4107–4116. A Multimodal Data Collection We first deploy various...

    2024

  30. [2024]

    arXiv preprint arXiv:2403.14403 (2024)

    Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. arXiv preprint arXiv:2403.14403 (2024)

    2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.