Pith. sign in

REVIEW 5 major objections 5 minor 62 references

Root-cause analysis at 200,000-service scale is tractable when a causal skeleton guides collaborating LLM agents.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A deployed RCA system using API-level drilldown, a skeleton causal graph, and memory-augmented multi-agent LLM reasoning localizes root-cause services and failure types with AC@1 of 0.88/0.79 in a 200k-service production environment.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A genuinely deployed, well-ablated industrial RCA system whose downstream-only drilldown caps its generalizability; worth refereeing, but the authors need to own and document that cap. the 5 major comments →

arxiv 2607.01788 v3 pith:LYPWAJOZ submitted 2026-07-02 cs.SE

KRCA: An Efficient Root Cause Analysis System in Hyper-scale Microservice Systems via Agentic AI

Jiamin Jiang , Jingfei Feng , Yu Luo , Qingliang Zhang , Yongqian Sun , Wenwei Gu , Shenglin Zhang , Tianyu Cui
show 4 more authors
Yao Wu Jielong Huang Nan Qi Dan Pei
This is my paper
classification cs.SE
keywords root cause analysismicroserviceslarge language modelsmulti-agent systemscausal graphincident diagnosissite reliability engineeringproduction deployment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that root-cause analysis in a system of hundreds of thousands of microservices is solvable in real time if the search space is cut down before any large language model (LLM) runs. Its pipeline first prunes the dependency graph using API-level failure-rate and latency signals, then maps each suspicious service's anomalous metrics onto a generic causal skeleton with four meta-metric types, and finally lets a memory-augmented multi-agent LLM team verify and refine the resulting graph. On 300 real production failures the authors report top-1 accuracy of 0.88 for locating the root-cause service and 0.79 for classifying the failure type, with a production deployment that reduced average diagnosis time from 52 to 11.8 minutes. The point that matters is that structured causal priors plus targeted LLM reasoning can replace both retrained deep models and unconstrained agent loops, which do not scale.

Core claim

The central claim is that the bottleneck in hyper-scale RCA is not LLM reasoning quality but uncontrolled search space and context length. KRCA shows that when anomalous metrics are first organized by a generic causal skeleton—External, Internal, Dependency, and KPI, with fixed propagation directions and KPI treated only as an effect—LLM agents can verify local causal links and discover finer-grained relations instead of hallucinating over a large raw metric set. On 300 production failures the system achieves top-1 accuracy of 0.88 for root-cause service localization and 0.79 for failure-type classification, gains of at least 31 absolute points over the strongest baseline, and in six months

What carries the argument

The load-bearing object is the service-local causal skeleton graph G_s, a generic prior with four meta-metric types—External (traffic, upstream anomalies), Internal (CPU, GPU, memory), Dependency (Kafka, MySQL, Redis), and KPI (latency, success rate)—connected by fixed directions (E→I, E→D, I→K, D→K). Anomalous metrics detected by a hybrid detector are mapped onto this skeleton to instantiate the initial graph G_o, constraining the causal search space before any LLM reasoning. The second mechanism is API-level drilldown, a recursive scoring of downstream APIs using time-lagged failure-rate correlation and a composite latency score, which shrinks the blast radius from thousands of services to

Load-bearing premise

The weakest link is the assumption that every real failure inside a service propagates along the fixed skeleton directions—external and internal issues flow into KPIs, dependencies sit in between—so that KPI changes are always effects and never causes; if a genuine propagation path runs backward, the initial graph cannot represent it and the LLM refinement starts from the wrong structure.

What would settle it

Collect a set of incidents whose ground truth includes a feedback mechanism the skeleton forbids—for example, an availability drop that triggers retries and overloads a downstream database (KPI→Dependency), or a dependency slowdown that drives up internal CPU usage (Dependency→Internal). If KRCA's top-1 accuracy on those cases is near chance or far below its 0.88 average, the fixed-direction skeleton is the limiting factor.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • RCA in hyper-scale systems would no longer depend on periodically retraining deep models; a static skeleton plus in-context LLM reasoning can track a constantly changing topology.
  • Starting from API-level signals rather than service-level aggregates should expose failures hidden when high-traffic APIs mask degraded low-traffic APIs.
  • Organizing anomalous metrics into a structured prior before LLM reasoning should keep accuracy from collapsing as the number of abnormal metrics grows.
  • The staged pipeline can fit a real-time budget: the reported end-to-end latency of about 2.5 minutes per case is compatible with live incident response.
  • A memory-and-retrieval design should let the system improve with experience, reusing service-specific incidents and transferable diagnostic insights.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the system's ceiling is set by the completeness of the fixed skeleton. Failure mechanisms with feedback—such as KPI degradation triggering retries that overload a dependency, or dependency slowdown heating up internal CPU—cannot be represented by G_s and would bias the initial graph despite the later LLM refinement.
  • Editorial inference: if the skeleton generalizes across companies, it could be reused as a standard metric taxonomy for other observability stacks, but its fixed directions likely need extension for stateful or event-driven services where propagation paths are not strictly E→I/D→K.
  • Editorial inference: the tiered memory design implies testable learning behavior—accuracy should improve as factual and experiential memory accumulate, and experiential memory alone should transfer diagnoses to unseen services; before/after measurements of memory accumulation would confirm this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents KRCA, a multi-stage root cause analysis system for hyper-scale microservice systems. The pipeline first performs an API-level drilldown along dependency edges to isolate suspicious services, then instantiates a service-level causal graph G_o by mapping anomalous metrics onto a fixed skeleton G_s with four meta-metric types (External, Internal, Dependency, KPI), and finally uses a memory-augmented multi-agent framework to refine the graph and produce a failure report. The system is evaluated on 300 real-world failures from Kuaishou, with reported AC@1 of 0.88 for root-cause service localization and 0.79 for failure type classification, outperforming the strongest baseline (RCA-Agent) by absolute gains of 31% and 32%. The authors also report a six-month production deployment that reduces average root-cause localization time by 77.3%.

Significance. If the claims hold, this is a meaningful systems contribution: KRCA is one of the first end-to-end RCA frameworks designed for the scale of hundreds of thousands of microservices, and it experimentally demonstrates that a progressive search-space reduction can make LLM-based RCA practical in production. The paper includes a clear ablation study, a hyperparameter sensitivity analysis, and deployment statistics, which are strengths. However, the evaluation is based on a single company's non-public dataset, and several structural design choices—especially the downstream-only drilldown and the hand-authored causal skeleton—limit the generality of the reported accuracy. The claims are plausible but need additional evidence to rule out systematic biases in the evaluation.

major comments (5)
  1. [Section 3.2, Eq. (1)–(5)] The drilldown module only scores downstream APIs. Score(P,C) is defined between an upstream parent P and a downstream child C, and the traversal 'recursively traverses ... downstream APIs'. There is no scoring function for the alerting API itself or for its upstream callers. Consequently, if the true root cause is the alerting service itself (e.g., a configuration change or code defect) or an upstream service (e.g., a traffic surge, which the paper lists as External E in §3.3), that service cannot enter the top-N suspicious set, and the later skeleton-graph and multi-agent stages cannot recover it. The motivation in Fig. 2(a) and the deployment failure analysis in §5.1 discuss only downstream propagation and missing downstream observability. The paper should report the fraction of the 300 failures for which the root cause is actually reachable by downstream traversal, add an evaluation v
  2. [Section 3.3, Fig. 5(a)] The generic skeleton G_s is a fixed structural prior with only the edge directions E→I/K/D, I→K/D, D→K, and with KPI always an effect. Real failure propagation can violate these directions: for example, KPI degradation can trigger retries that overload a dependency, or dependency anomalies can feed back into internal metrics. In such cases G_o cannot represent the true causal path, and the multi-agent refinement is initialized from a biased graph. The paper calls G_s a 'high-recall structural prior' (Design Insight 2) but never measures its recall against ground-truth causal paths. Please provide a recall/coverage analysis of G_s, or at least a sensitivity study on cases with feedback loops, to substantiate the claim that the prior does not cap accuracy.
  3. [Section 4.1] The description of memory configuration and temporal separation is ambiguous and affects the validity of the evaluation. The text says 'All system parameters and the tiered memory are configured with routine daily alerts after removing failure-related alerts', while §3.4 states that Factual Memory stores service-specific historical failures. If all failure-related alerts were removed from memory, Factual Memory would be empty, and the RAG mechanism would not be exercised as designed. If only the 300 evaluated incidents were removed, the memory is populated from the same six-month window (October 2025 to March 2026) as the test set, creating a potential temporal leakage in which near-duplicate failure knowledge is available. Clarify exactly which alerts populate each memory tier and whether the test cases are temporally separated from memory data. This directly affects the interpretation
  4. [Section 3.4 and Section 4.2] The failure type classification evaluation appears to be circular to a degree: the nine domain-specific Sub Agents (e.g., Traffic Expert, CPU Expert, GPU Expert) are aligned with the nine classical failure types used for ground-truth labeling, and the E/I/D/K skeleton organizes the metric space in the same taxonomy. This does not invalidate the approach, but it means that high AC@1 for failure type classification may partly reflect the fact that the system's hypothesis space is exactly the evaluation ontology. The paper should report a per-failure-type confusion matrix and include out-of-taxonomy or mixed-type failures to demonstrate generalization. Without this, the AC@1=0.79 claim is not yet fully supported.
  5. [Section 5.1] The 77.3% reduction in average root-cause localization time is based on a before/after comparison of manual diagnosis in different time periods and different incident populations (2024–Oct 2025 vs. Oct 2025–Mar 2026). This is an uncontrolled operational comparison; incident complexity, SRE familiarity, and other concurrent tooling changes could confound the result. Additionally, the paper does not state the relationship between the 483 deployment incidents and the 300-case evaluation dataset (e.g., overlap). Please clarify this relationship and provide a more careful comparison, for example by reporting case-complexity distributions or a matched-pair analysis.
minor comments (5)
  1. [Abstract] Typo: 'at lease 31%' should be 'at least 31%'. Also the sentence 'KRCA employs balances diagnostic accuracy' in the abstract is ungrammatical and should be rewritten.
  2. [Section 4.1] The evaluation reports only point estimates. Given 300 failures, confidence intervals or a significance test would be helpful to assess the stability of the reported AC@1 differences, especially against RCA-Agent.
  3. [Section 2.2, Fig. 2(b)] The empirical study for Fig. 2(b) uses only 20 incidents; the paper should state whether these incidents are included in the 300-case evaluation dataset, since overlap could affect the motivation analysis.
  4. [Data Availability Statement] The statement says source code will be released 'after desensitization and approval' but no timeline is provided. If code is available, a specific link or version identifier would help reproducibility.
  5. [Section 4.2] The baseline comparison excludes deep learning RCA methods with the justification that training is too expensive. That is defensible, but the paper should explicitly acknowledge that the comparison is limited to LLM-based and agent-based baselines, so the 'state-of-the-art' phrasing should be qualified.

Circularity Check

0 steps flagged

No circular derivation found: hand-authored priors and evaluation-validity risks do not make any prediction equivalent to its inputs by construction.

full rationale

KRCA is an engineering pipeline whose components are proposed, not derived from the evaluation labels, and whose claimed results are supported by held-out data and ablations. The drilldown scores (Eqs. 1-5) operate on measured failure-rate and latency time series; the skeleton graph G_s (Fig. 5a) is a hand-authored structural prior whose contribution is tested in Table 2 (w/o Skeleton Graph drops AC@1 to 0.72); and the RAG retrieval (Eq. 6) uses similarity over service and metric names, not ground-truth outcomes. The closest concerns are evaluation-validity risks rather than circular reductions: the downstream-only drilldown (§3.2) cannot propose upstream or self root causes; the nine Sub Agents (§3.4) align with the nine failure-type labels in the dataset (§4.1); and RAG memory shares the six-month production window with the evaluation (§4.1). These limit generality but do not make any reported prediction equivalent by construction to an input, especially since the paper states that evaluated incidents were excluded from parameter and memory tuning. Author-overlap citations (e.g., CIRCA [16] as inspiration for G_s, [58] for the report-scoring rubric) are not load-bearing. No equation defines its output in terms of the target label, so no significant circularity is established.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 2 invented entities

The central claims rest on several hand-authored priors: the E/I/D/K causal skeleton, the drilldown scoring weights, the memory retrieval weights, and the failure-type ontology. These are not derived from data; they are engineering choices tuned on routine alerts and expert knowledge. The evaluation additionally assumes that SRE-labeled ground truth and the sampled failures are trustworthy and representative.

free parameters (6)
  • drilldown threshold = 0.8
    Propagation threshold in Eq. 1; chosen in implementation, sensitivity range 0.79–0.88 AC@1.
  • maximum time lag L = 5
    Lag bound for time-lagged Pearson correlation in Eq. 2; sensitivity range 0.82–0.88.
  • latency score weights (w1,w2,w3) = (0.2, 0.5, 0.3)
    Balancing weights in Eq. 3; sensitivity range 0.70–0.88 AC@1.
  • Top-N suspicious services = 3
    Number of services passed to later modules; N=5 gives 0.90 but higher latency (191.2s vs 146.6s).
  • memory retrieval weights (alpha,beta,gamma) = Factual (0.5,0.2,0.3), Experiential (0.4,0.5,0.1)
    Weights in Eq. 6; sensitivity range 0.77–0.88 AC@1.
  • p-value significance level = 0.05
    Alpha for filtering spurious correlations in Eq. 2 (stated in Section 3.2).
axioms (6)
  • domain assumption The E/I/D/K causal skeleton with fixed edge directions (E→I→K, E→D→K, I→D→K) is a high-recall prior for service-internal failure propagation.
    Introduced in Section 3.3 (Fig. 5a); the entire causal graph instantiation and downstream LLM refinement depend on it.
  • domain assumption API-level failure rate and latency metrics are sufficient and reliable signals for drilldown; correlation (with p-value filtering) indicates causal propagation.
    Section 3.2, Eqs. 1-2; the drilldown cannot find root causes that do not manifest as correlated failure rate/latency changes.
  • domain assumption LLM agents with structured constraints and RAG can verify causality accurately enough for production RCA.
    Section 2.2 (Fig. 2b) and Section 3.4; 20-incident study is the empirical basis.
  • domain assumption The ground-truth labels (root cause service, failure type) assigned by five SREs are correct.
    Section 4.1; no inter-annotator agreement is reported.
  • domain assumption The 300 sampled failures are representative of hyper-scale microservice failures and are not contaminated by the memory system.
    Section 4.1; the evaluation and deployment share the same six-month window (Oct 2025–Mar 2026).
  • domain assumption The nine failure types used for classification form a complete ontology for hyper-scale system failures.
    Section 4.1; the skeleton and Sub Agents are aligned to these types, so out-of-ontology failures cannot be classified.
invented entities (2)
  • E/I/D/K meta-metric taxonomy no independent evidence
    purpose: Maps the hundreds of metrics per service onto four abstract types so a generic causal skeleton can be instantiated
    Section 3.3, Fig. 5; a modeling construct with no falsifiable handle outside the paper; it constrains the causal search space by design.
  • Three-tier memory (Working, Factual, Experiential) no independent evidence
    purpose: Provides short-term context, service-specific historical cases, and transferable insights to the LLM agents via RAG
    Section 3.4, Fig. 6; software architecture component, not independently observable.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of KRCA: An Efficient Root Cause Analysis System in Hyper-scale Microservice Systems via Agentic AI." pith.science (2026). https://pith.science/paper/LYPWAJOZ

@misc{pith2026260701788,
  author       = {Pith},
  title        = {Pith review of: KRCA: An Efficient Root Cause Analysis System in Hyper-scale Microservice Systems via Agentic AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LYPWAJOZ}},
  note         = {Machine review of arXiv:2607.01788}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Hyper-scale microservice systems have become the standard infrastructure for large-scale Internet companies. These systems consist of numerous loosely coupled microservices that evolve independently through continuous development and deployment. Such complexity makes failures unavoidable, necessitating efficient Root Cause Analysis (RCA) to help Site Reliability Engineers (SREs) quickly localize root cause services and classify failure types. However, existing RCA methods often struggle to adapt to the extreme dynamism and massive scale of these systems. In this paper, we present KRCA, an end-to-end RCA system designed for hyper-scale microservice systems. To manage the vast search space, KRCA employs a multi-stage pipeline that begins with an API-level drilldown to isolate suspicious services. It then instantiates a skeleton-based causal graph from anomalous metrics to serve as a high-recall structural prior, before utilizing a memory-augmented multi-agent framework to verify causality and generate the final failure report. By combining structured causal constraints with multi-agent reasoning, KRCA employs balances diagnostic accuracy with the efficiency requirements of real-time production use. Experimental results show that KRCA achieves AC@1 scores of 0.88 and 0.79 for root cause service localization and failure type classification, outperforming the strongest baseline by at lease 31% in absolute gains. KRCA has been deployed in Kuaishou's production environment for over six months, reducing the average diagnosis time by 77.3%.

Figures

Figures reproduced from arXiv: 2607.01788 by Dan Pei, Jiamin Jiang, Jielong Huang, Jingfei Feng, Nan Qi, Qingliang Zhang, Shenglin Zhang, Tianyu Cui, Wenwei Gu, Yao Wu, Yongqian Sun, Yu Luo.

Figure 1
Figure 1. Figure 1: Two primary characteristics of hyper-scale mi [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: Two primary characteristics of hyper-scale mi [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Empirical study on the limitations of existing RCA [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of the API-level drilldown process. Starting from the alerting API, KRCA recursively eval￾uates and prunes downstream APIs based on a scoring function. supplies similar historical cases and diagnostic experience. After several rounds of refinement, the final causal graph is used to gener￾ate a failure report that identifies the root cause service and failure type. 3.2 API-level drilldown In ou… view at source ↗
Figure 5
Figure 5. Figure 5: Skeleton-based causal graph instantiation. (a) The [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Overview of the Multi-Agent Collaboration framework. The Main Agent orchestrates domain-specific Sub Agents [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Sensitivity analysis of key hyperparameters in [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 7
Figure 7. Figure 7: Sensitivity analysis of key hyperparameters in [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Deployment architecture of KRCA. E-commerce, and Algorithms. During this period, we collected in￾ternal statistics on 483 emergency incidents3 . For each incident, the ground-truth root cause service and failure type were estab￾lished through postmortem analysis by the SREs responsible for the affected services. According to these records, KRCA correctly identified both the root cause service and the failu… view at source ↗
Figure 8
Figure 8. Figure 8: Deployment architecture of KRCA. to produce a correct diagnosis. These failures mainly fall into two categories. Firstly, some key downstream services lacked critical observability signals, such as availability-related metrics, which caused the drilldown process to terminate too early. Secondly, mul￾tiple cascading anomalies occurred simultaneously, causing several services to fail at once and making it di… view at source ↗

Review history (3 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 2 linked inside Pith

  1. [1]

    Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009. Pearson correlation coefficient. InNoise reduction in speech processing. Springer, 1–4

    2009

  2. [2]

    O’Reilly Media, Inc

    Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy. 2016.Site reliability engineering: how Google runs production systems. " O’Reilly Media, Inc. "

    2016

  3. [3]

    2011.Bayesian inference in statistical analysis

    George EP Box and George C Tiao. 2011.Bayesian inference in statistical analysis. John Wiley & Sons

    2011

  4. [4]

    Pengfei Chen, Yong Qi, Pengfei Zheng, and Di Hou. 2014. Causeinfer: Auto- matic and distributed performance diagnosis with hierarchical causality graph in large distributed systems. InIEEE INFOCOM 2014-IEEE Conference on Computer Communications. IEEE, 1887–1895

    2014

  5. [5]

    Yinfang Chen, Huaibing Xie, Minghua Ma, Yu Kang, Xin Gao, Liu Shi, Yunjie Cao, Xuedong Gao, Hao Fan, Ming Wen, et al. 2024. Automatic root cause analysis via large language models for cloud incidents. InProceedings of the Nineteenth European Conference on Computer Systems. 674–688

    2024

  6. [6]

    Yuxiao Cheng, Lianglong Li, Tingxiong Xiao, Zongren Li, Jinli Suo, Kunlun He, and Qionghai Dai. 2024. Cuts+: High-dimensional causal discovery from irregular time-series. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 11525–11533

    2024

  7. [7]

    Tianyu Cui, Shiyu Ma, Ziang Chen, Tong Xiao, Chenyu Zhao, Shimin Tao, Yilun Liu, Shenglin Zhang, Duoming Lin, Changchang Liu, et al . 2025. LogEval: A comprehensive benchmark suite for LLMs in log analysis.Empirical Software Engineering30, 6 (2025), 173

    2025

  8. [8]

    Huaming Du, Yujia Zheng, Baoyu Jing, Yu Zhao, Gang Kou, Guisong Liu, Tao Gu, Weimin Li, and Carl Yang. 2025. Causal discovery through synergizing large language model and data-driven reasoning. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 543–554

    2025

  9. [9]

    Tao Feng, Lizhen Qu, Niket Tandon, Zhuang Li, Xiaoxi Kang, and Gholamreza Haffari. 2025. On the reliability of large language models for causal discovery. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 9565–9590

    2025

  10. [10]

    Ruowei Fu, Yang Zhang, Zeyu Che, Xin Wu, Zhenyu Zhong, Zhiqiang Ren, Shenglin Zhang, Feng Wang, Yongqian Sun, Xiaozhou Liu, et al . 2025. LLM- Powered Multi-Agent Collaboration for Intelligent Industrial On-Call Automa- tion. In2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2222–2234

    2025

  11. [11]

    Google Cloud Platform. 2021. Online Boutique: A cloud-native microservices demo application. https://github.com/GoogleCloudPlatform/microservices-demo. Accessed: 2026-03-23

    2021

  12. [12]

    Clive WJ Granger. 1980. Testing for causality: A personal viewpoint.Journal of Economic Dynamics and control2 (1980), 329–352

    1980

  13. [13]

    Yongqi Han, Qingfeng Du, Ying Huang, Jiaqi Wu, Fulong Tian, and Cheng He

  14. [14]

    Azam Ikram, Sarthak Chakraborty, Subrata Mitra, Shiv Saini, Saurabh Bagchi, and Murat Kocaoglu. 2022. Root cause analysis of failures in microservices through causal discovery.Advances in Neural Information Processing Systems35 (2022), 31158–31170

    2022

  15. [15]

    2010.Random walk: a modern introduction

    Gregory F Lawler and Vlada Limic. 2010.Random walk: a modern introduction. Vol. 123. Cambridge University Press

    2010

  16. [16]

    Mingjie Li, Zeyan Li, Kanglin Yin, Xiaohui Nie, Wenchi Zhang, Kaixin Sui, and Dan Pei. 2022. Causal inference-based root cause analysis for online service systems with intervention recognition. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3230–3240

    2022

  17. [17]

    Zeyan Li, Junjie Chen, Yihao Chen, Chengyang Luo, Yiwei Zhao, Yongqian Sun, Kaixin Sui, Xiping Wang, Dapeng Liu, Xing Jin, et al. 2023. Generic and robust root cause localization for multi-dimensional data in online service systems. Journal of Systems and Software203 (2023), 111748

    2023

  18. [18]

    Zeyan Li, Junjie Chen, Rui Jiao, Nengwen Zhao, Zhijun Wang, Shuwei Zhang, Yanjun Wu, Long Jiang, Leiqin Yan, Zikai Wang, et al. 2021. Practical root cause localization for microservice systems via trace analysis. In2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS). IEEE, 1–10

    2021

  19. [19]

    Cheng-Ming Lin, Ching Chang, Wei-Yao Wang, Kuang-Da Wang, and Wen-Chih Peng. 2024. Root cause analysis in microservice using neural granger causal discovery. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 206–213

    2024

  20. [20]

    Dewei Liu, Chuan He, Xin Peng, Fan Lin, Chenxi Zhang, Shengfang Gong, Ziang Li, Jiayu Ou, and Zheshun Wu. 2021. Microhecl: High-efficient root cause local- ization in large-scale microservice systems. In2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 338–347

    2021

  21. [21]

    Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation forest. In2008 eighth ieee international conference on data mining. IEEE, 413–422

    2008

  22. [22]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models ASE ’26, October 12–16, 2026, Munich, Germany Jiang et al. use long contexts.Transactions of the association for computational linguistics12 (2024), 157–173

    2024

  23. [23]

    Qihan Liu, Pengfei Chen, Guangba Yu, Yuanhao Lai, and Xiaoyun Li. 2025. Cause- lens: Causality-Based Interpretable Root Cause Analysis for Microservice Systems. In2025 IEEE/ACM 33rd International Symposium on Quality of Service (IWQoS). IEEE, 1–10

    2025

  24. [24]

    Yilun Liu, Ziang Chen, Song Xu, Minggui He, Shimin Tao, Weibin Meng, Yuming Xie, Tao Han, Chunguang Zhao, Jingzhou Du, et al. 2025. R-Log: Incentivizing Log Analysis Capability in LLMs via Reasoning-based Reinforcement Learning. arXiv preprint arXiv:2509.25987(2025)

    arXiv 2025

  25. [25]

    Yilun Liu, Shimin Tao, Weibin Meng, Feiyu Yao, Xiaofeng Zhao, and Hao Yang

  26. [26]

    Meng Ma, Jingmin Xu, Yuan Wang, Pengfei Chen, Zonghua Zhang, and Ping Wang. 2020. Automap: Diagnose your microservice-based web applications automatically. InProceedings of The Web Conference 2020. 246–258

    2020

  27. [27]

    InProceedings of the 2024 IEEE/ACM 46th international conference on software engineering: Companion proceedings

    Logprompt: Prompt engineering towards zero-shot and interpretable log analysis. InProceedings of the 2024 IEEE/ACM 46th international conference on software engineering: Companion proceedings. 364–365

    2024

  28. [28]

    Changhua Pei, Zexin Wang, Fengrui Liu, Zeyan Li, Yang Liu, Xiao He, Rong Kang, Tieying Zhang, Jianjun Chen, Jianhui Li, et al . 2025. Flow-of-Action: SOP Enhanced LLM-Based Multi-Agent System for Root Cause Analysis. In Companion Proceedings of the ACM on Web Conference 2025. 422–431

    2025

  29. [29]

    Yuan Meng, Shenglin Zhang, Yongqian Sun, Ruru Zhang, Zhilong Hu, Yiyin Zhang, Chenyang Jia, Zhaogang Wang, and Dan Pei. 2020. Localizing failure root causes in a microservice through causality inference. In2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS). IEEE, 1–10

    2020

  30. [30]

    Devjeet Roy, Xuchao Zhang, Rashi Bhave, Chetan Bansal, Pedro Las-Casas, Ro- drigo Fonseca, and Saravan Rajmohan. 2024. Exploring llm-based agents for root cause analysis. InCompanion proceedings of the 32nd ACM international conference on the foundations of software engineering. 208–219

    2024

  31. [31]

    Stephen Robertson, Hugo Zaragoza, and Michael Taylor. 2004. Simple BM25 extension to multiple weighted fields. InProceedings of the thirteenth ACM inter- national conference on Information and knowledge management. 42–49

    2004

  32. [32]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems36 (2023), 8634–8652

    2023

  33. [33]

    Binpeng Shi, Yu Luo, Jingya Wang, Yongxin Zhao, Shenglin Zhang, Bowen Hao, Chenyu Zhao, Yongqian Sun, Zhi Zhang, Ronghua Sun, et al. 2025. FlowXpert: Expertizing Troubleshooting Workflow Orchestration with Knowledge Base and Multi-Agent Coevolution. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 4839–4850

    2025

  34. [34]

    Yongqian Sun, Bowen Hao, Xiaotian Wang, Chenyu Zhao, Yongxin Zhao, Binpeng Shi, Shenglin Zhang, Qiao Ge, Wenhu Li, Hua Wei, et al. 2025. LLM-Augmented Ticket Aggregation for Low-cost Mobile OS Defect Resolution. InProceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. 215–226

    2025

  35. [35]

    2000.Causation, prediction, and search

    Peter Spirtes, Clark N Glymour, and Richard Scheines. 2000.Causation, prediction, and search. MIT press

    2000

  36. [36]

    Yongqian Sun, Binpeng Shi, Mingyu Mao, Minghua Ma, Sibo Xia, Shenglin Zhang, and Dan Pei. 2024. Art: A unified unsupervised framework for incident manage- ment in microservice systems. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1183–1194

    2024

  37. [37]

    Yongqian Sun, Yu Luo, Xidao Wen, Yuan Yuan, Xiaohui Nie, Shenglin Zhang, Tong Liu, and Xi Luo. 2025. TrioXpert: An automated incident management framework for microservice system.arXiv preprint arXiv:2506.10043(2025)

    arXiv 2025

  38. [38]

    Guangya Wan, Yunsheng Lu, Yuqi Wu, Mengxuan Hu, and Sheng Li. 2024. Large language models for causal discovery: Current landscape and future directions. arXiv preprint arXiv:2402.11068(2024)

    Pith/arXiv arXiv 2024

  39. [39]

    Yuni Susanti and Michael Färber. 2025. Paths to causality: finding informative subgraphs within knowledge graphs for knowledge-based causal discovery. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 2778–2789

    2025

  40. [40]

    Dongjie Wang, Zhengzhang Chen, Yanjie Fu, Yanchi Liu, and Haifeng Chen. 2023. Incremental causal graph learning for online root cause analysis. InProceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining. 2269–2278

    2023

  41. [41]

    Chenxu Wang, Xumiao Zhang, Runwei Lu, Xianshang Lin, Xuan Zeng, Xinlei Zhang, Zhe An, Gongwei Wu, Jiaqi Gao, Chen Tian, et al. 2025. Towards llm- based failure localization in production-scale networks. InProceedings of the ACM SIGCOMM 2025 Conference. 496–511

    2025

  42. [42]

    Zefan Wang, Zichuan Liu, Yingying Zhang, Aoxiao Zhong, Jihong Wang, Fengbin Yin, Lunting Fan, Lingfei Wu, and Qingsong Wen. 2024. Rcagent: Cloud root cause analysis by autonomous agents with tool-augmented large language models. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management. 4966–4974

    2024

  43. [43]

    Zeying Wang, Junhong Liu, Penghao Zhang, Xiaoyang Sun, Xu Wang, Tianyu Wo, Chunming Hu, Chengru Song, Jin Ouyang, and Renyu Yang. 2025. KAIOPS: A Platform Solution of End-to-End Multi-Modal AIOps for AI Training at Scale. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 3192–3203

    2025

  44. [44]

    Canhua Wu, Nengwen Zhao, Lixin Wang, Xiaoqin Yang, Shining Li, Ming Zhang, Xing Jin, Xidao Wen, Xiaohui Nie, Wenchi Zhang, et al. 2021. Identifying root- cause metrics for incident diagnosis in online service systems. In2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). IEEE, 91–102

    2021

  45. [45]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

    2022

  46. [46]

    Zhe Xie, Shenglin Zhang, Yitong Geng, Yao Zhang, Minghua Ma, Xiaohui Nie, Zhenhe Yao, Longlong Xu, Yongqian Sun, Wentao Li, et al. 2024. Microservice root cause analysis with limited observability through intervention recognition in the latent space. InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining. 6049–6060

    2024

  47. [47]

    Zhe Xie, Zeyan Li, Xiao He, Shenglin Zhang, Longlong Xu, Yuzhuo Yang, Tieying Zhang, Jianjun Chen, Rui Shi, and Dan Pei. 2026. FoundRoot: Towards Foundation Model for Root Cause Analysis via Structured Deep Thinking. (2026)

    2026

  48. [48]

    Jian Yang, Zian Wang, Shuangwu Chen, Huasen He, Yunpeng Hou, and Xiaofeng Jiang. 2025. HG-PAD: Heterogeneous Graph Structure Learning Aided Perfor- mance Anomaly Diagnosis in Microservice Systems.IEEE Transactions on Services Computing(2025)

    2025

  49. [49]

    Junjielong Xu, Qinan Zhang, Zhiqing Zhong, Shilin He, Chaoyun Zhang, Qingwei Lin, Dan Pei, Pinjia He, Dongmei Zhang, and Qi Zhang. 2025. OpenRCA: Can large language models locate the root cause of software failures?. InThe Thirteenth International Conference on Learning Representations

    2025

  50. [50]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations

    2022

  51. [51]

    Xiaojie Yang, Hangli Ge, Jiawei Wang, Zipei Fan, Renhe Jiang, Ryosuke Shibasaki, and Noboru Koshizuka. 2025. Causalmob: Causal human mobility prediction with llms-derived human intentions toward public events. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 1773–1784

    2025

  52. [52]

    Guangba Yu, Pengfei Chen, Yufeng Li, Hongyang Chen, Xiaoyun Li, and Zibin Zheng. 2023. Nezha: Interpretable fine-grained root causes analysis for microser- vices on multi-modal observability data. InProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 553–565

    2023

  53. [53]

    Zhenhe Yao, Haowei Ye, Changhua Pei, Guang Cheng, Guangpei Wang, Zhiwei Liu, Hongwei Chen, Hang Cui, Zeyan Li, Jianhui Li, et al . 2024. Sparserca: Unsupervised root cause analysis in sparse microservice testing traces. In2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 391–402

    2024

  54. [54]

    Shenglin Zhang, Pengxiang Jin, Zihan Lin, Yongqian Sun, Bicheng Zhang, Sibo Xia, Zhengdan Li, Zhenyu Zhong, Minghua Ma, Wa Jin, et al. 2023. Robust failure diagnosis of microservice system through multimodal data.IEEE Transactions on Services Computing16, 6 (2023), 3851–3864

    2023

  55. [55]

    Shenglin Zhang, Xiaoyu Feng, Runzhou Wang, Minghua Ma, Wenwei Gu, Yongqian Sun, Zedong Jia, Jinrui Sun, and Dan Pei. 2025. Too Many Cooks: Assessing the Need for Multi-Source Data in Microservice Failure Diagnosis. In2025 IEEE 36th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 1–12

    2025

  56. [56]

    Shenglin Zhang, Yongxin Zhao, Xiao Xiong, Yongqian Sun, Xiaohui Nie, Jiacheng Zhang, Fenglai Wang, Xian Zheng, Yuzhi Zhang, and Dan Pei. 2024. Illuminating the gray zone: Non-intrusive gray failure localization in server operating sys- tems. InCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 126–137

    2024

  57. [57]

    Shenglin Zhang, Chenyu Zhao, Yicheng Sui, Ya Su, Yongqian Sun, Yuzhi Zhang, Dan Pei, and Yizhe Wang. 2021. Robust KPI anomaly detection for large-scale software services with partial labels. In2021 IEEE 32nd international symposium on software reliability engineering (ISSRE). IEEE, 103–114

    2021

  58. [58]

    Yongxin Zhao, Shenglin Zhang, Yuxin Sun, Wenwei Gu, Yongqian Sun, Lup- ing Wang, Li Shi, Cheng Huang, Guodong Yang, Liping Zhang, et al . [n. d.]. When LLMs Listen to Experts: Accurate Failure Diagnosis in Operating Systems. ([n. d.])

  59. [59]

    Chenyu Zhao, Minghua Ma, Zhenyu Zhong, Shenglin Zhang, Zhiyuan Tan, Xiao Xiong, LuLu Yu, Jiayi Feng, Yongqian Sun, Yuzhi Zhang, et al . 2023. Robust multimodal failure detection for microservice systems. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5639–5649

    2023

  60. [60]

    Zhouruixing Zhu, Cheryl Lee, Xiaoying Tang, and Pinjia He. 2024. HeMiRCA: Fine-grained root cause analysis for microservices with heterogeneous data sources.ACM Transactions on Software Engineering and Methodology33, 8 (2024), 1–25

    2024

  61. [61]

    Xuanhe Zhou, Guoliang Li, Zhaoyan Sun, Zhiyuan Liu, Weize Chen, Jianming Wu, Jiesi Liu, Ruohang Feng, and Guoyang Zeng. 2023. D-bot: Database diagnosis system using large language models.arXiv preprint arXiv:2312.01454(2023)

    Pith/arXiv arXiv 2023

  62. [2024]

    InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering

    The potential of one-shot failure root cause analysis: Collaboration of the large language model and small classifier. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 931–943

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.