Pith. sign in

REVIEW 4 major objections 5 minor 29 references

RADICE: Causal Graph Based Root Cause Analysis for System Performance Diagnostic

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read The paper claims that augmenting time-series causal discovery with partial expert knowledge and correlation-based filtering yields a root-cause causal sub-graph that outperforms correlation-only and discovery-only baselines in simulated…

desk verdict Useful hybrid RCA pipeline with a clear spec and real simulation gains, but the headline causal sub-graph output is never directly evaluated and the domain knowledge is only tested in an oracle setup. read the letter →

arxiv 2501.11545 v1 pith:IXB3PYCK submitted 2025-01-20 cs.SE

classification cs.SE
keywords rootcauseanalysissystemperformancediagnosticcausalgraphdiscoverydomainknowledgepartialadjustedcorrelationscoretimeseriesanomaly
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to solve a practical diagnostic problem: when a system performance metric drops, which underlying component caused it, and through which causal path did the anomaly spread? Its answer, RADICE, is a pipeline that starts from a causal graph learned from monitoring data, sharpens it with partial domain knowledge (a few expert-directed edges and a node-level ordering), filters candidate metrics by an adjusted correlation score, and outputs a root-cause causal sub-graph connecting the root causes to the performance metric. The paper argues this matters because full expert causal models are usually unavailable, while correlation alone produces false root causes. In simulations over ground-truth graphs of 5 to 25 nodes, RADICE with partial knowledge attained the highest recall and precision among the compared methods, and a real advertising-system case study produced sub-graphs that domain experts confirmed.

What carries the argument

The load-bearing mechanism is the causal domain knowledge model plus the four-phase pipeline. The knowledge model has three parts: domain directed edges, which are known instantaneous cause-effect pairs; a domain node level function level(·), which encodes higher-order causal ordering so that a metric cannot be caused by a higher-level metric; and graph refinement rules, which state necessary conditions such as the required correlation sign with the performance metric. The enhancement phase, formalized in Algorithm 1, relies on a constraint-based time-series causal discovery algorithm, PCMCI+, to produce the initial graph; it then inserts every domain edge, removes learned edges that violate level(·), orients undirected edges first by level(·) and then by an entropy-based orientation strategy that discards inconclusive or cycle-forming orientations. The refinement phase replaces plain correlation with an adjusted correlation score computed under smoothing and time-shift penalties, and the subtraction phase trims the enhanced graph to the paths connecting candidate root causes to the performance metric.

What would settle it

In the paper's simulation setup, randomly corrupt a fraction of the supplied domain directed edges or reverse the node level ordering and rerun the method; if recall and precision do not fall relative to the no-knowledge variant, the claim that correct partial knowledge drives the improvement is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that a root cause analysis algorithm can exploit partial, not complete, causal domain knowledge to build a usable root-cause causal sub-graph. RADICE takes a time-series dataset and a causal domain knowledge model, learns causal relations with a time-series causal discovery algorithm, then enhances the learned graph by inserting expert-directed edges without question, deleting learned edges that contradict the expert level ordering, and orienting remaining undirected edges with an entropy-based rule plus the level ordering. A refinement phase computes an adjusted correlation score, defined as the maximum Pearson correlation over moving-average smoothing and negative time-shift transformations minus penalties for the amount of transformation, and applies graph refinement rules to discard metrics that cannot be root causes. A subtraction phase keeps, for each surviving candidate, the causal path to the performance metric that passes through the most root-cause candidates, with ties broken by shortest path, yielding the root cause causal sub-graph. On simulated data with ground-truth graphs, supplying the level function and a fraction of true directed edges lifted recall and precision above all baselines, and the real-data case study reports expert-confirmed root causes.

Load-bearing premise

The load-bearing premise is that the partial domain knowledge supplied by experts is correct: a wrong expert edge or a wrong level ordering is inserted or used to delete learned edges regardless of what the data shows.

Editorial extensions

If this is right

  • A diagnostic system no longer needs a complete expert-defined causal graph; a few reliable directed edges and a level ordering can push root-cause recall and precision above correlation-based baselines.
  • The method outputs a causal sub-graph rather than only a list of root causes, so an engineer can see the intermediate components along which the anomaly propagated to the performance metric.
  • In the simulations, the largest gain came from the node level function, indicating that coarse causal-ordering information accounts for most of the improvement.
  • Even with no domain knowledge, the entropy-based orientation and adjusted-correlation refinement improve over the underlying PCMCI+ discovery algorithm on most tested graph sizes.
  • Recall and precision both drop on the largest simulated system (25 nodes), so scaling to larger systems will require additional pruning or hierarchical modeling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, a deliberately corrupted or reversed level ordering should produce worse results than no knowledge at all, because the enhancement step deletes any learned edge that conflicts with the ordering; a random-corruption robustness test would settle how much the method depends on knowledge quality.
  • The binary insert-or-delete treatment of domain edges could be replaced by probabilistic or confidence-weighted edges, letting low-confidence expert statements be overruled by strong data evidence; the paper lists probabilistic domain knowledge as future work, and such a variant is testable in the same simulations.
  • The adjusted-correlation phase already ranks candidate root causes by score, so the same score could drive a prioritized inspection list for engineers, not merely a binary filter.
  • The real-data evidence is two anomalies in one advertising system, so the method's generality across system types remains open; a natural next test is to apply RADICE to annotated outage data from different architectures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RADICE, a causal-graph-based root-cause analysis method for system performance diagnostics. Given multivariate time series and a partial causal domain knowledge model, RADICE (i) discovers an initial causal graph with PCMCI+; (ii) enhances it by inserting domain-directed edges, using a domain node-level function to discard conflicting edges and orient undirected relations, and applying an entropy-based orientation strategy; (iii) refines the candidate root-cause set with an adjusted correlation score and graph refinement knowledge; and (iv) subtracts the graph to output a root cause causal sub-graph connecting root causes to the performance metric. Experiments on simulated data with four ground-truth graph sizes compare RADICE variants to CoFlux, ε-diagnosis, PCMCI+, and TCDF, and a real advertising-system use case illustrates the outputs. The paper claims RADICE outperforms the baselines and produces root cause causal sub-graphs.

Significance. If the claims are borne out, the main contribution is a practical hybrid: a causal-discovery pipeline that accepts partial expert knowledge rather than requiring a complete structural causal model, and that returns a causal sub-graph rather than a bare list of root causes. The algorithm is specified in enough detail to be reimplemented, the ablation with RADICE P indicates the adjusted correlation score adds value, and the simulated data are made available. However, the evaluation as presented does not yet substantiate the headline sub-graph claim, and several experimental design choices (threshold tuning on the test data, oracle-sourced domain knowledge, no sub-graph scoring) leave the quantitative results weaker than the text suggests.

major comments (4)
  1. [Section IV-B and Table I] The quantitative evaluation measures only root-cause set recall and precision, yet the abstract and Section III-E define the output as a root cause causal sub-graph G_RC. No metric assesses edge correctness, path correctness, or structural distance between G_RC and the ground-truth sub-graph. This is load-bearing because the subtraction phase (Section III-E) could return the correct root-cause set while containing wrong intermediate edges; the paper's central claim is therefore unsupported by the current experiments. I ask the authors to add sub-graph-level evaluation on the simulated data, for example edge/path precision and recall or structural Hamming distance.
  2. [Section IV-B, Fig. 3 and Table I] The minSim threshold is selected on the same simulated datasets that are then used to report the final recall/precision in Table I ('we set minSim = 0.5 in the remaining of our evaluation'), and CoFlux's threshold is also chosen as 'the best results' on the same data. This parameter selection on the test set can inflate the reported gains; it should be redone with a held-out validation split, or results should be reported across the full minSim range with confidence intervals.
  3. [Algorithm 1 and Section IV-B] The domain knowledge is assumed correct and is drawn from the ground-truth graph in the simulations, but Algorithm 1 (lines 2-8) unconditionally inserts every domain directed edge and discards learned edges that conflict with the level function. There is no experiment with noisy, incorrect, or adversarially supplied domain edges or level assignments, so the claimed robustness to 'partial domain knowledge' from experts is not tested. Please add a sensitivity analysis that flips or randomizes a fraction of domain edges and level constraints and reports recall/precision of both root-cause sets and sub-graphs.
  4. [Section IV-B and Section III-A2] The graph refinement knowledge is never exercised in the simulated experiments because the paper states that correlation signs are unknown under randomized edge weights; it is only demonstrated qualitatively in the real-data case (Section IV-C). Since this is one of the four core phases and a claimed differentiator, its contribution to recall/precision is unmeasured. I ask for a simulated scenario with fixed edge signs so that refinement rules can be defined and ablated.
minor comments (5)
  1. [Section III-C1] The entropy orientation procedure is described but no ablation isolates its effect from the level-function orientation; adding RADICE with the level function but without entropy orientation would clarify the contribution of each enhancement step.
  2. [Section IV-C and Fig. 4] The real-data graphs have no ground truth and rely on expert confirmation; this is appropriate as a use case, but it should be described as illustrative rather than as evidence of sub-graph correctness.
  3. [Table I] The table reports averages over 50 runs without standard deviations or significance tests, so it is difficult to judge whether differences such as 0.74 vs. 0.70 are meaningful; please add variance information.
  4. [Section II] The comparison to reference [20] states that its improvements 'only affect the computational complexity', but no direct comparison to [20] is run; this claim should be softened or supported.
  5. [Throughout] There are several typographical and spacing issues, for example 'iflevel(u) =level(v)' in Section III-C1, 'the next session' in Section IV-A, and the acronym expansion 'causal gRAph baseD root cause analysIs for system performanCE diagnostic' in Section III; these should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RADICE's derivation is a pipeline whose outputs are evaluated against external ground truth, not equivalent to its own inputs.

full rationale

RADICE's claimed derivation chain is a pipeline: causal discovery (PCMCI+), domain-knowledge enhancement (Algorithm 1), correlation-based refinement (Algorithm 2), and graph subtraction (Section III-E). Each stage consumes external inputs (time series, expert-supplied domain edges/levels, thresholds), and the output root-cause set is compared with injected ground-truth anomalies in Table I. No equation defines the predicted root causes in terms of the evaluation scores, and no fitted parameter is renamed as a prediction: the adjusted correlation score selects candidates, but the final root cause is additionally required to have a causal path to the performance metric in the enhanced graph, so the output is not equal to the correlation input by construction. The simulation derives domain knowledge from the ground-truth graph and minSim is tuned on the simulated data; these are threats to external validity and may make the reported gains optimistic, but they are disclosed and do not make the derivation self-referential, because the ground-truth root cause is not fed into Algorithm 1 or Algorithm 2. There are no load-bearing self-citations: references [8], [17], and [21] are external method sources, not prior papers by these authors. The fact that Table I scores root-cause sets rather than the structural correctness of the returned sub-graph is an evaluation gap, not a circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method's central claim rests on the reliability of the causal discovery step, the correctness of expert-supplied domain knowledge, and the assumption that anomalies propagate through observed metrics. The most fragile item is the unconditional trust Algorithm 1 places in domain edges, because no experiment corrupts that input. The remaining parameters are user-configurable and are not justified by theory.

free parameters (4)
  • minSim = 0.5
    Chosen as best trade-off between recall and precision on the same simulated datasets used for the reported evaluation (Section IV-B, Fig. 3); not a held-out choice.
  • tau_max = 1
    User-provided maximum lag; justified by hourly sampling in the real system, but it bounds what causal structure can be found.
  • shiftPenalty = 0.004
    Hand-set constant in the adjusted correlation score (Algorithm 2); no sensitivity analysis.
  • smoothPenalty = 0.01
    Hand-set constant in the adjusted correlation score; no sensitivity analysis.
assumptions (5)
  • domain assumption Causal sufficiency, faithfulness, and stationarity assumptions required by PCMCI+ and entropy-based orientation.
    Invoked in Sections III-B and III-C via references [6], [17], [21]; authors acknowledge these are difficult to verify in real systems.
  • domain assumption Supplied domain knowledge is correct and consistent with the true causal structure.
    Algorithm 1 lines 2-8 insert domain edges unconditionally and drop conflicting learned edges; experiments only feed ground-truth-derived or expert-derived knowledge, never noisy versions.
  • domain assumption Entropic orientation rule: lower-entropy noise direction is the true cause direction.
    Used in Section III-C1 to orient undirected edges; the correctness of this assumption is not tested on the target systems.
  • domain assumption Correlation sign rules represent necessary conditions for root cause membership.
    Section III-A2; if a true root cause can be positively correlated with a performance drop in some scenarios, refinement will discard it.
  • domain assumption Anomalies are generated by a single root cause that propagates through measurable descendants to the performance metric.
    Section I and IV-B; unobserved or multiple simultaneous root causes violate the setup and are not evaluated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RADICE: Causal Graph Based Root Cause Analysis for System Performance Diagnostic." pith.science (2026). https://pith.science/paper/IXB3PYCK

@misc{pith2026250111545,
  author       = {Pith},
  title        = {Pith review of: RADICE: Causal Graph Based Root Cause Analysis for System Performance Diagnostic},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IXB3PYCK}},
  note         = {Machine review of arXiv:2501.11545}
}
read the original abstract

Root cause analysis is one of the most crucial operations in software reliability regarding system performance diagnostic. It aims to identify the root causes of system performance anomalies, allowing the resolution or the future prevention of issues that can cause millions of dollars in losses. Common existing approaches relying on data correlation or full domain expert knowledge are inaccurate or infeasible in most industrial cases, since correlation does not imply causation, and domain experts may not have full knowledge of complex and real-time systems. In this work, we define a novel causal domain knowledge model representing causal relations about the underlying system components to allow domain experts to contribute partial domain knowledge for root cause analysis. We then introduce RADICE, an algorithm that through the causal graph discovery, enhancement, refinement, and subtraction processes is able to output a root cause causal sub-graph showing the causal relations between the system components affected by the anomaly. We evaluated RADICE with simulated data and reported a real data use case, sharing the lessons we learned. The experiments show that RADICE provides better results than other baseline methods, including causal discovery algorithms and correlation based approaches for root cause analysis.

Figures

Figures reproduced from arXiv: 2501.11545 by the authors.

Figure 1
Figure 1. Example of a whole causal graph of a system (left) and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Schema of RADICE. filtering out possible false associations, as RADICE does. Instead, [1] and [9] require full knowledge of the underlying system that is almost always infeasible to have. In contrast, RADICE allows partial domain knowledge to improve the causal discovery performance and filter out spurious root causes. Further works [12], [13] do not incorporate domain knowledge and rely solely on (modified) causal … view at source ↗
Figure 3
Figure 3. Results for minSim. It shows recall and precision of RADICE w/o DK and RADICE(L, 50E) with simulated data (with N = 10 and N = 15 nodes) varying minSim. RADICE P(L) and RADICE P(L, kE) with different do￾main knowledge models) in which we replaced the adjusted correlation score (see section III-D) with the standard Pear￾son correlation computed between candidate and performance metrics, to show the benefits of the ca… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Results with advertising data. It shows the causal sub [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 26 canonical work pages

  1. [1]

    In: International Conference on Machine Learning

    Budhathoki, K., Minorics, L., Bl ¨obaum, P., Janzing, D.: Causal structure- based root cause analysis of outliers. In: International Conference on Machine Learning. pp. 2357–2369. PMLR (2022)

  2. [2]

    1887–1895 (2014)

    Chen, P., Qi, Y ., Zheng, P., Hou, D.: Causeinfer: Automatic and distributed performance diagnosis with hierarchical causality graph in large distributed systems pp. 1887–1895 (2014)

  3. [3]

    In: The world wide web conference

    Chen, Y ., Yang, X., Lin, Q., Zhang, H., Gao, F., Xu, Z., Dang, Y ., Zhang, D., Dong, H., Xu, Y ., et al.: Outage prediction and diagnosis for cloud service systems. In: The world wide web conference. pp. 2659–2665 (2019)

  4. [4]

    https://www.dynatrace.com/support/ help/platform/davis-ai/on-demand-analysis/causal-correlation-analysis, last accessed 1 Nov 2024

    Davis causal correlation analysis. https://www.dynatrace.com/support/ help/platform/davis-ai/on-demand-analysis/causal-correlation-analysis, last accessed 1 Nov 2024

  5. [5]

    In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

    Gan, Y ., Liang, M., Dev, S., Lo, D., Delimitrou, C.: Sage: practical and scalable ml-driven performance debugging in microservices. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. pp. 135– 151 (2021)

  6. [6]

    ACM Computing Surveys (CSUR) 53(4), 1–37 (2020)

    Guo, R., Cheng, L., Li, J., Hahn, P.R., Liu, H.: A survey of learning causality with data: Problems and methods. ACM Computing Surveys (CSUR) 53(4), 1–37 (2020)

  7. [7]

    Advances in Neural Information Processing Systems 35, 31158–31170 (2022)

    Ikram, A., Chakraborty, S., Mitra, S., Saini, S., Bagchi, S., Kocaoglu, M.: Root cause analysis of failures in microservices through causal discovery. Advances in Neural Information Processing Systems 35, 31158–31170 (2022)

  8. [8]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Kocaoglu, M., Dimakis, A., Vishwanath, S., Hassibi, B.: Entropic causal inference. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 31 (2017)

Show all 29 references
  1. [9]

    In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    Li, M., Li, Z., Yin, K., Nie, X., Zhang, W., Sui, K., Pei, D.: Causal inference-based root cause analysis for online service systems with intervention recognition. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 3230–3240 (2022)

  2. [10]

    Proceedings of the ACM on Measurement and Analysis of Computing Systems 4(2), 1–23 (2020)

    Lin, F., Muzumdar, K., Laptev, N.P., Curelea, M.V ., Lee, S., Sankar, S.: Fast dimensional analysis for root cause investigation in a large- scale service environment. Proceedings of the ACM on Measurement and Analysis of Computing Systems 4(2), 1–23 (2020)

  3. [11]

    In: Proceed- ings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining

    Luo, C., Lou, J.G., Lin, Q., Fu, Q., Ding, R., Zhang, D., Wang, Z.: Correlating events with time series for incident diagnosis. In: Proceed- ings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 1583–1592 (2014)

  4. [12]

    In: Proceedings of The Web Conference 2020

    Ma, M., Xu, J., Wang, Y ., Chen, P., Zhang, Z., Wang, P.: Automap: Diagnose your microservice-based web applications automatically. In: Proceedings of The Web Conference 2020. pp. 246–258 (2020)

  5. [13]

    2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS) pp

    Meng, Y ., Zhang, S., Sun, Y ., Zhang, R., Hu, Z., Zhang, Y ., Jia, C., Wang, Z., Pei, D.: Localizing failure root causes in a microservice through causality inference. 2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS) pp. 1–10 (2020)

  6. [14]

    Machine Learning and Knowledge Extraction 1(1), 19 (2019)

    Nauta, M., Bucur, D., Seifert, C.: Causal discovery with attention- based convolutional neural networks. Machine Learning and Knowledge Extraction 1(1), 19 (2019)

  7. [15]

    ACM Transactions on Knowledge Discovery from Data 18(3), 1–31 (2023)

    Ou, W., Chen, B., Dai, X., Zhang, W., Liu, W., Tang, R., Yu, Y .: A survey on bid optimization in real-time bidding display advertising. ACM Transactions on Knowledge Discovery from Data 18(3), 1–31 (2023)

  8. [16]

    https://figshare.com/s/ e5a2ee4aa3ff502f46c8, last accessed 8 Nov 2024

    Radice: simulated data repository. https://figshare.com/s/ e5a2ee4aa3ff502f46c8, last accessed 8 Nov 2024

  9. [17]

    Biometrika 82(4), 669– 688 (1995)

    Pearl, J.: Causal diagrams for empirical research. Biometrika 82(4), 669– 688 (1995)

  10. [18]

    Pearl, J.: Causal inference in statistics: An overview (2009)

  11. [19]

    Cambridge university press (2009)

    Pearl, J.: Causality. Cambridge university press (2009)

  12. [20]

    Applied Sciences 10(6), 2166 (2020)

    Qiu, J., Du, Q., Yin, K., Zhang, S.L., Qian, C.: A causality mining and knowledge graph based method of root cause diagnosis for performance anomaly in cloud applications. Applied Sciences 10(6), 2166 (2020)

  13. [21]

    In: Conference on Uncertainty in Artificial Intelligence

    Runge, J.: Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets. In: Conference on Uncertainty in Artificial Intelligence. pp. 1388–1397. PMLR (2020)

  14. [22]

    In: The World Wide Web Conference

    Shan, H., Chen, Y ., Liu, H., Zhang, Y ., Xiao, X., He, X., Li, M., Ding, W.: ε-diagnosis: Unsupervised and real-time diagnosis of small-window long-tail latency in large-scale microservice platforms. In: The World Wide Web Conference. pp. 3215–3222 (2019)

  15. [23]

    ACM Computing Surveys (CSUR) 55(3), 1–39 (2022)

    Soldani, J., Brogi, A.: Anomaly detection and failure root cause analysis in (micro) service-based cloud applications: A survey. ACM Computing Surveys (CSUR) 55(3), 1–39 (2022)

  16. [24]

    MIT press (2000)

    Spirtes, P., Glymour, C.N., Scheines, R.: Causation, prediction, and search. MIT press (2000)

  17. [25]

    In: Proceedings of the International Symposium on Quality of Service

    Su, Y ., Zhao, Y ., Xia, W., Liu, R., Bu, J., Zhu, J., Cao, Y ., Li, H., Niu, C., Zhang, Y ., et al.: Coflux: robustly correlating kpis by fluctuations for service troubleshooting. In: Proceedings of the International Symposium on Quality of Service. pp. 1–10 (2019)

  18. [26]

    https://github.com/ M-Nauta/TCDF, last accessed 8 Nov 2024

    Tcdf: temporal causal discovery framework. https://github.com/ M-Nauta/TCDF, last accessed 8 Nov 2024

  19. [27]

    https://github.com/ jakobrunge/tigramite, last accessed 8 Nov 2024

    Tigramite – causal inference for time series datasets. https://github.com/ jakobrunge/tigramite, last accessed 8 Nov 2024

  20. [28]

    419–429 (2021)

    Wang, H., Wu, Z., Jiang, H., Huang, Y ., Wang, J., Kopru, S., Xie, T.: Groot: An event-graph-based approach for root cause analysis in industrial settings pp. 419–429 (2021)

  21. [29]

    31–36 (2021)

    Wu, L., Tordsson, J., Bogatinovski, J., Elmroth, E., Kao, O.: Microdiag: Fine-grained performance diagnosis for microservice systems pp. 31–36 (2021)

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.