Pith. sign in

REVIEW 4 major objections 5 minor 39 references

SequenceFI makes microservice fault injection temporally precise by gating every fault on an occurrence-count guard synthesized from clean traces.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 10:54 UTC pith:B4X2AEMO

load-bearing objection Genuinely new sidecar-based TFI with a neat guard-synthesis algorithm, but the 100% success claim is a self-consistency result on the same workload and needs held-out validation before the strong claims can be trusted. the 4 major comments →

arxiv 2607.20050 v1 pith:B4X2AEMO submitted 2026-07-22 cs.SE

SequenceFI: Non-intrusive Temporal Fault Injection for Microservice Systems

classification cs.SE
keywords temporal fault injectionmicroserviceschaos engineeringsidecar proxytemporal guardevent-based tracingfault injectionresilience testing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to solve the 'when' of fault injection in microservice systems: existing injectors control which API to fail and how, but not the moment within a distributed execution at which the failure should strike. SequenceFI observes message send/receive events, propagates compact evidence through sidecar proxies, and injects only when an occurrence-count guard is satisfied. The central claim is that such temporal guards can be generated automatically from a handful of fault-free traces by finding evidence that distinguishes the intended fault moment from earlier trigger points. On four benchmarks and nine scenarios, the paper reports 100% temporal success without premature or multiple injections, effective configuration on the first attempt, and 95.91% less end-to-end search time than a trace-guided random baseline. If correct, this makes timing-dependent resilience testing practical without touching application code.

Core claim

SequenceFI's core claim is that a temporal fault-injection configuration can be decomposed into a static target (which API, which occurrence k, request or response phase, fault type) and an After temporal guard over event counts, and that the guard can be synthesized from clean traces. For each candidate target, the generator collects event-prefix counts at the target position across traces, keeps atoms that are true there, and then solves a minimum hitting set over the distinguishing sets that separate the target from every earlier occurrence of the same API-phase operation. The resulting guard is false at all premature trigger points and true at the intended moment. The paper reports that

What carries the argument

The After temporal guard: a conjunction of occurrence-count atoms of the form 'event e has occurred at least t times,' evaluated over a compact binary evidence vector propagated across service boundaries by sidecar proxies. Guards are built by selecting atoms from the candidate set U_r,σ that are satisfied at the k-th matching occurrence in every training trace but fail at each earlier occurrence p; choosing at least one atom per distinguishing set Dτ(p, iσ(τ)) reduces the problem to a minimum hitting set. This monotone, prefix-count formulation is what lets SequenceFI distinguish the intended execution context from premature triggers while keeping the propagated metadata small.

Load-bearing premise

The load-bearing premise is that the event-count patterns seen in the handful of clean traces used to build a guard will recur in live, fault-injected executions; if a live trial produces lower counts, different interleavings, or fault-induced trace changes, the guard can fire early or miss.

What would settle it

Take one k-of-n scenario, train the guard on 10 clean traces, then run 50 trials in which a workload change inserts one extra homogeneous downstream call before the target occurrence; any trial with premature firing or no injection in the intended window falsifies the 100% temporal-success claim. A simpler check: find a trace in which the (k-1)-th and k-th occurrences of the target API have identical prefix-count vectors for all candidate atoms, in which case no After guard can distinguish them and the construction must fail.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Timing-dependent failure modes — failures after a side effect, order-sensitive concurrent responses, and k-of-n partial failures — become reproducible in routine resilience testing.
  • Because interposition is at the communication layer, the technique applies to polyglot microservice stacks without serialization-library instrumentation or application changes.
  • Automatic guard synthesis removes the need to hand-write or randomly enumerate temporal conditions; the reported one-attempt success implies the search burden collapses.
  • The low sidecar overhead (97.3% of no-proxy throughput, 2.3% of a full mesh sidecar's memory in the paper's measurements) makes always-on temporal monitoring feasible.
  • The prototype's support for HTTP/1.x and gRPC over HTTP/2 positions the scheme for cloud-native deployments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The trace-stability assumption is untested: guards are learned from 10 clean traces and evaluated on 50 live trials under the same workload; under a different arrival pattern or interleaving, prefix counts could shift and cause premature or missed injections.
  • The monotone After-only design is also its boundary: scenarios requiring event absence, bounded intervals, or Before/Until semantics would need a richer evidence model, and the hitting-set formulation might not extend.
  • The claimed 100% success pertains to the evaluated scenarios; generalizing to larger fan-out or nondeterministic timing would require either many more training traces or runtime guard adaptation.
  • One could test transferability directly by training on one workload and evaluating on a perturbed one; that experiment is a natural next step the paper does not report.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SequenceFI, a Kubernetes sidecar-based temporal fault injection framework for microservices. It models a request execution as a trace of message-level send/receive events, propagates compact evidence through an x-fic-evidence header, and injects a fault only when a static target and an After guard over occurrence-count atoms are both satisfied. Guard synthesis uses clean traces to compute minimum prefix counts at the intended target occurrence (Eq. (3)), build distinguishing sets against earlier occurrences (Eq. (4)), and solve a minimum hitting-set problem (Eq. (5)). The evaluation on four benchmarks and nine scenarios reports 100% temporal success over 450 valid trials, one-attempt TFIC generation, 95.91% end-to-end search-time reduction over H-Random, and low sidecar overhead.

Significance. If the central claims hold, the paper addresses a real and underexplored problem: controlling when a fault becomes active inside a distributed execution, not just where and what to inject. The formalization is clean, and the reduction of temporal-guard synthesis to a minimum hitting set is a principled way to avoid exhaustive enumeration. The sidecar implementation and the overhead measurements are also concrete and useful. However, the main evidence for the 100% temporal-success claim is currently a self-consistency check on the same workload used to synthesize the guards, and the oracle that defines 'valid trials' is underspecified. The approach is plausible, but the paper does not yet demonstrate that the synthesized guards transfer to varied live executions. The paper would be substantially strengthened by held-out or perturbed-trace validation and a fully auditable trial-validity definition.

major comments (4)
  1. [§VI-B and Eqs. (3)-(6)] The temporal guard is generated from 10 fault-free traces of the exact workload that is then used for the 50 injection trials. Eq. (3) takes minima over T_r and Eq. (4) only distinguishes earlier occurrences that appear in T_r. Consequently, the 100% temporal success is consistent with the guard being overfit to T_r, not with a transferable guarantee. A live trace with a lower prefix count at the target than the training minimum would cause a Miss; a different interleaving could make an earlier occurrence satisfy the guard and cause a Prem. The paper reports no held-out traces, no workload perturbation (e.g., varying concurrency or service delays), and no per-scenario confidence intervals. This is load-bearing for the central claim and needs direct validation.
  2. [§VI-B, metric definitions] The Miss condition is defined as 'no injection occurs within the intended temporal window when that window remains observable or can be aligned with a clean execution.' This introduces an unspecified oracle: what makes a window 'observable,' and how is alignment performed? The abstract reports '450 valid trials,' but the paper does not state how many raw trials were excluded per scenario or why. Without this information, the denominator of the 100% temporal-success result is not auditable. Please specify the validity rules and report both raw and valid trial counts for every scenario.
  3. [§IV-B, Eqs. (4)-(5)] For an earlier occurrence p, Dτ(p, iσ(τ)) is the set of atoms that are true at the target but false at p. If, in some trace, every atom in U_r,σ is already true at p, then D is empty and constraint (5) is unsatisfiable. The paper does not discuss this feasibility condition or report whether any scenario produced an empty distinguishing set. Since the 'no premature injection' guarantee rests on hitting every D, the presentation should either prove that a nonempty D always exists for the modeled patterns or qualify the claim and describe what the implementation does when the hitting set is infeasible.
  4. [§VI-C, Table VI] The RQ2 efficiency results report only single aggregate values for fault-injection attempts, solving time, and end-to-end time. H-Random and 3MileBeach-Random are stochastic baselines, so a single average over an unspecified number of independent runs is insufficient; the paper should report the number of repetitions and the distribution (or at least min/max) of attempts and times. Moreover, SequenceFI's one-attempt result is expected because the guard is synthesized from exactly the same traces used to define success; the comparison should be repeated with fresh traces and should report the full distribution of attempts. This is necessary to support the 95.91% reduction claim.
minor comments (5)
  1. [§V] The text says 'encodes only FIC-relevant events'; this appears to be a typo for 'TFIC-relevant events.'
  2. [Abstract and §V] The term 'non-intrusive' should be qualified: the approach avoids modifying application code and serialization libraries, but it does inject a sidecar and an admission webhook into the deployment. That is an infrastructure-level change and should be described as such.
  3. [§VI-D and Figs. 5-6] Throughput and resource-overhead results are averaged over 'at least six' repetitions, but the figures show no error bars or confidence intervals. Reporting variance would help assess the stability of the overhead comparison.
  4. [§VII-B] The threats-to-validity discussion does not mention trace variability or the train/test overlap between guard generation and RQ1 trials. The paper should address this explicitly, since it is the main external threat to the temporal-success claim.
  5. [General] No artifact link or code-release statement is provided, which makes independent replication of the Kubernetes prototype and the trace-generation experiments difficult.

Circularity Check

0 steps flagged

No significant circularity; guard synthesis and the RQ1/RQ2 evaluations are separate fitting and testing steps, with only a validity concern about trace transfer.

full rationale

SequenceFI's guard synthesis (Eqs. 3-6) does construct a guard that is true at the target occurrence and false at earlier occurrences on the trace set T_r, so on the 10 training traces the guard enforces the RQ1 success conditions by construction. However, RQ1 then evaluates on 50 separate injection trials after the guard is fixed; TS is defined by the behavioral oracle (CW/PS/Prem/Miss/Mult), not merely by the guard's truth value. A live trial with different event counts or interleaving could make the guard fire prematurely or miss, so the 100% result is not logically forced by Eq. (5) on held-out trials. The same-workload training/testing protocol in Section VI-B is a legitimate external-validity concern: the paper does not show the guard transfers to varied traces, and the Miss condition's 'aligned with a clean execution' qualifier is underspecified. But that is an evaluation weakness, not a circular reduction. The one-attempt RQ2 result is the intended behavior of a solver whose objective closely matches the RQ1 oracle, yet the comparison against H-Random is still an empirical efficiency measurement. The only self-citation (FastFI, ref. [9]) is used to adapt static configuration solving; the central temporal-guard contribution does not depend on a self-cited uniqueness theorem or an ansatz smuggled in by citation. No equation is defined in terms of the outcome it purportedly predicts.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 3 invented entities

The central claim rests on a small number of hand-chosen experimental settings (occurrence index k, 10 training traces) and on domain assumptions about the expressiveness of monotonic After guards, the sufficiency of message-level events, the representativeness of clean traces, the correctness of prior lineage-driven target selection, and the proxy nature of the state-changing classification. The invented entities are implementation artifacts, not independently verified external predictions.

free parameters (2)
  • Occurrence index k per scenario = 1-3 across scenarios (Table V)
    Equation (1) includes k, the local occurrence index of the target API/phase operation. In each RQ1 scenario the authors choose k to pin the intended occurrence; success is defined relative to that chosen k, so this hand-selected value is part of the tested claim.
  • Number of clean traces used for guard generation = 10
    Section VI-B says 'collect 10 fault-free traces for temporal guard generation'. The min operation in Eq. (3) depends on this sample; with only 10 traces the synthesized guard may overfit to the observed interleavings.
axioms (5)
  • domain assumption All target temporal scenarios can be expressed as monotonic After guards over positive occurrence-count atoms.
    Section IV.B restricts guards to the After form; Section VII.A explicitly notes that Before/Until, bounded intervals, event absence, and deadline conditions are unsupported.
  • domain assumption Message-level send/receive events observed at sidecars are sufficient and correctly propagated to reconstruct temporal context.
    Section IV.A and Figure 3 assume event evidence can be encoded in x-fic-evidence and associated via W3C Trace Context; if proxies or middleware strip or lose headers, the guards fail.
  • domain assumption The 10 clean traces used for guard generation are representative of the 50 live trials.
    Equations (3)-(4) use min counts and distinguishing sets over T_r; runtime variance not seen in T_r can cause premature triggering or missed injections. This is assumed in Section VI-B.
  • domain assumption Lineage-driven static-target selection from FastFI/LDFI produces correct candidate static configurations.
    Section IV.B states 'SequenceFI adapts FastFI' to derive static candidates; the correctness of σ is inherited from prior work and not re-verified independently.
  • domain assumption HTTP method-level classification (POST/PUT/PATCH/DELETE as state-changing) approximates side-effect behavior.
    Section II-B1 explicitly calls this a 'method-level approximation'; it motivates temporal FI but is not used to compute the guard-synthesis results.
invented entities (3)
  • TFIProxy sidecar no independent evidence
    purpose: Intercepts HTTP/gRPC messages, records send/receive events, evaluates temporal guards, and injects faults without modifying application code.
    Implemented and evaluated only within this paper; no external falsifiable prediction beyond the reported benchmarks.
  • x-fic-evidence header no independent evidence
    purpose: Carries compact binary temporal evidence across service boundaries for guard evaluation.
    Custom header introduced by the paper; no evidence it interoperates with arbitrary middleware that may strip unknown headers.
  • tracestate token map no independent evidence
    purpose: Associates downstream responses with their parent request contexts so evidence can be merged correctly.
    In-memory local token-to-evidence map; no external validation.

pith-pipeline@v1.3.0-alltime-deepseek · 16096 in / 12407 out tokens · 102661 ms · 2026-08-01T10:54:40.031484+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of SequenceFI: Non-intrusive Temporal Fault Injection for Microservice Systems." pith.science (2026). https://pith.science/paper/B4X2AEMO

@misc{pith2026260720050,
  author       = {Pith},
  title        = {Pith review of: SequenceFI: Non-intrusive Temporal Fault Injection for Microservice Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B4X2AEMO}},
  note         = {Machine review of arXiv:2607.20050}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Fault injection is widely used to evaluate the resilience of microservice systems, where client requests often span multiple services and execution stages. Existing request-level techniques usually control where and what faults are injected, but not when they are activated within a distributed execution. This limitation makes it difficult to reproduce timing-dependent failures, such as failures after state-changing side effects, order-sensitive concurrent responses, and partial failures among repeated downstream calls. This paper presents SequenceFI, a non-intrusive framework for temporal fault injection in microservice systems. SequenceFI observes message-level send and receive events, propagates compact temporal evidence along request executions, and triggers faults only when occurrence-sensitive temporal guards are satisfied. It further synthesizes temporal guards from traces, reducing the need for exhaustive enumeration of temporal fault-injection configurations, while requiring no modifications to application code or serialization libraries. We implement SequenceFI on Kubernetes and evaluate it on four widely used microservice benchmarks. Across nine temporal-fault scenarios and 450 valid trials, SequenceFI achieves 100.0\% temporal success without premature or multiple injections, finds effective configurations in one attempt on average, and reduces aggregate end-to-end search time by 95.91\% compared with H-Random.

Figures

Figures reproduced from arXiv: 2607.20050 by Bing Li, Jian Wang, Shaolin Tan, Yuzhen Tan.

Figure 1
Figure 1. Figure 1: Message-level view of a microservice request execution. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Three representative temporal fault patterns in microservices. Event-guarded fault injection uses send/receive events as temporal evidence to target [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of SequenceFI. behavior depends on the arrival order of concurrent responses. Static FI can make RiskCheck fail, but cannot ensure that the failure occurs after CreateUser has influenced the caller in the intended interleaving. TFI uses ordering evidence to activate the RiskCheck failure after the CreateUser success has been observed, making the relevant interleaving reproducible. C. k-of-n Partia… view at source ↗
Figure 4
Figure 4. Figure 4: Implementation overview of SequenceFI’s TFIProxy sidecar. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Throughput under different deployment modes. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Sidecar CPU and memory usage. require reasoning about non-monotonic temporal conditions. Future work will extend SequenceFI with richer temporal operators, event types, and temporal predicates. B. Threats to Validity 1) Internal Threats to Validity: Internal threats concern baseline fairness and implementation artifacts. Because no prior system simultaneously supports sidecar-based deploy￾ment and temporal… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 1 canonical work pages

  1. [1]

    Chaos engineering,

    A. Basiri, N. Behnam, R. de Rooij, L. Hochstein, L. Kosewski, J. Reynolds, and C. Rosenthal, “Chaos engineering,”IEEE Software, vol. 33, no. 3, pp. 35–41, 2016

  2. [2]

    Automating chaos experiments in production,

    A. Basiri, L. Hochstein, N. Jones, and H. Tucker, “Automating chaos experiments in production,” in2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2019, pp. 31–40

  3. [3]

    Chaos monkey,

    Netflix, “Chaos monkey,” https://github.com/Netflix/chaosmonkey/, 2025

  4. [4]

    Chaos mesh,

    C. N. C. Foundation, “Chaos mesh,” https://chaos-mesh.org/, 2025

  5. [5]

    Chaos blade,

    C. N. C. Foundation, “Chaos blade,” https://chaosblade.io/, 2025

  6. [6]

    Microfi: Non-intrusive and prioritized request-level fault injection for microservice applications,

    H. Chen, P. Chen, G. Yu, X. Li, and Z. He, “Microfi: Non-intrusive and prioritized request-level fault injection for microservice applications,” IEEE Transactions on Dependable and Secure Computing, vol. 21, no. 5, pp. 4921–4938, 2024

  7. [7]

    3milebeach: A tracer with teeth,

    J. Zhang, R. Ferydouni, A. Montana, D. Bittman, and P. Alvaro, “3milebeach: A tracer with teeth,” inProceedings of the ACM Symposium on Cloud Computing, ser. SoCC ’21. New York, NY , USA: Association for Computing Machinery, 2021, p. 458–472. [Online]. Available: https://doi.org/10.1145/3472883.3486986

  8. [8]

    Lineage-driven fault injection,

    P. Alvaro, J. Rosen, and J. M. Hellerstein, “Lineage-driven fault injection,” inProceedings of the 2015 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD ’15. New York, NY , USA: Association for Computing Machinery, 2015, p. 331–346. [Online]. Available: https://doi.org/10.1145/2723372.2723711

  9. [9]

    Fastfi: Enhancing api call-site robustness in microservice-based systems with fault injection,

    Y . Tan, J. Wang, S. Xie, B. Li, Y . Yong, N. Zhang, and S. Tan, “Fastfi: Enhancing api call-site robustness in microservice-based systems with fault injection,”ACM Trans. Softw. Eng. Methodol., May 2026, just Accepted. [Online]. Available: https://doi.org/10.1145/3813806

  10. [10]

    RFC 9110: HTTP Semantics,

    R. T. Fielding, M. Nottingham, and J. Reschke, “RFC 9110: HTTP Semantics,” Internet Engineering Task Force, Request for Comments 9110, 2022, accessed: 2026-06-01. [Online]. Available: https://www.rfc-editor.org/rfc/rfc9110.html

  11. [11]

    Microsoft Graph OpenAPI Metadata,

    Microsoft, “Microsoft Graph OpenAPI Metadata,” GitHub repository, 2026, accessed: 2026-06-01. [Online]. Available: https://github.com/ microsoftgraph/msgraph-metadata

  12. [12]

    Azure REST API Specifications,

    Microsoft Azure, “Azure REST API Specifications,” GitHub repository, 2026, accessed: 2026-06-01. [Online]. Available: https://github.com/ Azure/azure-rest-api-specs

  13. [13]

    Google APIs Discovery Artifact Manager,

    Google, “Google APIs Discovery Artifact Manager,” GitHub repository, 2026, accessed: 2026-06-01. [Online]. Available: https://github.com/ googleapis/discovery-artifact-manager

  14. [14]

    Cloudflare API Schemas,

    Cloudflare, “Cloudflare API Schemas,” GitHub repository, 2026, accessed: 2026-06-01. [Online]. Available: https://github.com/cloudflare/ api-schemas

  15. [15]

    GitHub REST API OpenAPI Description,

    GitHub, “GitHub REST API OpenAPI Description,” GitHub repository, 2026, accessed: 2026-06-01. [Online]. Available: https://github.com/ github/rest-api-description

  16. [16]

    DigitalOcean API v2 OpenAPI Specification,

    DigitalOcean, “DigitalOcean API v2 OpenAPI Specification,” GitHub repository, 2026, accessed: 2026-06-01. [Online]. Available: https: //github.com/digitalocean/openapi

  17. [17]

    Stripe OpenAPI Specification,

    Stripe, “Stripe OpenAPI Specification,” GitHub repository, 2026, accessed: 2026-06-01. [Online]. Available: https://github.com/stripe/ openapi

  18. [18]

    Jira Cloud Platform REST API v3,

    Atlassian, “Jira Cloud Platform REST API v3,” Atlassian Developer Documentation, 2026, accessed: 2026-06-01. [Online]. Available: https://developer.atlassian.com/cloud/jira/platform/rest/v3/intro/

  19. [19]

    GitHub Topics: Serialization Library,

    GitHub, “GitHub Topics: Serialization Library,” https://github.com/ topics/serialization-library, accessed: 2026-06-01

  20. [20]

    Kubernetes webhook,

    C. N. C. Foundation, “Kubernetes webhook,” https://kubernetes.io/docs/ reference/access-authn-authz/webhook/, 2026

  21. [21]

    Opentelemetry,

    C. N. C. Foundation, “Opentelemetry,” https://opentelemetry.io/, 2026

  22. [22]

    Trace context,

    W. W. W. Consortium, “Trace context,” https://www.w3.org/TR/ trace-context/, 2021

  23. [23]

    Online boutique,

    G. C. Platform, “Online boutique,” https://github.com/ GoogleCloudPlatform/microservices-demo/, 2025

  24. [24]

    An open-source benchmark suite for microservices and their hardware- software implications for cloud & edge systems,

    Y . Gan, Y . Zhang, D. Cheng, A. Shetty, P. Rathi, N. Katarki, A. Bruno, J. Hu, B. Ritchken, B. Jackson, K. Hu, M. Pancholi, Y . He, B. Clancy, C. Colen, F. Wen, C. Leung, S. Wang, L. Zaruvinsky, M. Espinosa, R. Lin, Z. Liu, J. Padilla, and C. Delimitrou, “An open-source benchmark suite for microservices and their hardware- software implications for cloud...

  25. [25]

    Sock shop,

    microservices demo, “Sock shop,” https://github.com/ microservices-demo/microservices-demo/, 2025

  26. [26]

    Benchmarking microservice systems for software engineering research,

    X. Zhou, X. Peng, T. Xie, J. Sun, C. Xu, C. Ji, and W. Zhao, “Benchmarking microservice systems for software engineering research,” inProceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, ser. ICSE ’18. New York, NY , USA: Association for Computing Machinery, 2018, p. 323–324. [Online]. Available: https://doi.or...

  27. [27]

    Coverage based failure injection toward efficient chaos engineering,

    H. Ikeuchi, A. Watanabe, and Y . Takahashi, “Coverage based failure injection toward efficient chaos engineering,” inICC 2023 - IEEE International Conference on Communications, 2023, pp. 4571–4577

  28. [28]

    Fault injec- tion analytics: A novel approach to discover failure modes in cloud- computing systems,

    D. Cotroneo, L. De Simone, P. Liguori, and R. Natella, “Fault injec- tion analytics: A novel approach to discover failure modes in cloud- computing systems,”IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 3, pp. 1476–1491, 2022

  29. [29]

    Fault injection based interventional causal learning for distributed applications,

    Q. Wang, J. Rios, S. Jha, K. Shanmugam, F. Bagehorn, X. Yang, R. Filepp, N. Abe, and L. Shwartz, “Fault injection based interventional causal learning for distributed applications,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 13, pp. 15 738–15 744, 2024

  30. [30]

    Identifying and prioritizing chaos experiments by using established risk analysis techniques,

    D. Kesim, A. van Hoorn, S. Frank, and M. H ¨aussler, “Identifying and prioritizing chaos experiments by using established risk analysis techniques,” in2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), 2020, pp. 229–240

  31. [31]

    Maximizing error injection realism for chaos engineering with system calls,

    L. Zhang, B. Morin, B. Baudry, and M. Monperrus, “Maximizing error injection realism for chaos engineering with system calls,”IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 4, pp. 2695–2708, 2022

  32. [32]

    Push-Button reliability testing for Cloud-Backed applications with rainmaker,

    Y . Chen, X. Sun, S. Nath, Z. Yang, and T. Xu, “Push-Button reliability testing for Cloud-Backed applications with rainmaker,” in20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). Boston, MA: USENIX Association, 2023, pp. 1701–1716. [Online]. Available: https://www.usenix.org/conference/ nsdi23/presentation/chen-yinfang

  33. [33]

    Efficient reproduction of fault-induced failures in distributed systems with feedback-driven fault injection,

    J. Pan, H. Wu, T. Leesatapornwongsa, S. Nath, and P. Huang, “Efficient reproduction of fault-induced failures in distributed systems with feedback-driven fault injection,” inProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles, ser. SOSP ’24. New York, NY , USA: Association for Computing Machinery, 2024, pp. 46–62. [Online]. Availa...

  34. [34]

    One-Size-Fits- None: Understanding and enhancing Slow-Fault tolerance in modern distributed systems,

    R. Lu, Y . Lu, Y . Jiang, G. Xue, and P. Huang, “One-Size-Fits- None: Understanding and enhancing Slow-Fault tolerance in modern distributed systems,” in22nd USENIX Symposium on Networked Systems Design and Implementation (NSDI 25). Philadelphia, PA: USENIX Association, 2025, pp. 359–378. [Online]. Available: https://www.usenix.org/conference/nsdi25/prese...

  35. [35]

    Automating failure testing research at internet scale,

    P. Alvaro, K. Andrus, C. Sanden, C. Rosenthal, A. Basiri, and L. Hochstein, “Automating failure testing research at internet scale,” inProceedings of the Seventh ACM Symposium on Cloud Computing, ser. SoCC ’16. New York, NY , USA: Association for Computing Machinery, 2016, pp. 17–28. [Online]. Available: https://doi.org/10.1145/2987550.2987555

  36. [36]

    Fitness- guided resilience testing of microservice-based applications,

    Z. Long, G. Wu, X. Chen, C. Cui, W. Chen, and J. Wei, “Fitness- guided resilience testing of microservice-based applications,” in2020 IEEE International Conference on Web Services (ICWS), 2020, pp. 151– 158

  37. [37]

    Service-level fault injection testing,

    C. S. Meiklejohn, A. Estrada, Y . Song, H. Miller, and R. Padhye, “Service-level fault injection testing,” inProceedings of the ACM Symposium on Cloud Computing, ser. SoCC ’21. New York, NY , USA: Association for Computing Machinery, 2021, pp. 388–402. [Online]. Available: https://doi.org/10.1145/3472883.3487005

  38. [38]

    Microres: Versatile resilience profiling in microservices via degradation dissemination indexing,

    T. Yang, C. Lee, J. Shen, Y . Su, C. Feng, Y . Yang, and M. R. Lyu, “Microres: Versatile resilience profiling in microservices via degradation dissemination indexing,” inProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2024. New York, NY , USA: Association for Computing Machinery, 2024, pp. 325–337. ...

  39. [39]

    Gremlin: Systematic resilience testing of microservices,

    V . Heorhiadi, S. Rajagopalan, H. Jamjoom, M. K. Reiter, and V . Sekar, “Gremlin: Systematic resilience testing of microservices,” in2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS), 2016, pp. 57–66