{"id":"9772c516-9b10-469f-84ff-e3f7de9b06ef","arxiv_id":"2607.20050","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A sidecar-based framework triggers microservice faults only when trace-derived temporal guards are satisfied, reporting 100% success across nine scenarios and one-attempt configuration search.","lead":"SequenceFI injects faults into microservice requests only after specified message-level events have occurred, without modifying application code. It makes timing-dependent failure scenarios—such as failures after a payment is charged—reproducible and automates the search for the right injection moment.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Guard synthesis from 10 clean traces lacks held-out validation; the 100% temporal-success claim may not transfer to varied live executions.","rationale":"The reader's weakest assumption correctly identifies the load-bearing point: the guard synthesis in Equations (3)-(4) is defined over the 10 clean traces T_r, and the paper gives no argument or experiment showing that live trials (with fault-induced state changes, scheduling variation, or different interleavings) will satisfy the same prefix-count bounds. The method is coherent and the internal consistency of the reported results is plausible, but the 100.0% temporal-success figure is exactly what would be observed if the live trials stayed within the training distribution. The undefined 'valid trials' oracle and the absence of an artifact or error bars further weaken the empirical claim. My concern does not change the verdict: the paper is a solid contribution that should be conditionally accepted pending reproducible evidence of generalization beyond the training traces. The concrete test (varying execution conditions) would directly determine whether the guard-transfer assumption is safe.","tokens_in":16403,"tokens_out":4640,"duration_ms":87963,"concrete_test":"Re-run RQ1 with the same scenarios but generate guards from the 10 clean traces, then subject the 50 live trials to deliberately varied execution conditions: random 0-200ms jitter on downstream responses before the target and a modest background load (e.g., 4 additional concurrent clients). If TS remains 100.0% with Prem=Miss=Mult=0, the trace-transfer assumption is supported; if any Prem or Miss occurs, the 100.0% claim holds only for the training distribution and the central claim must be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SequenceFI achieves 100.0% temporal success rests on the assumption that a temporal guard synthesized from 10 fault-free traces (Section VI-B) remains correct on the 50 live trials. Equations (3)-(4) define t_e as the minimum prefix count at the target position over T_r, and D_tau as atoms that distinguish earlier occurrences in those same traces. On a live trace not in T_r, if any selected event's prefix count at the target is below the training minimum, the conjunction in (6) is false and the fault is missed; if an earlier occurrence's count reaches the threshold t_e, the 'distinguishing' atom no longer distinguishes, so the guard fires prematurely. The paper reports no held-out validation, no error bars, and no precise definition of the 'valid trials' oracle (the Miss condition is qualified by 'when that window remains observable or can be aligned with a clean execution'). Since the same workload is used for guard generation and testing, the 100.0% result is consistent with the guard being overfit to T_r, not with a transferable guarantee. The evaluation therefore does not establish that the guard is robust to the interleaving and count variability that motivated temporal fault injection in the first place.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SequenceFI, a Kubernetes sidecar-based temporal fault injection framework for microservices. It models a request execution as a trace of message-level send/receive events, propagates compact evidence through an x-fic-evidence header, and injects a fault only when a static target and an After guard over occurrence-count atoms are both satisfied. Guard synthesis uses clean traces to compute minimum prefix counts at the intended target occurrence (Eq. (3)), build distinguishing sets against earlier occurrences (Eq. (4)), and solve a minimum hitting-set problem (Eq. (5)). The evaluation on four benchmarks and nine scenarios reports 100% temporal success over 450 valid trials, one-attempt TFIC generation, 95.91% end-to-end search-time reduction over H-Random, and low sidecar overhead.","tokens_in":16743,"tokens_out":5247,"duration_ms":51341,"significance":"If the central claims hold, the paper addresses a real and underexplored problem: controlling when a fault becomes active inside a distributed execution, not just where and what to inject. The formalization is clean, and the reduction of temporal-guard synthesis to a minimum hitting set is a principled way to avoid exhaustive enumeration. The sidecar implementation and the overhead measurements are also concrete and useful. However, the main evidence for the 100% temporal-success claim is currently a self-consistency check on the same workload used to synthesize the guards, and the oracle that defines 'valid trials' is underspecified. The approach is plausible, but the paper does not yet demonstrate that the synthesized guards transfer to varied live executions. The paper would be substantially strengthened by held-out or perturbed-trace validation and a fully auditable trial-validity definition.","major_comments":[{"comment":"The temporal guard is generated from 10 fault-free traces of the exact workload that is then used for the 50 injection trials. Eq. (3) takes minima over T_r and Eq. (4) only distinguishes earlier occurrences that appear in T_r. Consequently, the 100% temporal success is consistent with the guard being overfit to T_r, not with a transferable guarantee. A live trace with a lower prefix count at the target than the training minimum would cause a Miss; a different interleaving could make an earlier occurrence satisfy the guard and cause a Prem. The paper reports no held-out traces, no workload perturbation (e.g., varying concurrency or service delays), and no per-scenario confidence intervals. This is load-bearing for the central claim and needs direct validation.","section":"§VI-B and Eqs. (3)-(6)"},{"comment":"The Miss condition is defined as 'no injection occurs within the intended temporal window when that window remains observable or can be aligned with a clean execution.' This introduces an unspecified oracle: what makes a window 'observable,' and how is alignment performed? The abstract reports '450 valid trials,' but the paper does not state how many raw trials were excluded per scenario or why. Without this information, the denominator of the 100% temporal-success result is not auditable. Please specify the validity rules and report both raw and valid trial counts for every scenario.","section":"§VI-B, metric definitions"},{"comment":"For an earlier occurrence p, Dτ(p, iσ(τ)) is the set of atoms that are true at the target but false at p. If, in some trace, every atom in U_r,σ is already true at p, then D is empty and constraint (5) is unsatisfiable. The paper does not discuss this feasibility condition or report whether any scenario produced an empty distinguishing set. Since the 'no premature injection' guarantee rests on hitting every D, the presentation should either prove that a nonempty D always exists for the modeled patterns or qualify the claim and describe what the implementation does when the hitting set is infeasible.","section":"§IV-B, Eqs. (4)-(5)"},{"comment":"The RQ2 efficiency results report only single aggregate values for fault-injection attempts, solving time, and end-to-end time. H-Random and 3MileBeach-Random are stochastic baselines, so a single average over an unspecified number of independent runs is insufficient; the paper should report the number of repetitions and the distribution (or at least min/max) of attempts and times. Moreover, SequenceFI's one-attempt result is expected because the guard is synthesized from exactly the same traces used to define success; the comparison should be repeated with fresh traces and should report the full distribution of attempts. This is necessary to support the 95.91% reduction claim.","section":"§VI-C, Table VI"}],"minor_comments":[{"comment":"The text says 'encodes only FIC-relevant events'; this appears to be a typo for 'TFIC-relevant events.'","section":"§V"},{"comment":"The term 'non-intrusive' should be qualified: the approach avoids modifying application code and serialization libraries, but it does inject a sidecar and an admission webhook into the deployment. That is an infrastructure-level change and should be described as such.","section":"Abstract and §V"},{"comment":"Throughput and resource-overhead results are averaged over 'at least six' repetitions, but the figures show no error bars or confidence intervals. Reporting variance would help assess the stability of the overhead comparison.","section":"§VI-D and Figs. 5-6"},{"comment":"The threats-to-validity discussion does not mention trace variability or the train/test overlap between guard generation and RQ1 trials. The paper should address this explicitly, since it is the main external threat to the temporal-success claim.","section":"§VII-B"},{"comment":"No artifact link or code-release statement is provided, which makes independent replication of the Kubernetes prototype and the trace-generation experiments difficult.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The approach is technically interesting and likely worth publishing after the evaluation is strengthened. My main concern is that the current evaluation cannot distinguish a guard that memorizes the training traces from one that transfers to live executions. I would require held-out or perturbed-trace validation, a precise trial-validity oracle, and per-scenario variance for the RQ2 efficiency claims before accepting. Additionally, the manuscript currently lacks an artifact link, which is increasingly expected for a systems/software-engineering venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is genuinely new and the implementation is clean: a sidecar that intercepts message-level events and uses trace-derived After guards to activate faults at a precise temporal window. That is a real step beyond 3MileBeach, which works at the serialization layer and randomizes, and beyond LDFI/FastFI, which pick static targets but have no temporal predicates. The hitting-set construction in Equations (3)–(6) is a sensible way to choose a minimal set of count atoms that separate the target occurrence from earlier matching points, and the overhead numbers match what you would expect from a small proxy that only inspects headers.\n\nThe soft spot is exactly where the stress-test note points. The guard is synthesized from 10 clean traces of a workload and then scored on 50 fresh runs of the same workload. That is a reproducibility experiment, not a generalization experiment. The 100% temporal success is partly by construction against those 10 traces; nothing here shows robustness to interleavings or request mixes that differ from the training distribution. The 'valid trials' oracle also has an ambiguous escape hatch in the Miss definition, and the paper gives no error bars, no artifact, and no trace-variability information. The 95.91% search-time reduction is against randomized baselines that are deliberately undirected; fine as a headline, but not a strong comparison.\n\nNone of this is fatal. The paper is honest in Section VII about the limitations of After-only, monotonic guards and the limited protocol support. The prototype is a plausible engineering contribution, and the motivating statistics on state-changing operations, while method-level, do make the temporal window a real concern. The right fix is to strengthen the evaluation before it is accepted: hold out a workload or request mix, vary load, report trace variability, and define the oracle precisely. If the guard still fires correctly under those conditions, the claims will be much more convincing.\n\nThis deserves a serious referee. I would send it to peer review with a request for major revision on the evaluation. I would cite the design in my own work if I were building fault-injection tooling, though I would wait for the held-out validation before treating the 100% result as anything more than a demo.","headline":"Genuinely new sidecar-based TFI with a neat guard-synthesis algorithm, but the 100% success claim is a self-consistency result on the same workload and needs held-out validation before the strong claims can be trusted.","tokens_in":17175,"tokens_out":2489,"would_cite":true,"duration_ms":24633,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SequenceFI makes microservice fault injection temporally precise by gating every fault on an occurrence-count guard synthesized from clean traces.","keywords":["temporal fault injection","microservices","chaos engineering","sidecar proxy","temporal guard","event-based tracing","fault injection","resilience testing"],"falsifier":"Take one k-of-n scenario, train the guard on 10 clean traces, then run 50 trials in which a workload change inserts one extra homogeneous downstream call before the target occurrence; any trial with premature firing or no injection in the intended window falsifies the 100% temporal-success claim. A simpler check: find a trace in which the (k-1)-th and k-th occurrences of the target API have identical prefix-count vectors for all candidate atoms, in which case no After guard can distinguish them and the construction must fail.","tokens_in":16334,"feed_emoji":"🎯","tokens_out":4506,"duration_ms":43319,"temperature":0.7,"pith_summary":"The paper sets out to solve the 'when' of fault injection in microservice systems: existing injectors control which API to fail and how, but not the moment within a distributed execution at which the failure should strike. SequenceFI observes message send/receive events, propagates compact evidence through sidecar proxies, and injects only when an occurrence-count guard is satisfied. The central claim is that such temporal guards can be generated automatically from a handful of fault-free traces by finding evidence that distinguishes the intended fault moment from earlier trigger points. On four benchmarks and nine scenarios, the paper reports 100% temporal success without premature or multiple injections, effective configuration on the first attempt, and 95.91% less end-to-end search time than a trace-guided random baseline. If correct, this makes timing-dependent resilience testing practical without touching application code.","feed_headline":"Sidecar injector lands every fault at the right moment in 450 trials","feed_subtitle":"SequenceFI synthesizes event guards from clean traces to trigger timing-dependent failures on cue — no app changes.","key_machinery":"The After temporal guard: a conjunction of occurrence-count atoms of the form 'event e has occurred at least t times,' evaluated over a compact binary evidence vector propagated across service boundaries by sidecar proxies. Guards are built by selecting atoms from the candidate set U_r,σ that are satisfied at the k-th matching occurrence in every training trace but fail at each earlier occurrence p; choosing at least one atom per distinguishing set Dτ(p, iσ(τ)) reduces the problem to a minimum hitting set. This monotone, prefix-count formulation is what lets SequenceFI distinguish the intended execution context from premature triggers while keeping the propagated metadata small.","core_discovery":"SequenceFI's core claim is that a temporal fault-injection configuration can be decomposed into a static target (which API, which occurrence k, request or response phase, fault type) and an After temporal guard over event counts, and that the guard can be synthesized from clean traces. For each candidate target, the generator collects event-prefix counts at the target position across traces, keeps atoms that are true there, and then solves a minimum hitting set over the distinguishing sets that separate the target from every earlier occurrence of the same API-phase operation. The resulting guard is false at all premature trigger points and true at the intended moment. The paper reports that","pith_inferences":["The trace-stability assumption is untested: guards are learned from 10 clean traces and evaluated on 50 live trials under the same workload; under a different arrival pattern or interleaving, prefix counts could shift and cause premature or missed injections.","The monotone After-only design is also its boundary: scenarios requiring event absence, bounded intervals, or Before/Until semantics would need a richer evidence model, and the hitting-set formulation might not extend.","The claimed 100% success pertains to the evaluated scenarios; generalizing to larger fan-out or nondeterministic timing would require either many more training traces or runtime guard adaptation.","One could test transferability directly by training on one workload and evaluating on a perturbed one; that experiment is a natural next step the paper does not report."],"forward_implications":["Timing-dependent failure modes — failures after a side effect, order-sensitive concurrent responses, and k-of-n partial failures — become reproducible in routine resilience testing.","Because interposition is at the communication layer, the technique applies to polyglot microservice stacks without serialization-library instrumentation or application changes.","Automatic guard synthesis removes the need to hand-write or randomly enumerate temporal conditions; the reported one-attempt success implies the search burden collapses.","The low sidecar overhead (97.3% of no-proxy throughput, 2.3% of a full mesh sidecar's memory in the paper's measurements) makes always-on temporal monitoring feasible.","The prototype's support for HTTP/1.x and gRPC over HTTP/2 positions the scheme for cloud-native deployments."],"fun_headline_variants":["Temporal fault injection hits the exact moment, no code changes","SequenceFI times microservice faults with event-count guards","Fault injection on cue: 100% temporal success without premature hits","Timing faults without touching code: SequenceFI's guard synthesis","SequenceFI: 95.9% faster search, perfect timing in 450 trials"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the event-count patterns seen in the handful of clean traces used to build a guard will recur in live, fault-injected executions; if a live trial produces lower counts, different interleavings, or fault-induced trace changes, the guard can fire early or miss.","fun_headline_variants_meta":{"raw":{"variants":["Temporal fault injection hits the exact moment, no code changes","SequenceFI times microservice faults with event-count guards","Fault injection on cue: 100% temporal success without premature hits","Timing faults without touching code: SequenceFI's guard synthesis","SequenceFI: 95.9% faster search, perfect timing in 450 trials"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1104,"prompt_tokens":734,"completion_tokens":370,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":478,"completion_tokens_details":{"reasoning_tokens":279}},"tokens_in":478,"tokens_out":370,"duration_ms":3508,"temperature":1.0,"reasoning_tokens":279,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T10:54:40.031484+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one k-of-n scenario, train the guard on 10 clean traces, then run 50 trials in which a workload change inserts one extra homogeneous downstream call before the target occurrence; any trial with premature firing or no injection in the intended window falsifies the 100% temporal-success claim. A simpler check: find a trace in which the (k-1)-th and k-th occurrences of the target API have identical prefix-count vectors for all candidate atoms, in which case no After guard can distinguish them and the construction must fail.","supporting_citations":[],"review_version":1}