{"id":"ba2cbf8b-6ab3-4fc9-a7d7-eeb88acd41e3","arxiv_id":"2412.03464","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Conformal CUSUM can be tuned so its pre-change statistics match the ideal likelihood-ratio CUSUM exactly, giving controlled false alarms under exchangeability alone.","lead":"This paper proves that a change-detection alarm called conformal CUSUM behaves exactly like an ideal alarm before any change, and it studies how fast the alarm reacts after a real change. The result makes false-alarm rates easier to calibrate while keeping the method valid under weaker assumptions about the data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 1's exact match with the oracle CUSUM is conditional on the true pair (Q0,Q1); under misspecification only the conservative bound (2) survives, so the practical reach of the validity/efficiency claim is narrower than it appears.","rationale":"The paper's central mathematical result, Proposition 1, is correct as stated: the randomized conformal p-values are iid uniform under exchangeability, so the CAO CTM increments f(p_n) are iid copies of L under the chosen Q0, matching the LRM increments. The proof's independence step is standard. I found no internal inconsistency or gap in the argument. The load-bearing weakness is the conditional nature of the match: because f is defined from a specified pair (Q0,Q1), the distributional identity ties the conformal CUSUM to the LRM under that assumed pair, not under the true data-generating law. In the realistic case where Q0 or Q1 is unknown or misspecified, the exact threshold transfer from classical CUSUM theory is unavailable, and the efficiency comparison in Theorem 5 (which is already limited to the Bernoulli case and N1=O(√N0)) does not apply. The paper is transparent about this soft-model framing, so this is a scope limitation rather than a fatal flaw, but it is the condition that must hold for the strongest reading of Proposition 1 to deliver practical value. The experiments do not resolve this, since they use the true pair to build the CTM and provide no misspecification analysis. The proposed simulation check would directly quantify the degradation.","tokens_in":11574,"tokens_out":30354,"duration_ms":299204,"concrete_test":"Simulate N0=1000 observations from N(0,1), N1=1000 from N(0.5,1). Build the CAO CTM using a deliberately misspecified pair, e.g., pre-change N(0,1.5) and post-change N(0.5,1). Calibrate the threshold c via a classical ARL2F approximation for the true pre-change N(0,1), as Proposition 1 suggests, then measure the empirical ARL2F and average detection delay of the conformal CUSUM over at least 10^4 runs. If the empirical ARL2F deviates materially from the target (say more than 20%) or the detection delay is markedly worse than the oracle LRM CUSUM with the same c, the transferability claim fails under misspecification; if not, the concern is minor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theorem, Proposition 1, is an identity relative to a chosen pair (Q0,Q1). The CAO CTM is built from L=f1/f0 (Eq. 4) and from the inverse survival function f of L under Q0 (Section 3). Its increments are f(p_n) with p_n the randomized conformal p-values, which are iid uniform under any IID model. Hence the conformal CUSUM under the true pre-change law Q0* has exactly the distribution of the LRM under the assumed Q0^∞, not under Q0*^∞. Consequently, if Q0* differs from Q0, a threshold c read off classical CUSUM theory for Q0* gives a false-alarm rate different from the target; only the conservative bound E(τ_k−τ_{k−1}) ≥ c (inequality (2)) is guaranteed. The efficiency comparison in Theorem 5 is similarly anchored to the assumed θ0, θ1 and covers only N1=O(√N0), so a misspecified or unknown Q1 voids the efficiency claim. The paper frames Q0,Q1 as a soft model, so this is an external limitation rather than an internal inconsistency, but it is the load-bearing condition for the 'same as the true-likelihood-ratio CUSUM' reading of Proposition 1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies the conformal CUSUM change-detection procedure within the 'soft model' (Burnaev–Wasserman) programme. The authors define the canonical asymptotically optimal (CAO) betting function f as a quantile inverse of the survival function of the likelihood ratio L = f1/f0 under an assumed pre-change law Q0. Proposition 1 claims that the distribution of any asymptotically optimal conformal test martingale under mere IID randomness coincides with the distribution of the likelihood ratio martingale under Q0^∞; the proof uses the uniformity and independence of randomized conformal p-values together with the quantile-transform property of f. Section 4 derives closed-form CAO betting functions for Gaussian location and scale changes and connects f to the derivative of the Neyman–Pearson ROC curve (Proposition 2). Section 5 reports simulations (Bernoulli and Gaussian changes) showing that the CAO conformal martingale tracks the likelihood ratio martingale closely after the changepoint, with validity boxplots confirming the distributional match before it, and with one example (Fig. 7) where the conformal e-procedure violates validity. Section 6 states Theorem 5, a finite-sample Bernoulli-case bound on the log-ratio of the likelihood ratio martingale to the CAO conformal test martingale after a changepoint, valid when the post-change window is O(√N0), with the proof deferred to Appendix A and a martingale multiplicative Chernoff bound proved in Appendix B.","tokens_in":11883,"tokens_out":25783,"duration_ms":225816,"significance":"The central observation, Proposition 1, is clean, correct, and genuinely useful: it transfers the pre-change distributional behavior of the oracle likelihood-ratio CUSUM to a fully nonparametric conformal procedure, so that thresholds calibrated in classical CUSUM analysis can in principle be carried over under the soft model, while the conservative validity bound E(τ_k − τ_{k−1}) ≥ c holds under exchangeability alone. The proof is elementary and verifiable: it rests on two well-established facts (randomized conformal p-values are iid uniform; any quantile inverse of a survival function composed with a uniform variable has the target law). The explicit formulas (9), (13), (17), (19), and (20) are correct, and Proposition 2 (f equals the derivative of the ROC curve) is a nice interpretive result. The experiments are honest and instructive, particularly the demonstration that the conformal e-procedure can violate validity (Fig. 7) while the conformal test martingale does not.","major_comments":[{"comment":"The application of the martingale multiplicative Chernoff bound (Theorem 6) at Eq. (24) is not justified as written. Theorem 6 requires a constant μ such that Σ_{i=1}^{N} θ_i ≤ μ almost surely, where θ_i = P(ξ_i = 1 | F_{i-1}), but the proof supplies only the expectation bound E[A] ≤ N1δ + N1(N1+1)/(2N0) on the number of anomalous steps. The conditional anomaly probabilities are not almost surely bounded by δ + n/N0 with the stated constant: for example, when the pre-change count takes its upper-deviation value K0 = θ0N0 + δN0, the anomaly probability at the first post-change step is about 2δ for the case z_1 = 1, exceeding δ + 1/N0 for large N0, and on atypical pasts the conditional probability can be of order θ1. The proof must be reworked, e.g., by first establishing a high-probability bound on Σ θ_i and then applying a two-event Chernoff argument, or by proving a genuine almost-sure bound on the conditional probabilities with correctly tracked constants. Because Theorem 5 is the paper's only theoretical efficiency result, this gap is load-bearing.","section":"Appendix A (proof of Theorem 5), Eq. (24)"},{"comment":"The scope of the distributional identity in Proposition 1 should be stated more carefully. The result says that any asymptotically optimal conformal test martingale under an arbitrary IID law behaves like the likelihood ratio martingale under the assumed Q0^∞, and the betting function f is built from the assumed pair (Q0, Q1) through L. Consequently, the recommendation to use 'the whole arsenal of the existing results in the standard theory of change detection' for choosing the threshold c is exact only when the assumed pair equals the true pre- and post-change distributions. Under misspecification, the pre-change behavior of the conformal CUSUM matches the oracle for the assumed (Q0, Q1), not for the true pair, and only the conservative bound (2) is guaranteed. The authors do flag the soft model at the outset, but the unqualified phrases 'perfectly valid' (conclusion) and the threshold-transfer claim are easy to over-read; an explicit caveat at both locations would make the practical reach of the main claim accurate.","section":"Section 3 (paragraph after Proposition 1); Section 7"}],"minor_comments":[{"comment":"The symbol N is used for the standard normal distribution function without being defined, after Φ had been used for the same object; please use one symbol consistently throughout.","section":"Section 4, Eq. (12)"},{"comment":"The right-hand side 'qk · · ·+ qK' is missing the plus signs among the terms; it should read q_k + ··· + q_K.","section":"Section 3, Eq. (6)"},{"comment":"The phrase 'assuming (ξn) is a martingale' is imprecise; the needed hypothesis is that E(ξn | F_{n−1}) = θn, not that the difference sequence itself is a martingale.","section":"Appendix B, proof of Theorem 6"},{"comment":"The phrase 'Lorden's [7, 8] worst-case definition' cites both Lorden [7] and Moustakides [8] in a way that blurs which result is whose; consider writing 'Lorden's worst-case criterion [7], for which CUSUM is optimal [8]'.","section":"Introduction, first paragraph"},{"comment":"The stray repeated text 'log LRM log CAO CTM log CeP' in the captions should be removed or formatted as a proper legend description.","section":"Section 5, captions of Figures 5 and 6"},{"comment":"The sentence 'In this paper we will refer to the assumption that the observations are IID as the assumption of randomness' is grammatically tangled; a plainer phrasing such as 'we refer to the assumption that the observations are IID as the randomness assumption' would be clearer, and 'equivalent to that of exchangeability' should be completed, e.g., 'equivalent to exchangeability for observations taking values in a standard Borel space'.","section":"Section 1, last paragraph"}],"recommendation":"major_revision","confidential_remarks":"The core of the paper — Proposition 1 and the explicit betting-function formulas — is sound and publishable. My main concern is the proof of Theorem 5, which is the only theoretical efficiency result and currently invokes a martingale Chernoff bound whose almost-sure hypothesis is not established; this needs a genuine reworking rather than a cosmetic fix. The paper is honest about the soft model, but the 'perfect validity' language could mislead readers about behavior under misspecified (Q0, Q1); a prominent caveat would be advisable. The fit with math.ST is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: Proposition 1 is the paper, and it is correct. For any chosen pair (Q0,Q1), the LRM under Q0^∞ has exactly the same distribution as any asymptotically optimal CTM under randomness. That transfers classical CUSUM threshold calibration to the conformal setting without assuming the pre-change distribution. The proof is clean and uses only that conformal p-values are iid uniform.\n\nWhat is new: the closed-form CAO betting functions for Gaussian location and scale changes (via the derivative of the Neyman-Pearson ROC curve), and the Bernoulli efficiency bound in Theorem 5. The Gaussian closing functions are elegant, and the ROC derivative interpretation is a genuinely nice touch. The paper also correctly notes that the identity extends to conformal Shiryaev-Roberts.\n\nWhere it is soft: the efficiency part is thin. Theorem 5 covers only Bernoulli, gives a loose bound, and only for N1 on the order of sqrt(N0). The paper says so itself; that is honest, but it means the title's 'efficiency' side is mostly a proof of concept. The experiments are illustrative: single seeds, no code, no multi-seed summaries. None of this undermines Proposition 1.\n\nThe stress-test concern about misspecification is real but not fatal. The identity is relative to the assumed pair; if Q0* differs from Q0, you lose the exact match and fall back on conservative bound (2). The paper explicitly uses 'soft model' language, so this is a scope condition, not a contradiction. It should be stated more prominently in the abstract, but it doesn't change the math.\n\nBottom line: the validity result is a solid, citable contribution to conformal change detection. The efficiency claims are modest and labeled as such. The experimental section needs better reproducibility for a journal version. I'd send this to a serious referee—the proposition is worth a careful check, and the paper places it in the literature honestly.","headline":"Proposition 1 is a genuine distributional identity and the paper is honest about scope; the efficiency part is thin, but the validity result alone justifies a serious referee.","tokens_in":12397,"tokens_out":1767,"would_cite":true,"duration_ms":18080,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62L10","60G42","62G10"],"pacs":[],"model":"deepseek-v4-flash","headline":"Proposition 1 shows that the conformal CUSUM based on the likelihood-ratio quantile betting function has exactly the same pre-change distribution as the classical CUSUM, so classical false-alarm thresholds transfer while only…","keywords":["conformal CUSUM","change detection","conformal test martingale","betting function","likelihood-ratio martingale","Neyman-Pearson ROC curve","exchangeability","sequential analysis"],"falsifier":"Simulate, say, 10,000 pre-change observations from $Q_0 = \\text{Bernoulli}(0.5)$ with post-change $Q_1 = \\text{Bernoulli}(0.6)$, compute conformal p-values with the likelihood-ratio nonconformity measure and the CAO betting function $f(p) = 1.2$ for $p \\le 0.5$, $0.8$ otherwise (using random tie-breaking), and test whether the empirical distribution of $f(p_n)$ equals the $Q_0$-law of $L$ (1.2 with probability 0.5, 0.8 with probability 0.5) by a Kolmogorov–Smirnov test. A significant mismatch would refute Proposition 1.","tokens_in":11380,"feed_emoji":"🔔","tokens_out":11450,"duration_ms":98888,"temperature":0.7,"pith_summary":"The paper aims to show that the conformal CUSUM procedure can match the classical CUSUM in validity and, under a soft model, approximate its efficiency, while assuming only exchangeability of pre-change observations rather than knowledge of the pre-change distribution. Its central result is that a conformal test martingale using the likelihood ratio as nonconformity measure and a betting function equal to the inverse survival function of the likelihood ratio under $Q_0$ has exactly the same pre-change distribution as the likelihood-ratio martingale. Consequently, false-alarm thresholds calibrated for the classical CUSUM transfer to the conformal procedure. The paper also derives closed-form betting functions for Gaussian mean and variance changes via the Neyman–Pearson ROC curve and proves a Bernoulli-case efficiency bound, with experiments illustrating the behavior.","feed_headline":"Conformal CUSUM inherits classical false-alarm control exactly","feed_subtitle":"The right betting function gives the conformal CUSUM the same pre-change law as the classical one.","key_machinery":"The load-bearing object is the canonical asymptotically optimal (CAO) betting function $f(p)$, defined as the left-continuous inverse of the survival function of the likelihood ratio $L$ under the pre-change law $Q_0$ (equivalently, the upper quantile function of $L$). It is the exact increment map that converts uniform conformal p-values into variables with the law of $L$ under $Q_0$. The argument rests on two identities: conformal p-values are independent and uniform under exchangeability, and the pushforward of uniform by the inverse survival function is the law of $L$. Proposition 2 adds that $f$ is the derivative of the Neyman–Pearson ROC curve, which is how the paper computes $f$ in closed form for $N(0,1)\\to N(\\mu,1)$ and $N(0,1)\\to N(0,\\sigma^2)$.","core_discovery":"The central claim is Proposition 1: for any pair $(Q_0, Q_1)$ of pre- and post-change distributions, the distribution of the likelihood ratio martingale under $Q_0^\\infty$ coincides with the distribution of any asymptotically optimal conformal test martingale (CTM) under randomness. An asymptotically optimal CTM is built by taking the likelihood ratio $L = f_1/f_0$ as the nonconformity measure and betting according to the inverse survival function of $L$ under $Q_0$. Because conformal p-values are independent and uniform under exchangeability, the betting function $f$ maps them into exactly the $Q_0$-law of $L$, so the conformal CUSUM's pre-change behavior is distributionally identical to the standard CUSUM's. Consequently, false-alarm thresholds calibrated for the classical CUSUM carry over to the conformal CUSUM without assuming $Q_0$ or $Q_1$ is known—only exchangeability of the pre-change data is required. The paper also proves that the CAO betting function is the derivative of the Neyman–Pearson ROC curve, giving closed forms for Gaussian location and scale changes, and establishes a Bernoulli-case efficiency bound: for post-change run lengths of order $\\sqrt{N_0}$, the log ratio of the LRM to the CAO CTM stays bounded with high probability.","pith_inferences":["The ROC-derivative characterization suggests the same construction works for any parametric change pair with a known power function (e.g., Poisson or exponential), where the optimal conformal betting function is simply $R'$—a recipe the paper does not spell out for non-Gaussian cases.","Proposition 1's exact validity relies on randomized tie-breaking in conformal p-values; measuring how the non-randomized version degrades in discrete settings such as the Bernoulli 0.1 to 0.9 case would quantify the practical cost of dropping randomization.","A testable extension is to use the CAO betting function with plug-in estimates of $Q_0$ from a short pre-change sample; the paper's soft-model stance suggests validity should degrade gracefully, but no guarantee is proven."],"forward_implications":["The threshold $c$ for the conformal CUSUM can be set using classical CUSUM approximations (e.g., Gaussian ARL formulas) instead of separate conformal simulations, because the pre-change law is identical by Proposition 1.","The same reasoning gives the conformal version of the Shiryaev–Roberts procedure the same perfect validity, as the paper notes.","Closed-form CAO betting functions make the efficient conformal CUSUM directly implementable for Gaussian location and scale changes.","Theorem 5 provides a quantitative guarantee in the Bernoulli case: for post-change run lengths $N_1$ of order $\\sqrt{N_0}$, the log ratio of likelihood-ratio martingale to CAO conformal martingale is bounded with probability $1-\\epsilon$, so the conformal alarm is not much slower after a genuine change."],"supporting_citations":[{"why":"Page's CUSUM procedure, the classical method that the paper conformalizes and whose optimality motivates the efficiency target.","marker":"[9]"},{"why":"Supplies the definition of conformal test martingales, the validity bound (2), and Theorem 11.1 on independence of conformal p-values used in the proof of Proposition 1.","marker":"[15]"},{"why":"Provides the definition of the inverse survival function (Definition A.18) and Lemma A.23 used to equate the distribution of f(p) with the law of L under Q0.","marker":"[2]"},{"why":"Introduced the conformal CUSUM procedure (inductive conformal martingales for change-point detection) that this paper analyzes.","marker":"[14]"},{"why":"The companion work on conformal e-testing; its universal e-values are the experimental baseline, and its Burnaev–Wasserman programme is extended here.","marker":"[16]"},{"why":"Gaussian ARL-to-false-alarm approximations referenced as the classical way to set CUSUM thresholds, which Proposition 1 makes available to the conformal procedure.","marker":"[3]"}],"fun_headline_variants":["Conformal CUSUM preserves classical false-alarm control","Exact CUSUM thresholds carry over to conformal version","Conformal CUSUM inherits classical pre-change distribution","Same false-alarm control, no distribution assumptions","Conformal CUSUM matches classical CUSUM pre-change law"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The asymptotically optimal betting function is defined from the true pre- and post-change distributions (the likelihood ratio and its quantile under $Q_0$), so the full distributional match of Proposition 1 and the efficiency results require those distributions to be specified exactly.","fun_headline_variants_meta":{"raw":{"variants":["Conformal CUSUM preserves classical false-alarm control","Exact CUSUM thresholds carry over to conformal version","Conformal CUSUM inherits classical pre-change distribution","Same false-alarm control, no distribution assumptions","Conformal CUSUM matches classical CUSUM pre-change law"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1358,"prompt_tokens":831,"completion_tokens":527,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":445}},"tokens_in":447,"tokens_out":527,"duration_ms":4932,"temperature":1.0,"reasoning_tokens":445,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:22:41.967620+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate, say, 10,000 pre-change observations from $Q_0 = \\text{Bernoulli}(0.5)$ with post-change $Q_1 = \\text{Bernoulli}(0.6)$, compute conformal p-values with the likelihood-ratio nonconformity measure and the CAO betting function $f(p) = 1.2$ for $p \\le 0.5$, $0.8$ otherwise (using random tie-breaking), and test whether the empirical distribution of $f(p_n)$ equals the $Q_0$-law of $L$ (1.2 with probability 0.5, 0.8 with probability 0.5) by a Kolmogorov–Smirnov test. A significant mismatch would refute Proposition 1.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Page's CUSUM procedure, the classical method that the paper conformalizes and whose optimality motivates the efficiency target."},{"cited_title":"Algorithmic Learning in a Random World","cited_arxiv_id":null,"evidence_quote":"Supplies the definition of conformal test martingales, the validity bound (2), and Theorem 11.1 on independence of conformal p-values used in the proof of Proposition 1."},{"cited_title":"Stochastic Finance: An Introduction in Discrete Time","cited_arxiv_id":null,"evidence_quote":"Provides the definition of the inverse survival function (Definition A.18) and Lemma A.23 used to equate the distribution of f(p) with the law of L under Q0."},{"cited_title":"Inductive conformal martingales for change-point detection","cited_arxiv_id":null,"evidence_quote":"Introduced the conformal CUSUM procedure (inductive conformal martingales for change-point detection) that this paper analyzes."},{"cited_title":"Conformal e-testing","cited_arxiv_id":"2006.02329","evidence_quote":"The companion work on conformal e-testing; its universal e-values are the experimental baseline, and its Burnaev–Wasserman programme is extended here."},{"cited_title":"Goel and S","cited_arxiv_id":null,"evidence_quote":"Gaussian ARL-to-false-alarm approximations referenced as the classical way to set CUSUM thresholds, which Proposition 1 makes available to the conformal procedure."}],"review_version":1}