Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

ConformalShift: Targeted Event Reordering Against Adaptive ECG Monitoring

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Reordering only the timing of authentic ECG events can suppress ventricular rescue events without altering any waveforms, labels, scores, or event contents.

desk verdict Careful empirical demonstration of a new threat model—bounded reordering of authentic events—with honest methodology, but the practical 'can be compromised' claim rests on an unvalidated network capability. read the letter →

arxiv 2608.03628 v1 pith:J6VV2HER submitted 2026-08-04 cs.LG

Arash Vashagh , Yasmin Vashagh This is my paper · ORCID
classification cs.LG
keywords adversarialmachinelearningconformalpredictionECGmonitoringeventreorderingdelayedfeedbackventricularectopicbeatstemporalintegrityadaptiveinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to prove that adaptive conformal ECG monitors, which keep per-class thresholds updated from delayed clinician feedback, can be attacked without changing any clinical content. An adversary who can reorder a small window of authentic events before a target beat can lower the ventricular threshold enough to drop the ventricular class from the prediction set, suppressing exactly the rescued events the monitor is meant to recover. If true, this means the safety of such monitors depends on the temporal integrity of the feedback stream, not just on classifier robustness or input integrity. The authors demonstrate the attack on two ECG datasets and show that restricting the allowed displacement weakens it, pointing toward a concrete defense direction.

What carries the argument

The central object is the adaptive class-conditional conformal monitor, which maintains per-class thresholds $q_{c,t}$; after a fixed delay $\delta$, each event's missed or covered status updates its class threshold via equation (4), with thresholds decaying toward the initial value $q(0)$ by a recency factor $\rho$. The attack's lever is the feasible permutation set $\Pi_d = \{\pi\in S_W : \max_j |\pi_j-j|\le d\}$, which captures bounded reordering of the $W$ authentic events before the target. The objective is the suppression margin $m(\pi)=a_{t^\ast,v}-q^\pi_{v,t^\ast}$; since the target score is invariant, the search reduces to lowering the ventricular threshold by moving covered ventricular feedback earlier and ventricular misses later. Beam search with local refinement evaluates candidate schedules by exact replay of the monitor and accepts only legal modifications that increase $m(\pi)$.

What would settle it

Run the same ConformalShift configuration on the same MIT–BIH confirmation targets through a monitor that enforces FIFO delivery and authenticates timestamps (or rejects out-of-order packets); if targeted suppression drops to the random-schedule level (4.4% for Extra Trees, 12.0% for HistGradientBoosting) or lower, then the central claim—that timing alone can compromise adaptive monitors—fails for real networks.

Watch

Extended reading notes

Core claim

The paper claims that an adaptive class-conditional conformal monitor with delayed feedback has an order-dependent decision surface, and that a bounded reordering of authentic preceding events can steer the ventricular threshold downward before a target beat is evaluated. Because the target's nonconformity score is fixed, lowering the ventricular threshold is equivalent to increasing the suppression margin $m(\pi)=a_{t^\ast,v}-q^\pi_{v,t^\ast}$; a schedule that makes $m(\pi)>0$ excludes the ventricular class from the prediction set even though the event was rescued under FIFO processing. ConformalShift finds such schedules by beam search and local refinement over the set $\Pi_d$ of permutations in which no event moves more than $d$ positions. On disjoint MIT–BIH confirmation records it suppressed 66.7% of eligible Extra Trees targets and 60.0% of HistGradientBoosting targets, versus 4.4% and 12.0% for random schedules, and locked configurations transferred to INCART at 33.3% success. The paper concludes that adaptive monitors can be compromised through timing alone, with waveforms, labels, classifier outputs, and the event multiset unchanged.

Load-bearing premise

The attack presupposes that a networked attacker can delay or reorder up to 24 (or 16) authentic ECG transmissions within a bounded displacement of 8 (or 4) positions before the target without being detected or breaking protocol; if real monitors enforce FIFO delivery, authenticate timestamps, or reject out-of-order packets, the attack surface does not exist.

Editorial extensions

If this is right

  • Adaptive conformal monitors used in healthcare should be stress-tested for feedback order and delay, not only for input perturbations, because order alone can suppress class recovery.
  • Restricting the maximum displacement of out-of-order delivery weakens the attack: on MIT–BIH Extra Trees, reducing $d$ from 8 to 4 dropped targeted suppression from 66.7% to 16.7%.
  • Attack configurations locked on MIT–BIH transferred to INCART with 33.3% success for both classifiers, indicating the vulnerability is not specific to a single dataset.
  • High aggregate coverage does not imply event-level safety: the monitors maintained coverage above 92% while selected rescued ventricular events were still suppressed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same mechanism would apply to any adaptive system with delayed, class-specific feedback, not just ECG monitoring; the paper demonstrates it on ECG but does not claim generality.
  • A practical defense not evaluated in the paper is timestamp authentication or rejecting stale feedback; if such defenses are deployed, the attack's success should collapse to the random-schedule baseline.
  • Because the attack changes only arrival order, content-based intrusion detection cannot catch it; monitors would need timing-based anomaly detection, which the paper does not address.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces ConformalShift, an attack against an adaptive class-conditional conformal monitor used for ECG beat classification. The attacker is assumed to be able to reorder up to W authentic preceding events within a bounded displacement d, with feedback delayed by δ. The attack searches over feasible permutations to lower the ventricular threshold before a target ventricular beat that was rescued under FIFO processing, so that the ventricular class is excluded from the prediction set. The empirical protocol locks attack configurations on discovery records, evaluates on disjoint confirmation records (6 and 10 eligible targets), reports descriptive full-DS2 results and INCART transfer, and includes a displacement-budget ablation. The authors report targeted suppression rates well above random feasible schedules and conclude that adaptive monitors can be compromised through the timing of authentic information, even when waveforms, labels, classifier outputs, and event contents remain unchanged.

Significance. The paper has real strengths: the evaluation uses exact replay, configurations were frozen before confirmation, confirmation records were disjoint from discovery, Wilson intervals are reported, a random feasible-schedule baseline is included, and a displacement ablation and cross-dataset transfer test are performed. If the replay semantics are correct, the paper identifies a genuinely understudied attack surface for adaptive conformal prediction: feedback-order manipulation without altering any event content. This is relevant to the security evaluation of adaptive monitoring systems and to the design of defenses that preserve temporal integrity. However, the practical reach of the results is limited by an unvalidated network-level reordering capability and by the very small number of eligible confirmation targets.

major comments (3)
  1. [Section II-A, Eq. (3)] The text says the miss indicator e_i is 'fixed when event i is originally evaluated,' which suggests it is computed in the original FIFO trajectory. Under a reordered schedule, however, the same event is evaluated at a different position with a different threshold, so whether it is covered or missed can change. If the replay in Section II-D keeps e_i fixed, the simulated monitor does not actually implement the update rule in Eq. (4), because a reordered covered event should be able to become a miss and vice versa. If e_i is instead recomputed at the event's actual position during replay, the sentence in Section II-A is incorrect, and the statement in Section II-C that 'reordering changes only the ventricular threshold' is not true for the preceding events. The manuscript must state unambiguously how e_i is obtained in the replayed schedule and confirm that the released code follows the same rule; the attack objective in Eq. (6) and the search algorithm both depend on this.
  2. [Section II-B, Eq. (5)] The threat model assumes that a networked attacker can delay or reorder up to W authentic transmissions within a displacement budget d without being detected or violating protocol, but no evidence is provided that clinical telemetry systems permit this. The Introduction itself describes FIFO as the normal mode, and Section IV acknowledges that authenticating timestamps or rejecting stale feedback would neutralize the attack. As written, the conclusion that adaptive monitors 'can be compromised' is stronger than what the experiments establish: they demonstrate sensitivity of the monitor to event order under an assumed reordering capability, not that such reordering is feasible in deployed systems. The paper should either provide a concrete protocol-level reordering scenario or restrict the headline claim to the stated threat model.
  3. [Section III-C, Table II] The independent confirmatory evidence rests on only 6 and 10 eligible targets on the two victims. The 95% Wilson intervals for the targeted suppression rates are 30.0%–90.3% and 31.3%–83.2%, so the point estimates 66.7% and 60.0% are not tightly determined; changing a single outcome moves the rate by 16.7 and 10 percentage points, respectively. Since the full-DS2 results include discovery records and the INCART transfer uses only 15 targets, the confirmatory claim should be accompanied by an explicit discussion of this precision, and ideally by a confidence interval or permutation test for the targeted-versus-random difference.
minor comments (5)
  1. [Abstract and Table II] The rates '66.7%' and '60.0%' in the abstract and results should be presented with their denominators (4/6 and 6/10) so that the small sample size is immediately visible.
  2. [Section III-A] The experimental setup does not state how many confirmation records were used or how the 6 and 10 eligible targets were selected from them; a per-record breakdown would improve transparency.
  3. [Table II] The random baseline is described as the mean over 128 feasible schedules per target, but no dispersion measure or per-target variability is reported; adding a standard deviation or range would help calibrate the reported random rates.
  4. [Section III-A] The phrase 'single-seed results' for the INCART transfer appears only in the text; it should also appear as a footnote to Table II so that the uncertainty is visible to a reader who consults only the table.
  5. [References [5] and [6]] References [5] and [6] are non-archival preprints by the same research group; if the related work relies on them, an archival version or independent source should be cited.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: attack rates are measured on held-out records under frozen configurations, and no equation reduces to its inputs.

full rationale

The derivation chain is self-contained and the central empirical claim is not forced by construction. The attack objective in Section II-C defines the suppression margin as m(pi) = a_{t*,v} - q^pi_{v,t*}, and the search in Section II-D maximizes this margin over feasible permutations from (5) using the explicit threshold update rules (2)-(4). This is an optimization over a defined mechanism, not a fitted parameter renamed as a prediction. The evaluation uses configurations frozen before application to disjoint MIT-BIH confirmation records, as stated in Section III-A: "Configurations were selected on discovery records, frozen before evaluation on disjoint MIT-BIH confirmation records," and the INCART results use those locked configurations without dataset-specific tuning. The reported success rates are therefore measured against data not used to select the attack settings, and the random feasible-schedule baselines provide an independent comparison. The self-citations in the introduction and related work ([1], [2], [5]) are contextual and are not load-bearing for the attack's correctness or evaluation. No uniqueness theorem, ansatz, or prior result is imported to justify the central claim. The Discussion's limitation statement ("This study is limited to a white-box attacker, retrospective ECG datasets, one delayed class-conditional update rule, and relatively small confirmatory target sets") and the unvalidated assumption that real clinical networks permit bounded reordering are scope/external-validity concerns, not circularity. The paper also explicitly labels the full-DS2 results as descriptive rather than independent confirmation, which is the honest treatment of that portion. Overall, no equation or statistical claim reduces to its own input, so the score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the specific adaptive conformal update rule, the assumed network-level reordering capability, and hand-chosen attack and monitor parameters. No new physical or conceptual entities are introduced. The free parameters and domain assumptions are disclosed in the paper, but their influence on the quantitative success rates is not fully isolated.

free parameters (5)
  • Attack window W, displacement d, feedback delay delta (per victim) = ET: W=24,d=8,delta=4; HGB: W=16,d=4,delta=2
    Chosen on discovery records and frozen before confirmation. Success rates and the ablation depend directly on these values.
  • Monitor adaptation parameters (alpha, eta, q0, rho) = 0.1, 0.08, 0.92, 0.995
    Define the adaptive conformal monitor under attack. The attack results are conditional on these settings; no sensitivity analysis is provided.
  • Search hyperparameters (beam width, local-search passes, restarts, neighborhood size) = 256, 10, 24, 256
    Hand-chosen search effort settings that directly affect the success rate; no ablation. More search would likely raise the reported success rates, so the rates are lower bounds.
  • Random baseline evaluation size = 128 feasible schedules per target
    Baseline choice; a larger sample could change the random suppression rate.
  • INCART transfer seeds = model seed 7, attack seed 1007
    Single-seed result; randomness in target selection and schedules affects the 33.3% rate.
assumptions (4)
  • domain assumption The class-conditional adaptive conformal monitor updates per equations (2)-(4)
    The attack targets this specific update rule with delayed feedback; the paper does not test other adaptive conformal algorithms. Section II-A.
  • domain assumption An attacker can reorder up to W preceding events within displacement d without altering content or being detected
    Defines the threat model in Section II-B, Equation (5). The whole attack depends on this capability; feasibility in real clinical networks is not established.
  • domain assumption The frozen classifiers produce enough eligible targets (missed by top-1, rescued by conformal set) for evaluation
    The evaluation selects only such targets; their scarcity (6 and 10 in confirmation) limits statistical power. Section III-A/C.
  • standard math Standard permutation and replay computations are exact
    No additional assumptions; the margin in Equation (6) is computed by exact replay of the deterministic update rules.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ConformalShift: Targeted Event Reordering Against Adaptive ECG Monitoring." pith.science (2026). https://pith.science/paper/J6VV2HER

@misc{pith2026260803628,
  author       = {Pith},
  title        = {Pith review of: ConformalShift: Targeted Event Reordering Against Adaptive ECG Monitoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J6VV2HER}},
  note         = {Machine review of arXiv:2608.03628}
}
read the original abstract

Adaptive conformal prediction can recover clinically important heartbeat classes missed by a point classifier, but delayed feedback makes its decisions sensitive to event order. We introduce ConformalShift, a bounded event-reordering attack that suppresses the ventricular class for rescued events without modifying ECG waveforms, labels, classifier scores, or the event multiset. ConformalShift searches for feasible permutations of authentic preceding events that lower the ventricular threshold before a selected target is evaluated. On disjoint MIT--BIH confirmation records, the attack suppressed 66.7% of eligible targets for Extra Trees and 60.0% for HistGradientBoosting, compared with random-schedule rates of 4.4% and 12.0%, respectively. Transferred configurations also outperformed random scheduling on INCART, while reducing the displacement budget weakened the attack on both datasets. These results show that adaptive monitors in healthcare can be compromised through the timing of authentic information, even when waveforms, labels, classifier outputs, and event contents remain unchanged.

Figures

Figures reproduced from arXiv: 2608.03628 by the authors.

Figure 1
Figure 1. Overview of the ConformalShift threat model. Under ordinary FIFO processing, authentic events are handled in their [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Successful ConformalShift attack on MIT–BIH record 214. Panels (a) and (b) show the original FIFO order and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CohortHijack: Robustness of Single Cell Annotation to Companion Cell Removal

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Small, targeted removal of companion cells can change a single cell's refined annotation without touching the cell itself, revealing query cohort composition as an attack surface in single-cell annotation.

Reference graph

Works this paper leans on

23 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Enhanced atrial fibrillation (af) detection via data augmentation with diffusion model,

    A. Vashagh, A. Akhoondkazemi, S. J. Zahabi, and D. Shafie, “Enhanced atrial fibrillation (af) detection via data augmentation with diffusion model,” in2023 13th International Conference on Computer and Knowl- edge Engineering (ICCKE), 2023, pp. 457–462

    work page 2023

  2. [2]

    Atrial fibrillation (af) detection using deep learning with gan-based data augmentation,

    A. Akhoondkazemi, A. Vashagh, S. J. Zahabi, and D. Shafie, “Atrial fibrillation (af) detection using deep learning with gan-based data augmentation,” in2023 31st International Conference on Electrical Engineering (ICEE), 2023, pp. 955–959

    work page 2023

  3. [3]

    Intriguing properties of neural networks,

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,”arXiv preprint arXiv:1312.6199, 2013

    arXiv 2013

  4. [4]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,”arXiv preprint arXiv:1412.6572, 2014

    arXiv 2014

  5. [5]

    Recent ad- vances in adversarial attacks on model utility, privacy, and explainability: A comprehensive survey,

    A. Vashagh, R. Razavi-Far, M. Meymani, and B. Biggio, “Recent ad- vances in adversarial attacks on model utility, privacy, and explainability: A comprehensive survey,”TechRxiv, vol. 2026, no. 0305, 2026

    work page 2026

  6. [6]

    Defense against adversarial attacks: Foundations, strategies, and future direc- tions,

    M. Meymani, R. Razavi-Far, A. Vashagh, and B. Biggio, “Defense against adversarial attacks: Foundations, strategies, and future direc- tions,”Preprints, 2026

    2026

  7. [7]

    Density estimation helps adversarial robustness,

    A. Hasanebrahimi, B. K. Baghbaderani, R. Hosseini, and A. Kalhor, “Density estimation helps adversarial robustness,” in2023 13th Interna- tional Conference on Computer and Knowledge Engineering (ICCKE). IEEE, 2023, pp. 102–107

    work page 2023

  8. [8]

    Adversarial robustness evaluation with separation index,

    B. K. Baghbaderani, A. Hasanebrahimi, A. Kalhor, and R. Hosseini, “Adversarial robustness evaluation with separation index,” in2023 13th International Conference on Computer and Knowledge Engineering (ICCKE), 2023, pp. 162–167

    work page 2023

Show all 23 references
  1. [9]

    V ovk, A

    V . V ovk, A. Gammerman, and G. Shafer,Algorithmic Learning in a Random World. Springer, 2005, springer, New York

    2005

  2. [10]

    Online conformal predic- tion with decaying step sizes,

    A. N. Angelopoulos, R. Barber, and S. Bates, “Online conformal predic- tion with decaying step sizes,” inProceedings of the 41st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 235. PMLR, 2024, pp. 1616–1630

    2024

  3. [11]

    Adaptive conformal inference under distribu- tion shift,

    I. Gibbs and E. Candes, “Adaptive conformal inference under distribu- tion shift,”Advances in Neural Information Processing Systems, vol. 34, pp. 1660–1672, 2021

    2021

  4. [12]

    Adaptive conformal inference by betting,

    A. Podkopaev, D. Xu, and K.-C. Lee, “Adaptive conformal inference by betting,” inProceedings of the 41st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 235, 2024, pp. 40 886–40 907

    2024

  5. [13]

    Practical adversarial multivalid conformal prediction,

    O. Bastani, V . Gupta, C. Jung, G. Noarov, R. Ramalingam, and A. Roth, “Practical adversarial multivalid conformal prediction,”Advances in neural information processing systems, vol. 35, pp. 29 362–29 373, 2022

    2022

  6. [14]

    The relationship between no-regret learning and online conformal prediction,

    R. Ramalingam, S. Kiyani, and A. Roth, “The relationship between no-regret learning and online conformal prediction,” inForty-second International Conference on Machine Learning, 2025

    2025

  7. [15]

    Stochastic online conformal pre- diction with semi-bandit feedback,

    H. Ge, H. Bastani, and O. Bastani, “Stochastic online conformal pre- diction with semi-bandit feedback,”arXiv preprint arXiv:2405.13268, 2024

    2024 arXiv

  8. [16]

    Online learning with local permutations and delayed feedback,

    O. Shamir and L. Szlak, “Online learning with local permutations and delayed feedback,” inProceedings of the 34th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 70. PMLR, 2017, pp. 3086–3094

    2017

  9. [17]

    The pitfalls and promise of conformal inference under adversarial attacks,

    Z. Liu, Y . Cui, Y . Yan, Y . Xu, X. Ji, X. Liu, and A. B. Chan, “The pitfalls and promise of conformal inference under adversarial attacks,” in Proceedings of the 41st International Conference on Machine Learning, ser. ICML’24, 2024

    2024

  10. [18]

    Provably robust conformal prediction with improved efficiency,

    G. Yan, Y . Romano, and T.-W. Weng, “Provably robust conformal prediction with improved efficiency,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 31 974–32 010

    2024

  11. [19]

    Split conformal prediction under data contamination,

    J. Clarkson, W. Xu, M. Cucuringu, Y . Swan, and G. Reinert, “Split conformal prediction under data contamination,”arXiv preprint arXiv:2407.07700, 2024

    2024

  12. [20]

    Robust conformal prediction using privi- leged information,

    S. Feldman and Y . Romano, “Robust conformal prediction using privi- leged information,”Advances in Neural Information Processing Systems, vol. 37, pp. 117 813–117 852, 2024

    2024

  13. [21]

    Adversarial examples for electrocardiograms,

    X. Han, Y . Hu, L. Foschini, L. Chinitz, L. Jankelson, and R. Ran- ganath, “Adversarial examples for electrocardiograms,”arXiv preprint arXiv:1905.05163, 2019

    1905 arXiv

  14. [22]

    Application of adversarial examples to physical ecg signals,

    T. Ono, T. Sugawara, J. Sakuma, and T. Mori, “Application of adversarial examples to physical ecg signals,”arXiv preprint arXiv:2108.08972, 2021

    2021 arXiv

  15. [23]

    Robustness and visual explanation for black box image, video, and ecg signal classification with reinforcement learning,

    S. Sarkar, A. R. Babu, S. Mousavi, V . Gundecha, A. Naug, and S. Ghorbanpour, “Robustness and visual explanation for black box image, video, and ecg signal classification with reinforcement learning,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.