{"id":"bc2e5556-1753-4d35-8bdd-70735bfb82c2","arxiv_id":"2608.09204","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Artifact type and duration affect uncertainty in PPG-based atrial fibrillation detectors differently, with device displacement harming both and conformal uncertainty flagging reducing false positives by up to 64%.","lead":"This paper measures how four types of motion artifacts in wrist pulse signals change the uncertainty of two machine learning atrial fibrillation detectors. It finds the two detector types fail differently, and that flagging uncertain outputs as such can cut false alarms by up to 64%.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hard-edged artifact insertion and the SQI feature may drive the central Dr-vs-Df duration asymmetry, not detector architecture.","rationale":"Good-faith reading: the paper is a controlled simulation study intended to map artifact type and duration to detector uncertainty. The central empirical pattern is the Df/Dr duration asymmetry. The broad simulation-realness concern raised by the reader is valid, but the more immediately actionable weakness is internal: the simulated artifact insertion creates discontinuities that are not described as physiological. Because Dr is convolutional and Df's signal quality index is duration-correlated by definition, the exact pattern reported—duration-insensitive abrupt drop for Dr and monotonic duration dependence for Df—is what one would expect even if the detectors had no true duration sensitivity. The paper's own Discussion notes that the signal quality index 'may contribute' to Df's gradual error increase, but the main-text interpretation treats the asymmetry as detector-specific. The proposed dual-inference gating assumes the behavior is a stable property; if it is an edge artifact, the recommendation could fail on real signals with smooth artifact transitions. This concern does not question author integrity; it is a modeling-choice confound. The reader's conditional verdict remains appropriate; the concern adds a specific validation condition rather than changing the overall verdict.","tokens_in":14924,"tokens_out":9052,"duration_ms":97364,"concrete_test":"Regenerate the investigation dataset with the same artifact parameters but multiply each artifact segment by a tapered envelope (e.g., a 1-s raised-cosine ramp at onset and offset) before adding it to the PPG, keeping all other settings identical; recompute Figs. 3 and 5. If Dr's sensitivity drop becomes duration-dependent or markedly smaller, the duration-insensitive abrupt drop is caused by hard-edged insertion. A secondary check is to run Df without the signal quality index feature; if its uncertainty no longer grows with artifact duration, the SQI input is the driver.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that Dr drops abruptly but is duration-insensitive while Df degrades progressively with artifact duration—may be an artifact of two construction choices. First, in Sec. 2.2 the simulated artifact is generated as filtered noise, zero-padded outside the artifact interval and added directly to the clean PPG, with each artifact inserted at the same time point. This creates rectangular onset/offset discontinuities. A CNN with 64-sample kernels will see these sharp edges regardless of artifact length, so Dr's 'abrupt, duration-insensitive' response in Fig. 3 and Sec. 3.2 may reflect edge detection rather than sustained artifact content. Second, Df's duration dependence is partly mechanical: its input includes the signal quality index, defined as the proportion of usable signal, which decreases linearly with artifact duration; the paper acknowledges this in Sec. 4. Thus the headline asymmetry could be a property of rectangular insertion plus an explicit duration-correlated feature, not of CNN versus feature-based AF detectors. If real artifacts have gradual onsets/offsets, the recommended gating strategy may not transfer.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper quantifies how four simulated PPG artifact types (device displacement, forearm motion, hand motion, poor contact) and three durations (4, 8, 12 s) affect the uncertainty of two AF detectors: a CNN operating on 25-s raw PPG signals (Dr) and a feed-forward network operating on engineered rhythm-irregularity features plus a signal quality index (Df). The investigation uses 260,000 simulated PPG signals generated with an extended version of the authors' open-access PPG simulator. Uncertainty is assessed with a threshold-based error rate, conformal prediction, and Monte Carlo dropout. The central claim is that Df is more robust to short artifacts but becomes increasingly uncertain as artifact duration grows, whereas Dr exhibits an abrupt performance drop when artifacts occur but is less sensitive to artifact duration; device displacement causes the largest increase in uncertainty. The paper further proposes a dual-detector gating strategy and reports that conformal-prediction-based alert suppression reduces false positives by up to 12% for Dr and up to 64% for Df.","tokens_in":15107,"tokens_out":4286,"duration_ms":55579,"significance":"If the reported detector-specific behavior transfers to real wearable use, the findings are practically valuable because they suggest an artifact-aware detector-selection and uncertainty-based alert-suppression strategy for PPG-based AF screening. The study has clear strengths: it systematically varies artifact type and duration, evaluates two architecturally different detectors, uses three complementary uncertainty measures, and candidly discusses limitations such as the simulated nature of the data, the exchangeability assumption in conformal prediction, and the narrow output range of Dr. It also builds on an openly available simulator and prior validated models, which facilitates reproducibility. However, the significance is tempered by the fact that the investigation dataset is entirely simulated and the artifact model parameters come from the authors' own prior work [13] without independent validation on real artifact-contaminated wrist PPG.","major_comments":[{"comment":"The headline asymmetry between Dr and Df may be a consequence of the artifact insertion procedure rather than detector architecture. In Sec. 2.2, artifacts are generated as filtered noise, zero-padded outside the artifact interval, and added directly to the clean PPG at the same time point, creating rectangular onset and offset discontinuities. A CNN with 64-sample kernels could respond to these sharp edges regardless of artifact length, which would explain Dr's 'abrupt but duration-insensitive' behavior. At the same time, Df uses as an input the signal quality index defined as the proportion of usable signal, a quantity that decreases mechanically with artifact duration, as the Discussion partly acknowledges in Sec. 4. To make the central claim convincing, the authors should run control experiments that separate these effects, for example by applying ramped artifact envelopes, removing the SQI feature from Df, or feeding the SQI to Dr through a late-fusion layer.","section":"Sec. 2.2 and Fig. 3"},{"comment":"All investigation data are generated by an artifact model whose parameters (spectral slope, gamma amplitude parameters, transition probabilities) are taken from the authors' prior work [13] and are not revalidated against independent real artifact-contaminated wrist PPG. Since the reported ranking of artifact types and the duration dependence are the central quantitative findings, the conclusions are only as strong as this model. The paper should either validate the simulated artifacts against an independent real-world dataset or provide a sensitivity analysis showing that the qualitative conclusions are stable under plausible perturbations of the artifact parameters.","section":"Sec. 2.2, Table 1, and Discussion"},{"comment":"The conformal prediction analysis calibrates on artifact-free simulated signals but evaluates on artifact-contaminated signals, so the exchangeability assumption that guarantees coverage is violated. The authors acknowledge this in Sec. 4, but the reported '90% coverage' and the false-positive reduction percentages are still presented as quantitative results. Because the coverage property does not formally hold under this distribution shift, U and the false-positive reductions in Figs. 5-6 are descriptive rather than guaranteed. The authors should either apply a distribution-aware method such as weighted conformal prediction, or explicitly report the empirical coverage on the shifted test set and discuss how the false-positive reduction figures would change.","section":"Sec. 6.2.2 and Figs. 5-6"},{"comment":"The central claims about 'abrupt' versus 'progressive' degradation and about detector-specific differences are based on point estimates without confidence intervals or repeated-seed analyses. Although the dataset is large, the underlying signals are simulated and the detectors have stochastic training procedures; the absence of any error bars makes it difficult to judge whether the reported differences between detectors and between 4 s, 8 s, and 12 s artifacts are larger than run-to-run variability. Reporting confidence intervals, or at least a repeated-seed analysis for a subset of conditions, would materially strengthen the qualitative conclusions.","section":"Figs. 3-6"}],"minor_comments":[{"comment":"The manuscript states that each artifact is inserted at the same time point but does not specify what that time point is; please state the exact insertion time or justify that the choice does not affect the results.","section":"Sec. 2.2"},{"comment":"The figures use shades of gray to denote artifact durations, but the mapping between shade and duration is not always given numerically in the figure or caption; a colorbar or explicit legend would improve readability.","section":"Figs. 2, 3, 5, 7"},{"comment":"The explanation of the non-monotonic behavior of U for Dr at high coverage levels (γ = 0.99) is clear from the text, but Table 2 would benefit from a column showing the output range of each detector, since the restricted range of Dr is the cause of the effect.","section":"Sec. 3.4 and Table 2"},{"comment":"The definition of the signal quality index as 'the proportion of usable signal' depends on the template correlation threshold of 0.7; the authors should state whether the SQI is computed per window or per pulse and how borderline correlations are handled.","section":"Sec. 6.1.2"},{"comment":"Reference [45] in the bibliography lists the publisher as 'Springer, ???'; the missing place or publisher information should be completed.","section":"References"},{"comment":"The sentence 'Dr behaves differently, producing less confident predictions for the non-AF class' is somewhat ambiguous because the preceding discussion concerns the range of outputs rather than confidence; consider rewording to distinguish output range from calibration.","section":"Sec. 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a within-group extension of the authors' own PPG simulator and detector work, and the key conclusions depend on the authors' artifact model [13] without independent validation. The editor may want to ensure that the novelty relative to [13] and [43] is explicitly delineated. The most important required revision is the set of control experiments isolating the SQI feature and the rectangular artifact insertion from the detector-architecture comparison; without those, the central claim could be a property of the simulation setup rather than of the detectors themselves."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful new thing here is the systematic map: four artifact types × three durations × 260,000 simulated signals, with three uncertainty measures, applied to a raw-signal CNN and a feature-based detector. That has not been done before in the PPG-AF literature, and the paper is transparent about its setup and limits. The finding that conformal prediction can cut false positives by up to 64% for the feature detector is practically interesting, and the discussion of why the CNN's restricted output range distorts conformal uncertainty at high coverage is genuinely insightful.\n\nThe soft spots are real but not disqualifying. The whole investigation dataset is simulated using the same group's PPG and artifact simulator, with artifact parameters taken from their own prior work and not revalidated on independent real artifact-contaminated wrist PPG. That is the main limitation, and the paper says so honestly. The stress-test concern also lands. Artifacts are inserted as filtered noise zero-padded outside the interval, giving rectangular onsets/offsets. A CNN with 64-sample kernels will react to those sharp edges regardless of duration, which plausibly explains Dr's abrupt, duration-insensitive drop. And Df's duration dependence is partly mechanical because its input includes a signal quality index that declines linearly with artifact duration — the authors acknowledge this in the Discussion. So the central architectural claim is confounded by construction choices. It may still be true that real, gradual artifacts produce a similar asymmetry, but the current design cannot separate the confounds.\n\nThe conformal calibration is also done on artifact-free signals and applied to artifact-contaminated ones, which violates exchangeability; the authors cite weighted conformal prediction as a fix, so they know. Minor point: Table 2's \"Neither\" column at γ=0.99 for Dr is striking and could use a sentence of interpretation.\n\nAll that said, the paper is well-designed for what it is, clearly written, and unusually candid about its limitations. The citation pattern is appropriate, with self-citations to prior work that the simulator actually builds on. It deserves a serious referee: the question matters for wearable AF screening, and a good referee can push for disentangling the edge/amplitude effects and for validation on real artifact-contaminated signals before the gating strategy is recommended. I would bring it to a reading group to argue about the confounds.","headline":"A genuinely systematic simulation study of how artifact type and duration affect uncertainty in two PPG-based AF detectors, though the headline raw-vs-feature asymmetry may be partly baked into the artifact insertion method and the signal-quality feature.","tokens_in":15654,"tokens_out":1555,"would_cite":true,"duration_ms":65633,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper demonstrates that PPG motion artifacts affect atrial-fibrillation detectors in opposite ways: the feature-based detector is more robust to short artifacts but becomes increasingly uncertain as artifacts lengthen, while the…","keywords":["atrial fibrillation detection","photoplethysmogram","PPG artifacts","uncertainty quantification","conformal prediction","Monte Carlo dropout","wearable sensors","deep learning"],"falsifier":"Run both detectors on real wrist-PPG recordings with synchronized ECG in free-living conditions, annotate artifact type and duration, and compare the sensitivity drops and conformal-prediction uncertainty to the simulated values; the central claim would fail if device displacement is not the worst artifact, or if the feature-based detector does not grow more uncertain with artifact duration while the raw-signal CNN stays roughly flat.","tokens_in":14740,"feed_emoji":"⌚","tokens_out":12417,"duration_ms":107592,"temperature":0.7,"pith_summary":"The paper sets out to establish that motion artifacts in wrist photoplethysmogram (PPG) signals change the uncertainty of atrial fibrillation (AF) detectors in a detector-specific and artifact-specific way. On clean signals the feature-based detector $D_f$ outperforms the raw-signal convolutional network $D_r$ (sensitivity/specificity 0.94/0.94 versus 0.92/0.86), but this ordering depends on what corrupts the signal. Under 12-s device displacement, $D_r$'s sensitivity drops by 0.52 while $D_f$'s drops by 0.31; $D_f$ is steadier for short artifacts yet shows rising uncertainty with longer artifact durations, whereas $D_r$ degrades abruptly as soon as an artifact appears and stays roughly flat in duration. Device displacement is the most damaging artifact type, and conformal prediction with 90% coverage removes up to 12% of false positives for $D_r$ and up to 64% for $D_f$. If these patterns hold in real wearables, artifact-aware detector selection and uncertainty-based alert suppression could make PPG screening for AF less noisy.","feed_headline":"Device displacement is the artifact most damaging to PPG AF detection","feed_subtitle":"A 260,000-signal study shows the right detector to trust depends on artifact type and duration.","key_machinery":"The load-bearing object is a controlled artifact-injection experiment built on a validated PPG simulator. The simulator generates 25-s AF and non-AF waveforms from beat-to-beat interval series, and four artifact types, device displacement, forearm motion, hand motion, and poor contact, are produced by filtering white noise with spectral slopes and RMS amplitudes fitted to real wrist-PPG artifacts and added for 4, 8, or 12 s to 20,000 baseline signals, yielding 260,000 investigation signals. Uncertainty is measured three ways: threshold-based error rate $E$ at thresholds 0.5, 0.7, and 0.9; conformal prediction, which returns a decision set whose size gives the proportion of uncertain decisions $U$ at coverage $\\gamma$; and Monte Carlo dropout, where the detector is run 50 times with dropout active and the average predictive entropy $\\bar{H}$ is computed. These measurements are compared against artifact-free baselines to produce the sensitivity/specificity deltas and false-positive reductions that carry the conclusion.","core_discovery":"The central claim is that uncertainty in PPG-based AF detection is not a single property of the signal or detector but a joint property of artifact type, artifact duration, and detector design. The feature-based detector $D_f$, which uses seven rhythm-irregularity features plus a signal quality index, predicts across the full range $[0,1]$ and reacts to longer artifacts by flattening its prediction distribution and increasing the conformal-prediction uncertainty set. The raw-signal CNN $D_r$ produces outputs in a narrower range, so it loses sensitivity immediately when any artifact is present, most severely for device displacement, where sensitivity falls by 0.52 at 12 s, but its uncertainty barely moves with artifact duration. The paper further shows that the proportion of uncertain decisions under conformal prediction depends strongly on the coverage level and the detector's output range, and that suppressing non-singleton decisions reduces false positives substantially for both detectors. The conclusion is that artifact type and duration should be treated as first-class inputs when choosing an AF detector and deciding whether to alert.","pith_inferences":["Editorial inference: a wearable could estimate artifact duration from accelerometry or signal quality in real time and route each 25-s window to the detector with lower expected uncertainty, rather than running both and arbitrating by confidence alone.","Editorial inference: because the feature-based detector is cheaper and its false positives are reduced far more by conformal suppression, an on-device implementation could lean on it during quiet periods and fall back to the convolutional network when motion is sustained.","Editorial inference: the simulated artifacts omit physiological fluctuations such as respiration-induced baseline drift and vasomotor changes, so a field test with real annotated artifacts might shift the duration crossover point at which the raw-signal CNN becomes the more reliable detector.","Editorial inference: because calibration used artifact-free simulated signals while training used real patient signals, a natural extension is to calibrate conformal prediction on real artifact-contaminated data and measure whether the false-positive reductions change."],"forward_implications":["Device displacement is the artifact that most degrades both detectors, so AF screening pipelines should detect large-amplitude sensor shifts and treat their output as low-confidence rather than making a binary decision.","Discarding predictions that conformal prediction marks as uncertain (two-class or empty sets) cuts false positives by up to 12% for the raw-signal CNN and up to 64% for the feature-based detector at 90% coverage.","For short artifacts the feature-based detector is the safer choice, while for long artifacts the raw-signal CNN becomes comparatively more reliable, supporting a gating strategy that chooses between the two detectors based on estimated artifact duration.","Uncertainty metrics from conformal prediction are not directly comparable across detectors because the raw-signal CNN's narrower output range inflates its uncertain-decision proportion at high coverage; any alert-suppression rule must be calibrated per detector."],"supporting_citations":[{"why":"Supplies the artifact extraction method and the spectral-slope/RMS-amplitude parameter values used to simulate the four artifact types in the investigation dataset.","marker":"[13]"},{"why":"Supplies the validated PPG simulator that produces the AF and non-AF pulse waveforms for all 260,000 investigation signals.","marker":"[15]"},{"why":"Supplies the peak detector and signal quality index that the feature-based detector uses, and the artifact identification step behind [13].","marker":"[6]"},{"why":"Supplies the CNN architecture that defines the raw-signal detector $D_r$.","marker":"[43]"},{"why":"Supplies the seven rhythm-irregularity features that make up the feature-based detector $D_f$.","marker":"[45]"},{"why":"Supplies Monte Carlo dropout as the method for estimating predictive entropy in both detectors.","marker":"[55]"}],"fun_headline_variants":["Device displacement tops PPG artifacts that derail AF detection","Feature-based AF detector more robust to short PPG artifacts","PPG AF uncertainty tied to artifact type, duration, and detector","Conformal prediction cuts false positives for feature-based AF detector","Artifact type and duration dictate which PPG AF detector to trust"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulated artifacts, whose spectral slopes and amplitudes were fitted to wrist-PPG recordings in one prior study, faithfully represent real device displacement, forearm motion, hand motion, and poor contact; if real artifacts differ in amplitude, spectral shape, or timing, the measured detector-specific uncertainty differences may not transfer to wearable use.","fun_headline_variants_meta":{"raw":{"variants":["Device displacement tops PPG artifacts that derail AF detection","Feature-based AF detector more robust to short PPG artifacts","PPG AF uncertainty tied to artifact type, duration, and detector","Conformal prediction cuts false positives for feature-based AF detector","Artifact type and duration dictate which PPG AF detector to trust"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001966,"raw_usage":{"total_tokens":7788,"prompt_tokens":1154,"completion_tokens":6634,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":770,"completion_tokens_details":{"reasoning_tokens":6552}},"tokens_in":770,"tokens_out":6634,"duration_ms":50611,"temperature":1.0,"reasoning_tokens":6552,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:23:24.734856+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run both detectors on real wrist-PPG recordings with synchronized ECG in free-living conditions, annotate artifact type and duration, and compare the sensitivity drops and conformal-prediction uncertainty to the simulated values; the central claim would fail if device displacement is not the worst artifact, or if the feature-based detector does not grow more uncertain with artifact duration while the raw-signal CNN stays roughly flat.","supporting_citations":[{"cited_title":"Springer, ??? (2018)","cited_arxiv_id":null,"evidence_quote":"Supplies the seven rhythm-irregularity features that make up the feature-based detector $D_f$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the artifact extraction method and the spectral-slope/RMS-amplitude parameter values used to simulate the four artifact types in the investigation dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the validated PPG simulator that produces the AF and non-AF pulse waveforms for all 260,000 investigation signals."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the peak detector and signal quality index that the feature-based detector uses, and the artifact identification step behind [13]."},{"cited_title":"IEEE Trans","cited_arxiv_id":null,"evidence_quote":"Supplies the CNN architecture that defines the raw-signal detector $D_r$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Monte Carlo dropout as the method for estimating predictive entropy in both detectors."}],"review_version":1}