Pith. sign in

REVIEW 2 major objections 6 minor 118 references

Good Practice Guide for quantifying uncertainties for machine learning models applied to photoplethysmography signals

T0 review · 2 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This guide establishes that a machine learning prediction from a PPG signal is incomplete without a quantified, validated uncertainty, and gives a practical recipe for producing and checking one.

desk verdict A useful, honest synthesis of UQ practice for PPG-ML; the central claims hold up, but fix the mixture-of-Gaussians paragraph and don't oversell the empirical support. read the letter →

arxiv 2607.19999 v1 pith:XBM4DPRN submitted 2026-07-22 cs.LG

P. Harris , C. Bench , M. Rinkevičius , V. Marozas , L. Coquelin , A. Thompson , M. Nandi , U. Hackstein
show 1 more author
P.J. Aston
This is my paper
classification cs.LG
keywords uncertaintyquantificationphotoplethysmographymachinelearningmodelcalibrationconformalpredictionwearabledevicesbenchmarkdatasetsvalidationmetrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is a metrology-style good practice guide for anyone building or using machine learning models on photoplethysmography (PPG) signals from wearables. It claims that a model output is not a complete measurement result unless it comes with a quantitative statement of its own quality, and that such an uncertainty statement can be made reliable through a combination of uncertainty quantification methods, validation metrics, and benchmark datasets. The guide lays out three modelling routes—deep learning on raw signals, deep learning on image representations, and feature-based machine learning—together with model-dependent and model-independent ways to attach uncertainties to predictions. Its central recommendation is that uncertainty reliability must be validated with local and global calibration metrics, and that this is now practically achievable thanks to software and benchmarks released with the guide.

What carries the argument

The central object is the validated uncertainty statement: a predictive distribution for regression, or a probability mass function for classification, which is checked against observed errors using calibration metrics. Two mechanisms do the heavy lifting: post-hoc calibration methods (binning, isotonic regression, temperature scaling) and conformal prediction, which from a calibration set produces prediction intervals or sets with a user-specified coverage guarantee that is distribution-free. Around these, the guide organises a validation workflow—split data into training, validation, calibration and test sets; choose an uncertainty expression; compute local/global reliability metrics; insp

What would settle it

Take a benchmark such as AF detection and replace chart-derived or history-based labels with expert PPG/ECG-annotated labels for the same windows; if the guide's recommended calibration metrics report well-calibrated uncertainties on the original labels but systematically worse coverage or reliability on the expert-labelled subset, that would show the framework's label-reliability assumption is violated.

Watch

Extended reading notes

Core claim

The paper's core claim is that a prediction, by itself, is not enough: a PPG-based diagnostic decision is only as good as the reliability of the uncertainty attached to the prediction. The guide argues that uncertainty can be quantified either through model-dependent methods (maximum likelihood and maximum a posteriori estimation, deep ensembles, Monte Carlo dropout) or model-independent methods (calibration, temperature scaling, conformal prediction), and that in both cases the results must be validated using calibration metrics such as ECE, ACE, smECE, UCE, VCE and NLL for classification and ENCE, PICP, CCE, CRPS and MPIW for regression. Its contribution is to assemble this toolkit into a

Load-bearing premise

The load-bearing premise is that the reference labels used to validate uncertainty are reliable enough to serve as ground truth; the guide itself flags that some DeepBeat AF labels are based on patient history rather than active AF in the window, and that MIMIC rhythm annotations were inferred from chart events rather than expert PPG labelling.

Editorial extensions

If this is right

  • If followed, wearable device manufacturers can ship models whose predictions carry coverage intervals with known reliability, not just point estimates.
  • Clinicians can decide whether a single measurement is good enough to act on, because the stated uncertainty has been checked against observed errors rather than assumed.
  • The six benchmark problems and accompanying datasets let different models and uncertainty quantification methods be compared on common ground, making published accuracy claims harder to inflate.
  • The software implementations give practitioners a low-cost way to adopt the recommended methods.
  • The guide's distinction between global, local and individual reliability shifts evaluation practice toward what matters in real use: how reliable is this particular prediction?

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit consequence: because the framework judges uncertainty by calibration against reference labels, its recommendations inherit any label noise; the guide itself flags that some AF labels are based on patient history rather than active AF in the window, and that some rhythm annotations were inferred from chart events rather than expert PPG labelling.
  • The paper leaves open the development of quantitative individual-reliability metrics; a natural extension is to adapt per-prediction calibration tests to this framework.
  • The same validation workflow could transfer to other physiological signals or sensor modalities, provided the benchmark labels are built with the same care; a testable extension would be to run the recommended metrics on a modality where ground truth is unambiguous.
  • The recommendation that modern CNNs on raw time series are the safest default is based on two benchmark tasks; if transferred to the other four benchmark problems, the ranking may shift, especially where interpretable features are clinically validated.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This manuscript is a project deliverable (Good Practice Guide) from the QUMPHY consortium, aimed at practitioners and clinicians who use machine learning on photoplethysmography (PPG) signals. It reviews three ML modelling approaches (raw-signal deep learning, image-representation deep learning, and feature-based ML), surveys model-dependent and model-independent uncertainty quantification methods, discusses validation metrics for classification and regression, lists six benchmark problems with associated datasets, describes the accompanying open-source software, and includes a chapter on ethics. The guide does not present new benchmark experiments itself; it synthesizes results from companion papers [2] and [3] and from project reports. Its central assertion is that any prediction should be accompanied by a reliable uncertainty statement, and it aims to provide a practical framework for producing and validating such uncertainties.

Significance. If the guidance is followed, this document could serve as a useful specification for developers and regulators in the wearable-PPG domain. The guide is commendably explicit about many limitations, including label noise in benchmark datasets and the absence of robust individual-reliability metrics. It ships with publicly accessible software repositories and provides a structured comparison of UQ methods and metrics. The ethical discussion, while standard, is integrated into the project workflow rather than added as an afterthought. The main scientific value is in the synthesis and the identification of open problems, not in new empirical results.

major comments (2)
  1. [§3.2.1, Eqs. (5)–(6)] The text states that a uniformly-weighted mixture of Gaussian distributions is 'also Gaussian' with mean and variance given by the displayed formulas. This is mathematically incorrect: the mixture of Gaussians is a Gaussian mixture, not a Gaussian, unless all component means are equal. The mean and variance formulas are correct for the mixture, but the distribution itself is not Gaussian. This matters because the guide later treats the ensemble predictive distribution as Gaussian for constructing coverage intervals and for evaluating Gaussian NLL. Please correct the statement and indicate how coverage intervals and NLL should be computed for the mixture (or explicitly justify a Gaussian approximation). The phrase 'assuming independence between the distributions from the individual models' is also inaccurate for the marginal mixture distribution and should be removed or qualified.
  2. [§9.3, §4.1] The guide's own summary reports that, in the calibration-free setting (training and testing on disjoint subjects, which is the realistic wearable scenario), 'model-independent calibration methods for UQ did not generally improve uncertainty reliability according to the chosen metrics,' and that bivariate histograms 'revealed poor reliability for all models and uncertainty quantification techniques, where the worst reliability was exhibited in the “calibration-free” setting.' At the same time, §4.1 argues that individual reliability is what matters when a single prediction informs a clinical decision, and §9.3 concedes that 'there is a lack of robust quantitative metrics to assess individual reliability.' This tension undercuts the framing that reliable, validated UQ is attainable by following the recommended combination. The authors should either temper the central claim or provide a mor
minor comments (6)
  1. [§3.2.1] The sentence 'assuming independence between the distributions from the individual models' is conceptually misleading: the mixture distribution is a convex combination, not an independent sum. Please rephrase.
  2. [§3.3.1.1] For the regression version of histogram binning, the interval [b0, bM] for the variance is not defined. State how the lower and upper bounds are chosen in practice.
  3. [§4.3, Eq. (17)] The ENCE definition uses an unweighted average over bins. Many published definitions weight by bin frequency. Please clarify the convention used and justify the choice.
  4. [§5.2] DeepBeat is described as 'preferred due to its large scale ... despite some label noise,' and later as having AF labels based on patient history rather than active AF within the signal window. Please make the nature of the label noise explicit at the first mention to avoid confusion.
  5. [Table 1] The checkmark notation (e.g., '✓/✓' and '✗/✓') in the R/C and E/A columns is difficult to parse. Add a footnote explaining each pair.
  6. [References] Some references are to arXiv preprints and project deliverables that may not be permanently available. Consider adding DOIs where possible.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-citation in the validation-metric discussion; no construction-level circularity in the guide's derivations.

  1. self citation load bearing [Section 4.2, discussion of UCE versus VCE, references [93] and [94]]
    "While some justification is given in [93] for the use of UCE, it is not clear why the error E(Bm) and uncertainty U(Bm) as defined above are expected to be equal for well-calibrated uncertainties. In [94] ... It is shown that the uncertainties are well-calibrated according to both ECE and VCE, but poorly calibrated according to UCE."

    The guide's validation framework prefers VCE over UCE based on numerical evidence that is imported from reference [94], a same-consortium arXiv preprint (Thompson) rather than derived in the guide or independently machine-checked/reproduced here. The recommendation therefore rests on an unverified self-citation chain: the guide endorses VCE because the authors' own prior work reports that VCE behaves better than UCE, without an independent reduction from the guide's stated assumptions.

full rationale

The paper is a good-practice synthesis rather than a derivation of new predictions from fitted parameters. No step was found in which a quantity is defined in terms of the very quantity it is used to predict, and no fitted input is renamed as a prediction. The main load-bearing methodological content — descriptions of UQ methods, calibration metrics, conformal prediction, and benchmark datasets — is supported by standard external references or by companion project papers [2] and [3] that contain the actual benchmarking experiments; citing those project results is normal reporting, not circularity. The only notable self-citation issue is in Section 4.2, where the choice of VCE over UCE is justified by a same-author preprint [94] that is not independently verified in the guide; this is a minor, local reliance on self-citation rather than a construction-level circularity. The guide's own Section 9.3 admits that reliability is poor in the calibration-free setting and that individual-reliability metrics are lacking; this is an internal consistency/attainability concern, not a circularity reduction. Accordingly, the overall circularity score is 2.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The guide introduces no fitted free parameters and no invented physical or mathematical entities. Its central claims rest on standard statistical assumptions, on the reliability of external benchmark labels, and on standard signal-processing and ML background. The most fragile assumption is label reliability, which the authors themselves partially acknowledge.

assumptions (3)
  • domain assumption Split conformal prediction coverage holds when calibration and test data are exchangeable.
    The coverage guarantee in §3.3.2 depends on exchangeability. The guide states the assumption but does not verify it for the PPG benchmark tasks, several of which involve repeated measurements from the same subjects.
  • domain assumption Regression output distributions used by MLE, MAP, variance scaling, and related methods are adequately modelled as Gaussian.
    §3.2.1 and §3.3.1.2 assume y ∼ N(μ(x,θ), σ²(x,θ)). If the conditional distribution is substantially non-Gaussian, the derived variances and intervals may be misleading unless corrected by conformal or calibration procedures.
  • domain assumption Benchmark dataset labels are of sufficient quality to validate uncertainty estimates.
    The guide itself notes in §5.2 that DeepBeat AF labels are based on patient history rather than active AF within windows, and that MIMIC-III-Ext-PPG rhythm annotations were inferred from chart events. Noisy labels directly affect the correctness of calibration metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Good Practice Guide for quantifying uncertainties for machine learning models applied to photoplethysmography signals." pith.science (2026). https://pith.science/paper/XBM4DPRN

@misc{pith2026260719999,
  author       = {Pith},
  title        = {Pith review of: Good Practice Guide for quantifying uncertainties for machine learning models applied to photoplethysmography signals},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XBM4DPRN}},
  note         = {Machine review of arXiv:2607.19999}
}
read the original abstract

This Good Practice Guide presents work done in the QUMPHY project (Uncertainty quantification for machine learning models applied to photoplethysmography signals) that considered both machine learning and uncertainty quantification for problems which used photoplethysmography (PPG) signals from wearable devices as input. It provides high-level guidance on what types of machine learning model might be used and how different models compare when applied to both regression and classification tasks. It provides guidance on the implementation of different methods for uncertainty quantification, covering both model-dependent and model-independent techniques, and on the validation of the results provided by those methods. It also describes six benchmark problems together with pointers to different benchmark datasets for each problem. Software is described that can assist practitioners in implementing the methods described herein and there is a brief consideration of ethical issues. It concludes with a summary and recommendations.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

118 extracted references · 1 canonical work pages

  1. [2]

    Moulaeifard, M.; Coquelin, L.; Rinkevičius, M. et al. Machine-learning for photoplethysmography analysis: Benchmarking feature, image, and signal-based approaches. Biomed. Sig. Proc. Ctrl. 120:109831, 2026

    2026

  2. [3]

    Bench, C.; Pfeffer, O.; Desai, V. et al. A systematic evaluation of uncertainty quantification techniques in deep learning: a case study in photoplethysmography signal analysis. Mach. Learn. Hlth. 2(1): 015011, 2026. 22HLT01 QUMPHY 69 Page 61 of 69

    2026

  3. [4]

    https://www.qumphy.ptb.de/publications

    QUMPHY D3 report, 2026. https://www.qumphy.ptb.de/publications

    2026

  4. [1]

    Acquiring wearable photoplethysmography data in daily life: The PPG diary pilot study

    Charlton, P.H.; Kyriacou, P.; Mant, J.; Alastruey, J. Acquiring wearable photoplethysmography data in daily life: The PPG diary pilot study. Eng. Proc. 2(1):80, 2020

    2020

  5. [5]

    Hackstein, U.; Alastruey, J.; Aston, P.J. et al. Benchmark problems and benchmark datasets for the evaluation of machine and deep learning methods on photoplethysmography signals: The D4 report from the QUMPHY project. arXiv 2604.01398, 2026

    arXiv 2026

  6. [6]

    Solosenko, A.; Marozas, V

    Paliakaite, B.; Petrenas, A. ; Solosenko, A.; Marozas, V. Modeling of artifacts in the wrist photoplethysmogram: Application to the detection of life-threatening arrhythmias. Biomed. Sig. Proc. Ctrl. 66:102421, 2021

    2021

  7. [7]

    McEvoy, J.W.; McCarthy, C.P.; Bruno, R.M. et al. ESC Scientific Document Group, 2024 ESC Guidelines for the management of elevated blood pressure and hypertension: Developed by the task force on the management of elevated blood pressure and hypertension of the European Society of Cardiology (ESC) and endorsed by the European Society of Endocrinology (ESE)...

    2024

  8. [8]

    Krijthe, B.P.; Kunst, A.; Benjamin, E.J. et al. Projections on the number of individuals with atrial fibrillation in the European Union, from 2000 to 2060. Eur. Heart J. 34(35):2746-2751, 2013

    2000

Show all 118 references
  1. [9]

    Van Gelder, I.C.; Rienstra, M.; Bunting, K.V. et al. 24 ESC Guidelines for the management of atrial fibrillation developed in collaboration with the European Association for Cardio-Thoracic Surgery (EACTS). Eur. Heart J. 45(36):3314-3414, 2024

    2024

  2. [10]

    Petrenas, A.; Paliakaite, B.; Sornmo, L.; Marozas, V

    Solosenko, A. ; Petrenas, A.; Paliakaite, B.; Sornmo, L.; Marozas, V . Detection of atrial fibrillation using a wrist-worn device. Phys. Meas. 40(2):025003, 2019

    2019

  3. [11]

    Low-complexity detection of atrial fibrillation in continuous long-term monitoring

    Petrenas, A.; Marozas, V.; Sornmo, L . Low-complexity detection of atrial fibrillation in continuous long-term monitoring. Comp. Biol. Med. 65:184-191, 2015

    2015

  4. [12]

    Screening for atrial fibrillation: A call for evidence

    Jones, N.R.; Taylor, C.J.; Hobbs, F.R.; Bowman, L.; Casadei, B. Screening for atrial fibrillation: A call for evidence. Eur. Heart J. 41(10):1075-1085, 2020

    2020

  5. [13]

    Climie, R.E.; Alastruey, J.; Mayer, C.C. et al. Vascular ageing: moving from bench towards bedside. Eur. J. Prev. Cardiol. 30(11):1101-1117, 2023

    2023

  6. [14]

    Oscillometric assessment of arterial stiffness in everyday clinical practice

    Reshetnik, A.; Gohlisch, C.; Tolle, M.; Zidek, W.; van der Giet, M. Oscillometric assessment of arterial stiffness in everyday clinical practice. Hyperten. Res. 40(2):140-145, 2017

    2017

  7. [15]

    Biological versus chronological aging: JACC focus seminar

    Hamczyk, M.R.; Nevado, R.M.; Barettino, A.; Fuster, V.; Andres, V . Biological versus chronological aging: JACC focus seminar. J. Amer. Coll. Cardiol. 75(8):919-930, 2020

    2020

  8. [16]

    Abbasi, A.; Gupta, S.S.; Sabharwal, N. et al. A comprehensive review of obstructive sleep apnea. Sleep Science 14(2):142-154, 2021. 22HLT01 QUMPHY 69 Page 62 of 69

    2021

  9. [17]

    Kapur, V.; Auckley, D.; Chowdhuri, S. et al. Clinical practice guideline for diagnostic testing for adult obstructive sleep apnea: An American Academy of Sleep Medicine Clinical Practice Guideline. J. Cl. Sleep Med. 13:479-504, 2017

    2017

  10. [18]

    Yeghiazarians, Y.; Jneid, H.; Tietjens, J.R. et al. Obstructive sleep apnea and cardiovascular disease: A scientific statement from the American Heart Association . Circ. 144(3):e56-e67, 2021

    2021

  11. [19]

    Bubu, O.M.; Andrade, A.G.; Umasabor-Bubu, O.Q. et al. Obstructive sleep apnea, cognition and Alzheimer’s disease: A systematic review integrating three decades of multidisciplinary research. Sleep Med. Rev. 50:101250, 2020

    2020

  12. [20]

    Sleep disorders, medical conditions, and road accident risk

    Smolensky, M.H.; Milia, L.D.; Ohayon, M.M.; Philip, P. Sleep disorders, medical conditions, and road accident risk. Acc. Anal. Prev. 43(2):533-548, 2011

    2011

  13. [21]

    The economic burden of (obstructive) sleep apnea

    zur Nieden, P.B. The economic burden of (obstructive) sleep apnea. Costs and implications for Germany based on the results of an international systematic review. J. Pub. Health. 34:409- 427, 2024

    2024

  14. [22]

    Clinician-focused overview and developments in polysomnography

    Markun, L.C.; Sampat, A. Clinician-focused overview and developments in polysomnography. Curr. Sleep. Med. Rep. 6:309-321, 2020

    2020

  15. [23]

    Clinical use of a home sleep apnea test: An American Academy of Sleep Medicine position statement

    Rosen, I.; Kirsch, D.; Chervin, R. Clinical use of a home sleep apnea test: An American Academy of Sleep Medicine position statement. J. Clin. Sleep. Med. 13:1205-1207, 2017

    2017

  16. [24]

    STOP-Bang and NoSAS questionnaires as a screening tool for OSA: Which one is the best choice? Rev

    Costa, J.; Rebelo-Marques, A.; Machado, J. STOP-Bang and NoSAS questionnaires as a screening tool for OSA: Which one is the best choice? Rev. Assoc. Med. Bras. 66(9):1203- 1209, 2020

    2020

  17. [25]

    Screening commercial vehicle drivers for obstructive sleep apnea: Tools, barriers, and recommendations

    Evans, K.A.; Yap, T.; Turner, B. Screening commercial vehicle drivers for obstructive sleep apnea: Tools, barriers, and recommendations. Work. H&S. 65(1):487-492, 2017

    2017

  18. [26]

    Respiratory rate and pattern

    Braum, S.R. Respiratory rate and pattern. In Clinical Methods: The History, Physical, and Laboratory Examinations . Ed s H. Walker, W. Hall, and J. Hurst. Third edition. Boston: Butterworths, Chapter 43, 1990

    1990

  19. [27]

    Charlton, P.H.; Bonnici, T.; Tarassenko, L. et al. An assessment of algorithms to estimate respiratory rate from the electrocardiogram and photoplethysmogram. Phys. Meas. 37(4):610, 2016

    2016

  20. [28]

    Clinical antecedents to in- hospital cardiopulmonary arrest

    Schein, R.M.H.; Hazday, N.; Pena, M.; Ruben, B.H.; Sprung, C.L. Clinical antecedents to in- hospital cardiopulmonary arrest. Chest 98(6):1388–1392, 1990

    1990

  21. [29]

    Physiological values and procedures in the 24 h before ICU admission from the ward

    Goldhill, D.; White, S.; Sumner, A. Physiological values and procedures in the 24 h before ICU admission from the ward. Anaesthesia 54(6):529–534, 1999. 22HLT01 QUMPHY 69 Page 63 of 69

    1999

  22. [30]

    Cretikos, M.; Bellomo, R.; Hillman, K. et al. A. Respiratory rate: The neglected vital sign. Med. J. Aust. 188(11):657–659, 2008

    2008

  23. [31]

    The recognition and early management of critical illness

    Ridley, S. The recognition and early management of critical illness. Ann. R. Coll. Surg. Engl. 87(5):315–322, 2005

    2005

  24. [32]

    ECG-derived respiratory frequency estimation

    Bailon, R.; Sornmo, L.; Laguna, P. ECG-derived respiratory frequency estimation. In Advanced Methods and Tools for ECG Data Analysis. London: Artech Houes, Chapter 8, 215–44, 2006

    2006

  25. [33]

    Meredith, D.; Clifton, D.; Charlton, P.H. et al. Photoplethysmographic derivation of respiratory rate: A review of relevant physiology. J. Med. Eng. Technol. 36(1):1–7, 2012

    2012

  26. [34]

    Aston, P.J.; Adel, T.; Bench, C. et al. Machine learning tools in metrology. In Mathematical and Computational Approaches in Measurement, in preparation

  27. [35]

    Explaining deep learning for ECG analysis: Building blocks for auditing and knowledge discovery

    Wagner, P.; Mehari, T.; Haverkamp, W.; Strodthoff, N. Explaining deep learning for ECG analysis: Building blocks for auditing and knowledge discovery. Comput. Biol. Med. 176:108525, 2024

    2024

  28. [36]

    Fawaz, H.I.; Lucas, B.; Forestier, G. et al . Inceptiontime: Finding Alexnet for time series classification. Data Min. Know. Disc. 34(6), 1936–62, 2020

    1936

  29. [37]

    Deep learning for ECG analysis: Benchmarks and insights from PTB-XL

    Strodthoff, N.; Wagner, P.; Schaeffter, T.; Samek, W. Deep learning for ECG analysis: Benchmarks and insights from PTB-XL. IEEE J. Biomed. Health Inform. 25(5):1519–1528, 2020

    2020

  30. [38]

    Imagenet classification with deep convolutional neural networks

    Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process Syst. 25, 2012

    2012

  31. [39]

    Minirocket: A very fast (almost) deterministic transform for time series classification

    Dempster, A.; Schmidt, D.F.; Webb, G.I. Minirocket: A very fast (almost) deterministic transform for time series classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 248–57, 2021

    2021

  32. [40]

    An empirical evaluation of generic convolutional and recurrent networks for sequence modeling

    Bai, S.; Kolter, J.Z.; Koltun, V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv 1803.01271, 2018

    2018 arXiv

  33. [41]

    Deep residual learning for image recognition

    He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. arXiv 1512.03385, 2015

    2015 arXiv

  34. [42]

    A Wavelet Tour of Signal Processing

    Mallat, S. A Wavelet Tour of Signal Processing. 3rd ed. New York: Academic Press, 2009

    2009

  35. [43]

    Beyond HRV: Attractor reconstruction using the entire cardiovascular waveform data for novel feature extraction

    Aston, P.J.; Christie, M.I.; Huang, Y.H.; Nandi, M. Beyond HRV: Attractor reconstruction using the entire cardiovascular waveform data for novel feature extraction. Phys. Meas. 39(2):024001, 2018

    2018

  36. [44]

    A novel method to quantify arterial pulse waveform morphology: Attractor reconstruction for physiologists and clinicians

    Nandi, M.; Venton, J.; Aston, P.J. A novel method to quantify arterial pulse waveform morphology: Attractor reconstruction for physiologists and clinicians. Phys. Meas. 39:104008, 2018. 22HLT01 QUMPHY 69 Page 64 of 69

    2018

  37. [45]

    Symmetric Projection Attractor Reconstruction: Embedding in higher dimensions

    Lyle, J.V.; Aston, P.J. Symmetric Projection Attractor Reconstruction: Embedding in higher dimensions. Chaos 31:113135, 2021

    2021

  38. [46]

    Vardanega, S.; Segers, P.; Aston, P.J. et al. Attractor image-based deep learning of arterial pulse waves for age classification. Comp. Cardiol. 52:343, 2025

    2025

  39. [47]

    Mathieu, A.J.W.; Pascual, M.S.; Charlton, P.H. et al. Advanced waveform analysis of the photoplethysmogram signal using complementary signal processing techniques for the extraction of biomarkers of cardiovascular function. JRSM Cardiovasc . Dis. 13:20480040231225384, 2024

    2024

  40. [48]

    Assessing mental stress from the photoplethysmogram: A numerical study

    Charlton, P.H.; Celka, P.; Farukh, B.; Chowienczyk, P.; Alastruey, J. Assessing mental stress from the photoplethysmogram: A numerical study. Phys. Meas. 39(5):054001, 2018

    2018

  41. [49]

    Finnegan, E.; Davidson, S.; Harford, M. et al. Features from the photoplethysmogram and the electrocardiogram for estimating changes in blood pressure. Scientific Reports 13(1), 986, 2023

    2023

  42. [50]

    Kontaxis, S.; Gil, E.; Marozas, V. et al. Photoplethysmographic waveform analysis for autonomic reactivity assessment in depression. IEEE Trans. Biomed. Engng . 68(4):1273– 1281, 2021

    2021

  43. [51]

    Atrial fibrillation

    Burns, E; Buttner, R. Atrial fibrillation. Life in the Fast Lane, ECG Library. https://litfl.com/atrial- fibrillation-ecg-library/ Accessed 23/02/2026

    2026

  44. [52]

    Extraction of f waves

    Sörnmo, L.; Petrėnas, A.; Laguna, P,; Marozas, V. Extraction of f waves. In Atrial Fibrillation from an Engineering Perspective . L. Sörnmo (Ed.). Berlin: Springer, 137-220, 2018

    2018

  45. [53]

    Detection of atrial fibrillation using a wrist-worn device

    Sološenko, A.; Petrėnas, A.; Paliakaitė, B.; Sörnmo, L.; Marozas, V. Detection of atrial fibrillation using a wrist-worn device. Phys. Meas. 40(2):025003, 2019

    2019

  46. [54]

    A practical guide to wavelets for metrology

    Lord, G.J.; Pardo-Iguzquiza, E.; Smith, I.M. A practical guide to wavelets for metrology. NPL Report CMSC 02/00, 2000

    2000

  47. [55]

    Photoplethysmogram analysis and applications: An integrative review

    Park, J.; Seok, H.S.; Kim S.S.; Shin, H. Photoplethysmogram analysis and applications: An integrative review. Front. Physiol. 12:808451, 2022

    2022

  48. [56]

    Charlton, P.H.; Paliakaitė, B.; Pilt, K. et al. Assessing hemodynamics from the photoplethysmogram to gain insights into vascular age: a review from VascAgeNet . Am. J. Physiol.-Heart Circul. Physiol. 322(4):H493–H522, 2022

    2022

  49. [57]

    Feature dimensionality reduction: A review

    Jia, W.; Sun, M.; Lian, J.; Hou, S. Feature dimensionality reduction: A review. Compl. Intell. Syst. 8:2663-93, 2022

    2022

  50. [58]

    Maximum relevance and minimum redundancy feature selection methods for a marketing machine learning platform

    Zhenyu, Z.; Anand, R.; Wang, M. Maximum relevance and minimum redundancy feature selection methods for a marketing machine learning platform . In Proceedings of the IEEE International Conference on Data Science and Advanced Analytics, 442–452, 2019. 22HLT01 QUMPHY 69 Page 65 of 69

    2019

  51. [59]

    PPFS: Predictive permutation feature selection

    Hassan, A.; Paik, J.H.; Khare, S.; Hassan, S.A . PPFS: Predictive permutation feature selection. arXiv:2110.10713v1, 2021

    2021 arXiv

  52. [60]

    Regression shrinkage and selection via the lasso

    Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B Methodol. 58(1):267–288, 1996

    1996

  53. [61]

    Gewers, F.L.; Ferreira, G.R.; De Arruda, H.F. et al. Principal Component Analysis: A natural approach to data exploration. ACM Comp. Sur. (CSUR) 54(4):70, 2021

    2021

  54. [62]

    Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture

    Jutten, C; Herault, J. Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture. Sig. Proc. 24:1–10, 1991

    1991

  55. [63]

    Autoencoder and its various variants

    Zhai, J.; Zhang, S.; Chen, J.; He, Q. Autoencoder and its various variants. Proc. 2018 IEEE Int. Conf. Sys., Man, Cyber. (SMC), 415-419, 2018

    2018

  56. [64]

    Charlton, P.H

    Rinkevičius, M. ; Charlton, P.H. ; Marozas, V. Uncertainty in photoplethysmography-based cuffless blood pressure trend monitoring: A personalized approach. Comput. Cardiol. 51:098, 2024

    2024

  57. [65]

    Cisnal, A. ; Li, Y. ; Fuchs, B . et al . Robust feature selection for BP estimation in multiple populations: Towards cuffless ambulatory BP monitoring. IEEE J. Biomed. Health Inform. 28:5768-79, 2024

    2024

  58. [66]

    A unified approach to interpreting model predictions

    Lundberg, S.M.; Lee, S. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), 30, 2017

    2017

  59. [67]

    Why should I trust you?: Explaining the predictions of any classifier

    Ribeiro, M.T.; Singh, S.; Guestrin, C. Why should I trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144, 2016

    2016

  60. [68]

    The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition

    Hastie, T .; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition. Springer Series in Statistics , Springer New York, 2009

    2009

  61. [69]

    International Vocabulary of Metrology — Basic and general concepts and associated terms (VIM) (3rd edition), Joint Committee for Guides in Metrology, JCGM 200, 2012

    BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML. International Vocabulary of Metrology — Basic and general concepts and associated terms (VIM) (3rd edition), Joint Committee for Guides in Metrology, JCGM 200, 2012

    2012

  62. [70]

    Evaluation of measurement data — Guide to the expression of uncertainty in measurement, Joint Committee for Guides in Metrology, JCGM 100, 2008

    BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML . Evaluation of measurement data — Guide to the expression of uncertainty in measurement, Joint Committee for Guides in Metrology, JCGM 100, 2008

    2008

  63. [71]

    A metrological framework for uncertainty evaluation in machine learning classification models

    Bilson, S.; Cox, M.; Pustogvar, A.;Thompson , A. A metrological framework for uncertainty evaluation in machine learning classification models. arXiv 2504.03359, 2025. 22HLT01 QUMPHY 69 Page 66 of 69

    2025

  64. [72]

    Approaches for the production of reference materials with qualitative properties

    ISO 33406:2024 . Approaches for the production of reference materials with qualitative properties. International Organization for Standardization, Geneva, Switzerland, 2024

    2024

  65. [73]

    Towards trustworthy atrial fibrillation classification from wearables data: Quantifying model uncertainty

    Bench, C.; Strodthoff, N.; Moulaeifard, M.; Aston, P.J.; Thompson, A. Towards trustworthy atrial fibrillation classification from wearables data: Quantifying model uncertainty. Comput. Cardiol. 51:068, 2024

    2024

  66. [74]

    What uncertainties do we need in Bayesian deep learning for computer vision? In 31st Conference on Neural Information Processing Systems (NIPS 2017), 30, 2017

    Kendall, A; Gal, Y. What uncertainties do we need in Bayesian deep learning for computer vision? In 31st Conference on Neural Information Processing Systems (NIPS 2017), 30, 2017. arXiv 1703.04977

    2017 arXiv

  67. [75]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Lakshminarayanan, B.; Pritzel, A.; Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. In 31st Conference on Neural Information Processing Systems (NIPS 2017), 30, 2017

    2017

  68. [76]

    Strictly proper scoring rules, prediction, and estimation

    Gneiting, T.; Raftery, A.E. Strictly proper scoring rules, prediction, and estimation. J. Amer. Stat. Assoc. 102(477):359–78, 2007

    2007

  69. [77]

    Probabilistic forecasts, calibration and sharpness

    Gneiting, T.; Balabdaoui, F.; Raftery, A.E. Probabilistic forecasts, calibration and sharpness. J. Roy. Stat. Soc. Series B: Stat. Meth. 69(2):243–68, 2007

    2007

  70. [78]

    Dropout as a Bayesian approximation: Representing model uncertainty in deep learning

    Gal, Y.; Ghahramani, Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (PMLR 2016), 1050–59, 2016

    2016

  71. [79]

    Probabilistic Machine Learning: An Introduction

    Murphy, K.P. Probabilistic Machine Learning: An Introduction. MIT press, 2022

    2022

  72. [80]

    Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks

    Mucsányi, B.; Kirchhof, M.; Oh, S.J. Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks. In 3 8th Conference on Neural Information Processing Systems (NIPS 2024), 1614, 2024

    2024

  73. [81]

    Guo, C.; Pleiss, G.; Sun, Y.; Weinberger. K.Q. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (PMLR 2017), 1321– 30, 2017

    2017

  74. [82]

    Accurate uncertainties for deep learning using calibrated regression

    Kuleshov, V.; Fenner, N.; Ermon, S. Accurate uncertainties for deep learning using calibrated regression. In Proceeding of the 35th International Conference on Machine Learning (PMLR 2018), 2796–804, 2018

    2018

  75. [83]

    Transforming classifier scores into accurate multiclass probability estimates

    Zadrozny, B.; Elkan, C. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 694–699, 2002

    2002

  76. [84]

    A gentle introduction to conformal prediction and distribution - free uncertainty quantification

    Angelopoulos, A.N.; Bates, S. A gentle introduction to conformal prediction and distribution - free uncertainty quantification. arXiv 2107.07511, 2021. 22HLT01 QUMPHY 69 Page 67 of 69

    2021 arXiv

  77. [85]

    Conformal prediction

    Tibshirani, R. Conformal prediction. In Advanced Topics in Statistical Learning, UC Berkeley,

  78. [86]

    Conformalized quantile regression

    Romano, Y.; Patterson, E.; Candes, E. Conformalized quantile regression. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NIPS 2019), 32, 2019

    2019

  79. [87]

    Least ambiguous set-valued classifers with bounded error levels

    Sadinle, M.; Lei, J.; Wasserman, L. Least ambiguous set-valued classifers with bounded error levels. J. Amer. Stat. Ass. 114:223─234, 2019

    2019

  80. [88]

    Large -scale probabilistic predictors with and without guarantees of validity

    Vovk, V.; Petej, I.; Fedorova, V. Large -scale probabilistic predictors with and without guarantees of validity. In Proceedings of the 29th Conference on Neural Information Processing Systems (NIPS 2015), 28, 2015

    2015

  81. [89]

    Properties of the ENCE and other MAD-based calibration metrics

    Pernot, P . Properties of the ENCE and other MAD-based calibration metrics. arXiv 2305.11905, 2023

    2023 arXiv

  82. [90]

    Measuring calibration in deep learning

    Nixon, J.; Dusenberry, M.W.; Zhang, L.; Jerfel, G.; Tran, D. Measuring calibration in deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 38-41, 2019

    2019

  83. [91]

    Smooth ECE: Principled reliability diagrams via kernel smoothing

    Błasiok, J.; Nakkiran, P. Smooth ECE: Principled reliability diagrams via kernel smoothing. arXiv 2309.12236, 2023

    2023 arXiv

  84. [92]

    A unifying theory of distance from calibration

    Błasiok, J.; Gopalan, P.; Hu, L.; Nakkiran, P. A unifying theory of distance from calibration. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing, 1727–1740, New York, NY, USA, 2023

    2023

  85. [93]

    Uncertainty calibration error: A new metric for multi-class classification, 2021

    Laves, M.-H.; Ihler, S.; Kortmann, K.-P.; Ortmaier, T. Uncertainty calibration error: A new metric for multi-class classification, 2021. https://openreview.net/forum?id=XOuAOv_-5Fx

    2021

  86. [94]

    Extending confidence calibration to generalised measures of variation

    Thompson, A. Extending confidence calibration to generalised measures of variation. arXiv 2602.12975v1, 2026

    2026

  87. [95]

    Evaluating and calibrating uncertainty prediction in regression tasks

    Levi, D.; Gispan, L.; Giladi, N.; Fetaya, E. Evaluating and calibrating uncertainty prediction in regression tasks. Sensors 22(15):5540, 2022

    2022

  88. [96]

    A prediction interval-based approach to determine optimal structures of neural network metamodels

    Khosravi, A.; Nahavandi, S.; Creighton, D. A prediction interval-based approach to determine optimal structures of neural network metamodels. Expert Syst. Appl. 37(3):2377─87, 2010

    2010

  89. [97]

    The continuous ranked probability score for circular variables and its application to mesoscale forecast ensemble verification

    Grimit, E.P.; Gneiting, T.; Berrocal, V.J.; Johnson, N.A . The continuous ranked probability score for circular variables and its application to mesoscale forecast ensemble verification. Quart. J. Roy. Met. Soc. 132(621C):2925–42, 2006

    2006

  90. [98]

    Mieloszyk, R.; Twede, H.; Lester, J. et al . A comparison of wearable tonometry, photoplethysmography, and electrocardiography for cuffless measurement of blood pressure in an ambulatory setting. IEEE J. Biomed. Health Inform. 26(7):2864─2875, 2022. 22HLT01 QUMPHY 69 Page 68 of 69

    2022

  91. [99]

    PulseDB: A large, cleaned dataset based on MIMIC -III and VitalDB for benchmarking cuff -less blood pressure estimation methods

    Wang, W.; Mohseni, P.; Kilgore, K.L.; Najafizadeh, L. PulseDB: A large, cleaned dataset based on MIMIC -III and VitalDB for benchmarking cuff -less blood pressure estimation methods. Front. Dig. Health 4:1090854, 2023

    2023

  92. [100]

    Zhou, B

    Liu, Z. ; Zhou, B. ; Jiang, Z. et al. Multiclass arrhythmia detection and classification from photoplethysmography signals using a deep convolutional neural network. J. Amer. Heart Ass. 11(7):e023555, 2022

    2022

  93. [101]

    Bacevičius, J.; Pluščiauskaitė, V.; Abramikas, Ž. et al. Long-term electrocardiogram and wrist- based photoplethysmogram recordings with annotated atrial fibrillation episodes [Data set]”. Zenodo (2024)

    2024

  94. [102]

    Bashar, S.; Han, D.; Hajeb-Mohammadalipour, S. et al. Atrial fibrillation detection from wrist photoplethysmography signals using smartwatches. Scientific Reports 9:15054, 2019

    2019

  95. [103]

    Han, D.; Bashar, S.K.; Mohagheghian, F. et al. Premature atrial and ventricular contraction detection using photoplethysmographic data from a smartwatch. Sensors 20:5683, 2020

    2020

  96. [104]

    MIMIC-III-Ext-PPG, a PPG-based benchmark dataset for cardiovascular and respiratory signal analysis

    Moulaeifard, M.; Kutscher, M.; Aston, P.J.; Charlton, P.H.; Strodthoff, N. MIMIC-III-Ext-PPG, a PPG-based benchmark dataset for cardiovascular and respiratory signal analysis. Scientific Data 13:668, 2026

    2026

  97. [105]

    Johnson, A.E.; Pollard, T.J.; Shen, L. et al. MIMIC-III, a freely accessible critical care database. Scientific Data 3:1─9, 2016

    2016

  98. [106]

    Noise detection in electrocardiogram signals for intensive care unit patients

    Bashar, S.K.; Ding, E.; Walkey, A.J.; McManus, D.D.; Chon, K.H . Noise detection in electrocardiogram signals for intensive care unit patients. IEEE Access 7:88357─68, 2019

    2019

  99. [107]

    Charlton, P.H.; Harana, J.M.; Vennin, S. et al. Modeling arterial pulse waves in healthy aging: A database for in silico evaluation of hemodynamics and pulse wave indexes. Amer. J. Physiol.-Heart Circul. Physiol. 317:H1062─H1085, 2019

    2019

  100. [108]

    OSASUD: A dataset of stroke unit recordings for the detection of Obstructive Sleep Apnea Syndrome

    Bernardini, A.; Brunello, A.; Gigli, G.L.; Montanari, A.; Saccomanno, N. OSASUD: A dataset of stroke unit recordings for the detection of Obstructive Sleep Apnea Syndrome. Scientific Data 9:177, 2022

    2022

  101. [109]

    Chen, X.; Wang, R.; Zee, P. et al. Racial/ethnic differences in sleep disturbances: The Multi- Ethnic Study of Atherosclerosis (MESA). Sleep 38(6):877─88, 2015

    2015

  102. [110]

    MIMIC PERform Datasets

    Charlton, P.H. MIMIC PERform Datasets. Zenodo, Jul y 15, 2025. doi: 10.5281/zenodo.15906524

    2025 doi

  103. [111]

    The impact of skin tone on performance of pulse oximeters used by NHS England COVID Oximetry @home scheme: Measurement and diagnostic accuracy study

    Martin, D. The impact of skin tone on performance of pulse oximeters used by NHS England COVID Oximetry @home scheme: Measurement and diagnostic accuracy study. BMJ 392:e085535, 2026. 22HLT01 QUMPHY 69 Page 69 of 69

    2026

  104. [112]

    The validity and practicality of sun -reactive skin types I through VI

    Fitzpatrick, T . The validity and practicality of sun -reactive skin types I through VI. Arch. Dermatol. 124:869–871, 1988

    1988

  105. [113]

    Aston, P.J

    Krishnadas, P.; Hackstein, U.; Bosnjakovic, A. ; Aston, P.J. Fuzzy accuracy compensates for label subjectivity in classification of skin tone using wearable photoplethysmography signals. IEEE International Conference on Fuzzy Systems, 2026

    2026

  106. [114]

    Symmetric Projection Attractor Reconstruction: Sex differences in the ECG

    Lyle, J.V.; Nandi, M.; Aston, P.J. Symmetric Projection Attractor Reconstruction: Sex differences in the ECG. Front. Cardiovasc. Med. 8:709457, 2021

    2021

  107. [115]

    QUMPHY/D2-code/Repository: https://gitlab.com/qumphy/d2-code

  108. [116]

    QUMPHY/D1-code/Repository: https://gitlab.com/qumphy/d1-code

  109. [117]

    Wilkinson, M.D.; Dumontier, M.; Aalbersberg, I.J. et al. The FAIR guiding principles for scientific data management and stewardship. Scientific Data 3:160018, 2016

    2016

  110. [2023]

    https://www.stat.berkeley.edu/~ryantibs/statlearn-s23/lectures/conformal.pdf

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.