Pith. sign in

REVIEW 2 major objections 8 minor 76 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Machine learning unifies gravitational-wave pipelines, surfaces hidden signals

2026-07-09 15:24 UTC pith:SYK6VOLH

load-bearing objection ML+conformal prediction pipeline combination applied to O4 data; up-rankings rest on an untested feature configuration the 2 major comments →

arxiv 2607.07272 v1 pith:SYK6VOLH submitted 2026-07-08 gr-qc astro-ph.HEastro-ph.IM

Combining gravitational wave search pipelines to find subthreshold signals in GWTC-5.0

Ann-Kristin Malz , Samuel Russell , Gregory Ashton , Nicolo Colombo This is my paper
classification gr-qc astro-ph.HEastro-ph.IM
keywords candidateconfidenceeventsframeworkgravitationalwaveacrossdifferent
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Multiple independent search algorithms scan gravitational-wave detector data for signals, but they often disagree on whether a given candidate is real. Rather than simply trusting whichever pipeline reports the highest significance, this paper trains a machine-learning classifier to learn the correlations between pipeline outputs — recognising that genuine signals tend to be recovered consistently across multiple pipelines with compatible properties, while noise artefacts tend to trigger only one. The raw classifier output is then passed through conformal prediction, a statistical technique that converts uncalibrated scores into well-calibrated confidence values with rigorous uncertainty guarantees. Applied to the full gravitational-wave transient catalogue through the second part of the fourth observing run, the framework identifies several candidates that fall below standard detection thresholds but receive elevated confidence — most notably GW200311_103121, a subthreshold binary neutron star candidate that all tested classifiers consistently rank above threshold. The authors validate these up-rankings by showing that the high-confidence candidates have physically plausible source parameters, chirp-like features in time-frequency spectrograms, and feature-attribution patterns dominated by multi-pipeline detection significance rather than spurious correlations. They also demonstrate robustness across three different classifier architectures and two independent simulated training datasets, and show that when the statistical exchangeability assumption is deliberately broken, the framework still produces far fewer false positives than the standard maximum-significance approach.

Core claim

The central claim is that combining the outputs of multiple gravitational-wave search pipelines through supervised machine learning plus conformal prediction yields a single, well-calibrated confidence score per candidate that outperforms the current practice of taking the maximum significance across pipelines. This combined score can identify subthreshold candidates — events that no single pipeline flags as significant but that multiple pipelines detect consistently — as genuinely signal-like, with the binary neutron star candidate GW200311_103121 serving as the flagship example. The framework's up-rankings are shown to be robust to classifier choice, training-data choice, and deliberate分布

What carries the argument

The machinery is a two-stage pipeline: first, a binary classifier (logistic regression, multi-layer perceptron, or XGBoost) takes a feature vector assembled from the per-pipeline log₁₀(IFAR), signal-to-noise ratio, and chirp mass reported by the cWB, PyCBC, GstLAL, and MBTA search pipelines; second, Mondrian (label-conditional) conformal prediction calibrates the classifier output into a conditional confidence score in [0, 1] that carries finite-sample coverage guarantees for each class. Non-detections by a given pipeline are assigned feature values of zero. The conditional confidence is defined as the largest error rate α at which the signal label is included in the conformal prediction set

Load-bearing premise

Conformal prediction requires that the data used to calibrate the confidence scores be statistically exchangeable with the real observational data to which the framework is applied. The calibration data comes from mock data challenges that have inflated signal rates, pipelines that were still under development, and restricted noise statistics, so exchangeability with real detector data cannot be guaranteed. If this assumption is substantially violated, the calibrated scores

What would settle it

A subthreshold candidate that the framework up-ranks to high confidence could be shown to be a noise artefact — for instance, by identifying an instrumental glitch in both detectors at the trigger time, or by finding that the multi-pipeline agreement driving the up-ranking reflects correlated noise rather than a genuine signal. Conversely, the framework's value would be undermined if a large fraction of its up-ranked candidates turned out to be noise when subjected to deeper follow-up, or if the confidence scores lost calibration when applied to data from a new observing run with different

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. This manuscript extends a previously published framework (Ashton et al. 2026) that combines outputs from multiple gravitational-wave search pipelines using supervised machine learning and conformal prediction (CP) to produce a single, well-calibrated confidence score per candidate. The extensions include: (1) comparing multiple classifier architectures (LR, MLP, XGBoost, KNN) rather than only LR; (2) validating robustness using a second mock dataset (LLPIC) both as a consistency check and as an unseen test set under distribution shift; (3) applying the framework to new O4a (GWTC-4.1) and O4b (GWTC-5.0) candidates; and (4) physically validating up-ranked subthreshold candidates via Bayesian parameter estimation, time-frequency spectrograms, and SHAP feature analysis. The central observational claim is that several subthreshold candidates — most notably the BNS candidate GW200311_103121 — receive elevated confidence across classifiers and exhibit signal-like properties. The framework is well-motivated, the robustness checks are extensive, and the physical validation of up-ranked candidates is a genuine strength.

Significance. The problem of combining multi-pipeline outputs into a single significance measure is important and timely for the LVK collaboration, especially as the catalogue grows and real-time alert follow-up becomes increasingly competitive. The use of conformal prediction to provide calibrated uncertainties is a meaningful methodological contribution. The paper ships reproducible code and data releases (Ref [74]), performs SHAP-based interpretability analysis, and provides falsifiable validation through parameter estimation and spectrograms. The identification of GW200311_103121 as a consistently up-ranked BNS candidate, supported by physically plausible source parameters from an independent parameter estimation code (Bilby), is a concrete and testable claim. The work is a natural and substantial extension of the authors' prior PRL paper (Ref [44]).

major comments (2)
  1. Appendix A1: The paper states that for mock data studies it uses all 21 features per pipeline, but when applying to real GWTC candidates it restricts to 3 features per pipeline (IFAR, SNR, chirp mass). This means every validation metric — the ROC curves (Figs. 2, 3), the AUC scores, the LLPIC cross-check results (Table I, Fig. 5), and the false positive rates — is computed with the 21-feature configuration, while the actual up-rankings of real candidates use the 3-feature configuration. The LLPIC-as-test experiment (Section IIB2), which is the most direct probe of exchangeability violation, does not test the 3-feature configuration that produces the central claims. The only justification offered is a citation to Ashton et al. [44] stating that additional features do not improve performance significantly, but that result used different classifiers and data. The authors should either (a)重复
  2. Section IIB2, Table I: The false positive rates reported here (e.g., XGBoost: 4 FP out of 670 noise events, 0.6%) are the primary quantitative evidence that the framework's up-rankings remain reliable under distribution shift. However, these rates are computed on LLPIC data with 21 features and with an artificially inflated signal rate. The paper itself notes that the inflated signal rate means precision estimates are not physically representative. Given that the central application is to real data with 3 features and astrophysical signal rates, the FPR in Table I may not accurately reflect the true false positive risk for the up-rankings on real GWTC data. The authors should clarify whether the 3-feature, real-rate regime has been tested and, if so, report the corresponding metrics.
minor comments (8)
  1. Section II, paragraph beginning 'We combine the search pipelines': The text states that features are restricted to IFAR, SNR, and chirp mass 'to ensure better exchangeability across datasets.' This is a reasonable choice, but the paper should explicitly state that all validation metrics in Sections IIA and IIB use the 21-feature configuration, to avoid readers mistakenly assuming the metrics apply to the 3-feature configuration used for the claims.
  2. Figure 1 caption: states the plot is 'reproduced from Fig. 3 of Ashton et al. [44], but due to a correction in the data assembling now includes several candidates that were previously mistakenly excluded.' It would help to briefly note what the correction was, or at least how many candidates were added, so the reader can assess the impact on the prior results.
  3. Section IIIA, discussion of GW231109_235456: The paper notes that using updated GstLAL sub-threshold search features, MLP confidence increases above threshold while LR and XGBoost remain below. The statement 'this reanalysis violates exchangeability and is purely an illustrative observation' is appropriate, but the event is discussed at some length; consider condensing this paragraph.
  4. Section IIIB: The temporal coincidence of two pairs of up-ranked candidates on the same day (GW240426_100722/GW240426_151803 and GW240828_064943/GW240828_085522) is noted but not analyzed. A brief discussion of whether these could be related noise artifacts or genuine astrophysical coincidences would strengthen the analysis.
  5. Appendix B3: The threshold justification section is thorough but somewhat lengthy. The key result — that both approaches yield thresholds clustering around 0.5 — could be stated more concisely.
  6. Table XI: GW231110_171731 has a detector-frame chirp mass of 112.3 M_sun and luminosity distance of 9.50 Gpc. The text notes this is a 'high mass candidate' but does not discuss whether such a high-mass, high-distance event is physically plausible or could be a noise artifact misclassified due to its unusual parameters. A brief comment would be appropriate.
  7. The paper uses 'pastro' without italicizing or defining it on first use in the main text (Section I). A brief definition on first mention would improve readability.
  8. Reference [1] (GWTC-5.0) and several other LVK collaboration references are dated 2026 and may not yet be publicly available. The authors should ensure all cited results are accessible to referees and readers at the time of submission.

Circularity Check

0 steps flagged

No significant circularity; the framework is trained on labeled mock data and validated through independent diagnostics.

full rationale

The paper's central claims rest on a supervised ML classifier trained on mock data challenges (MDC, LLPIC) where ground-truth labels are known from injections, and conformal prediction is calibrated on a held-out portion of this labeled data. The up-rankings of real candidates (e.g., GW200311_103121) are outputs of this trained model applied to real pipeline outputs — they are not defined in terms of the up-rankings themselves, so there is no self-defitional circularity. The self-citation to Ashton et al. [44] establishes the methodological framework but does not constitute a circular argument: the present paper independently validates the framework through cross-classifier comparisons (LR, MLP, XGBoost), cross-dataset consistency (MDC vs. LLPIC), SHAP feature-attribution analysis, Bayesian parameter estimation with Bilby, and time-frequency spectrogram inspection. None of these validation steps reduce to the ML classifier's own output by construction. The skeptic's concern about the 21-feature vs. 3-feature configuration gap is a legitimate exchangeability and generalization concern (correctness risk), but it is not circularity: the 3-feature classifier is still trained on labeled mock data and applied to real data, and its performance is not defined by the up-rankings it produces. The paper explicitly acknowledges the exchangeability limitation and does not claim the CP guarantees hold unconditionally. No step in the derivation chain reduces to its inputs by definition or by a self-citation chain that is itself unverified.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical entities, particles, or forces. The framework is a methodological contribution combining existing ML classifiers with conformal prediction. The 'conditional confidence' score is a derived quantity from the CP framework, not a new entity.

free parameters (5)
  • Conditional confidence threshold = 0.5
    Chosen as the operating point for classifying candidates as signal vs noise. The paper acknowledges this is arbitrary (Appendix B3) and explores alternative thresholds, finding they cluster around ~0.5 but with inconsistent orderings across classifiers.
  • XGBoost hyperparameters = 96 trees, max depth 6
    Selected via cross-validated grid search (Appendix A2d). These control model complexity and affect classification behavior.
  • MLP architecture = K=100 neurons, single hidden layer
    Chosen after investigating deeper models; minor improvements at cost of computational efficiency (Appendix A2b).
  • KNN k value = k=19
    Obtained via cross-validation (Appendix A2c). KNN ultimately excluded from CP analysis.
  • IFAR noise threshold = 1 hour
    Candidates with IFAR > 1 hr are time-clustered, retaining only max SNR per pipeline (Appendix A1). This threshold affects the available noise statistics.
axioms (5)
  • domain assumption Exchangeability of calibration and test data for conformal prediction validity
    Invoked throughout Section II and discussed explicitly in Section I: 'CP assumes exchangeability between the data used to calibrate the framework and the data it is applied to.' This is the foundational assumption underlying all confidence score guarantees.
  • domain assumption Mock data challenge (MDC) signals and noise are representative of real astrophysical signals and detector noise
    The MDC is used as the primary training and calibration dataset. The paper acknowledges limitations: 'the injected event rate is far higher than astrophysically expected, the pipelines were still under development, and the minimum IFAR threshold restricts the available noise statistics' (Section IV).
  • domain assumption Multi-pipeline agreement is more characteristic of signals than noise
    This is the core empirical assumption learned by the classifiers. Stated in Section I: 'A signal should be consistently detected across pipelines with compatible properties, whereas a noise artifact may only trigger a single pipeline.' The SHAP analysis (Appendix B4) confirms the classifiers learn this pattern.
  • domain assumption Parameter estimation returning physically plausible source parameters supports astrophysical origin
    Used in Section III to validate up-ranked candidates. The paper acknowledges this is not definitive: 'parameter estimation cannot be used to determine whether a candidate is a real signal' (Section IIIA).
  • domain assumption log10(IFAR), SNR, and chirp mass are sufficient features for pipeline combination
    Features restricted to these three per pipeline for real data application (Section II). Justified by finding in [44] that additional features do not improve performance significantly, but this restriction may lose information.

pith-pipeline@v1.1.0-glm · 45378 in / 3633 out tokens · 153125 ms · 2026-07-09T15:24:39.523419+00:00 · methodology

0 comments
read the original abstract

The detection of transient gravitational wave signals relies on independent search algorithms that analyse detector data and assign significance measures to candidate events. However, varying performance complicates their interpretation. We use supervised machine learning combined with conformal prediction, a framework to quantify uncertainties, to merge multi-pipeline information into well-calibrated confidence scores. We demonstrate that this approach is robust across different classifier architectures and remains stable when trained on different simulated datasets. When applied to events across the GWTC catalogue up to and including the second part of the fourth observing run, the framework identifies several subthreshold candidates with elevated confidence, including the binary neutron star candidate GW200311_103121. We examine the reliability of these up-rankings, finding evidence that high-confidence predictions correspond to signal-like events. This framework enables simplified systematic candidate assessment for gravitational wave catalogues and real-time alerts by providing a single, well-calibrated confidence measure per candidate.

Figures

Figures reproduced from arXiv: 2607.07272 by Ann-Kristin Malz, Gregory Ashton, Nicolo Colombo, Samuel Russell.

Figure 1
Figure 1. Figure 1: FIG. 1. The conditional confidence in the signal label for [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2. ROC curve comparing the performance of four different [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4. The conditional confidence in the signal label for the O3 events in the GWTC-2.1 and GWTC-3.0 catalogue as obtained [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: FIG. 5. The conditional confidence in the signal label for the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: FIG. 6. The conditional confidence in the signal label for [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: FIG. 7. The conditional confidence in the signal label for [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: FIG. 8. Time-frequency spectrograms for some of the up-ranked O4a subthreshold candidates in GWTC-4.1. The dashed white [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: FIG. 10. The conditional confidence in the signal label for [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 9
Figure 9. Figure 9: FIG. 9. Violin plots of estimated posterior distributions for [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: FIG. 11. Time-frequency spectrograms for some of the up-ranked O4b subthreshold candidates in GWTC-5.0. The dashed white [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: FIG. 12. Violin plots of estimated posterior distributions [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: FIG. 13. The conditional confidence in the signal label for [PITH_FULL_IMAGE:figures/full_fig_p020_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: FIG. 14. The conditional confidence in the signal label for [PITH_FULL_IMAGE:figures/full_fig_p020_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: FIG. 15. Sensitivity for the [PITH_FULL_IMAGE:figures/full_fig_p023_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: FIG. 16. Precision curves for the [PITH_FULL_IMAGE:figures/full_fig_p024_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: FIG. 17 [PITH_FULL_IMAGE:figures/full_fig_p024_17.png] view at source ↗
Figure 19
Figure 19. Figure 19: (a)), but this is rare in the MDC, partly because 41% of injections are BNS signals that cWB is not well suited to detect. We can also examine feature importance for individ￾ual events to understand the reasoning behind a specific prediction. Focusing on the up-ranked BNS candidate GW200311_103121, the SHAP values, computed using GWTC-3.0 as the reference dataset, are shown in the waterfall plot in [PITH… view at source ↗
Figure 20
Figure 20. Figure 20: FIG. 20. Comparison of mean [PITH_FULL_IMAGE:figures/full_fig_p026_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: FIG. 21 [PITH_FULL_IMAGE:figures/full_fig_p027_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: FIG. 22 [PITH_FULL_IMAGE:figures/full_fig_p028_22.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

76 extracted references · 76 canonical work pages · 4 internal anchors

  1. [1]

    Klimenko and Guenakh Mitselmakher

    S. Klimenko and Guenakh Mitselmakher. A wavelet method for detection of gravitational wave bursts.Class. Quant. Grav., 21:S1819–S1830, 2004. doi:10.1088/0264- 9381/21/20/025

    work page doi:10.1088/0264- 2004

  2. [2]

    Klimenko, G

    S. Klimenko, G. Vedovato, M. Drago, G. Mazzolo, G. Mitselmakher, C. Pankow, G. Prodi, V. Re, F. Salemi, and I. Yakushin. Localization of gravitational wave sources with networks of advanced detectors.Phys. Rev. D, 83:102001, 2011. doi:10.1103/PhysRevD.83.102001

    work page doi:10.1103/physrevd.83.102001 2011

  3. [3]

    Machine-learningpipelineforreal-time detection of gravitational waves from compact binary coalescences.Phys

    EthanMarxetal. Machine-learningpipelineforreal-time detection of gravitational waves from compact binary coalescences.Phys. Rev. D, 111(4):042010, 2025. doi: 10.1103/PhysRevD.111.042010

    work page doi:10.1103/physrevd.111.042010 2025

  4. [4]

    Vasileios Skliris, Michael R. K. Norman, and Patrick J. Sutton. Toward real-time detection of unmodeled grav- itational wave transients using convolutional neural networks.Phys. Rev. D, 110(10):104034, 2024. doi: 10.1103/PhysRevD.110.104034

    work page doi:10.1103/physrevd.110.104034 2024

  5. [5]

    Nitz, Sumit Kumar, Yi-Fan Wang, et al

    Alexander H. Nitz, Sumit Kumar, Yi-Fan Wang, et al. 4-OGC: Catalog of Gravitational Waves from Compact Binary Mergers.Astrophys. J., 946(2):59, 2023. doi: 10.3847/1538-4357/aca591

    work page doi:10.3847/1538-4357/aca591 2023

  6. [6]

    New binary black hole mergers in the LIGO-Virgo O3b data.Phys

    Ajit Kumar Mehta, Seth Olsen, Digvijay Wadekar, Javier Roulet, Tejaswi Venumadhav, Jonathan Mushkin, Barak Zackay, and Matias Zaldarriaga. New binary black hole mergers in the LIGO-Virgo O3b data.Phys. Rev. D, 111 (2):024049, 2025. doi:10.1103/PhysRevD.111.024049

    work page doi:10.1103/physrevd.111.024049 2025

  7. [7]

    Abbott et al

    R. Abbott et al. Gwtc-3: Compact binary coalescences observed by ligo and virgo during the second part of the third observing run.Phys. Rev. X, 13:041039, Dec

    work page

  8. [8]

    URL https: //link.aps.org/doi/10.1103/PhysRevX.13.041039

    doi:10.1103/PhysRevX.13.041039. URL https: //link.aps.org/doi/10.1103/PhysRevX.13.041039

    work page doi:10.1103/physrevx.13.041039

  9. [9]

    A. G. Abac et al. GWTC-4.0: Updating the Gravitational-Wave Transient Catalog with Observations from the First Part of the Fourth LIGO-Virgo-KAGRA Observing Run. 8 2025

    work page 2025

  10. [10]

    Sharan Banagiri, Christopher P. L. Berry, Gareth S. Cabourn Davies, Leo Tsukada, and Zoheyr Doc- tor. Unified pastro for gravitational waves: Con- sistently combining information from multiple search pipelines.Phys. Rev. D, 108(8):083043, 2023. doi: 10.1103/PhysRevD.108.083043

    work page doi:10.1103/physrevd.108.083043 2023

  11. [11]

    Coughlin, Man Leong Chan, and Leo Singer

    Seiya Tsukamoto, Andrew Toivonen, Holton Griffin, Avyukt Raghuvanshi, Megan Averill, Frank Kerkow, Michael W. Coughlin, Man Leong Chan, and Leo Singer. Astrophysical or Terrestrial: Machine Learning Classifi- cation of Gravitational-wave Candidates Using Multiple- search Information.Publ. Astron. Soc. Pac., 138(1): 014503, 2026. doi:10.1088/1538-3873/ae1e2e

    work page doi:10.1088/1538-3873/ae1e2e 2026

  12. [12]

    The best of all worlds: gravitational-wave transient detection with multiple pipelines

    Nikolas Moustakidis, Theofilo Moustakidis, Deep Chat- terjee, Anastasios Tefas, and Erik Katsavounidis. The best of all worlds: gravitational-wave transient detection with multiple pipelines. 2026. in preparation

    work page 2026

  13. [13]

    Enhancing Gravitational-Wave Detection: A Machine 15 Learning Pipeline Combination Approach with Robust Uncertainty Quantification.Phys

    Gregory Ashton, Ann-Kristin Malz, and Nicolo Colombo. Enhancing Gravitational-Wave Detection: A Machine 15 Learning Pipeline Combination Approach with Robust Uncertainty Quantification.Phys. Rev. Lett., 136(1): 011402, 2026. doi:10.1103/yfb3-fgf2

    work page doi:10.1103/yfb3-fgf2 2026

  14. [14]

    Springer, 2005

    Vladimir Vovk, Alexander Gammerman, and Glenn Shafer.Algorithmic learning in a random world, vol- ume 29. Springer, 2005

    work page 2005

  15. [15]

    A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

    Anastasios N Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification.arXiv preprint arXiv:2107.07511, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  16. [16]

    Calibrating gravitational-wave search algorithms with conformal prediction.Phys

    Gregory Ashton, Nicolo Colombo, Ian Harry, and Surabhi Sachdev. Calibrating gravitational-wave search algorithms with conformal prediction.Phys. Rev. D, 109 (12):123027, 2024. doi:10.1103/PhysRevD.109.123027

    work page doi:10.1103/physrevd.109.123027 2024

  17. [17]

    Redshift Prediction with Images for Cosmology Using a Bayesian Convolutional Neural Network with Conformal Predictions.Astrophys

    Evan Jones, Tuan Do, Yun Qi Li, Kevin Alfaro, Jack Sin- gal, and Bernie Boscoe. Redshift Prediction with Images for Cosmology Using a Bayesian Convolutional Neural Network with Conformal Predictions.Astrophys. J., 974 (2):159, October 2024. doi:10.3847/1538-4357/ad6d5a

    work page doi:10.3847/1538-4357/ad6d5a 2024

  18. [18]

    Improving Generalization and Uncertainty Quantification of Photometric Redshift Models.Astron

    Jonathan Soriano, Tuan Do, Srinath Saikrishnan, Vikram Seenivasan, Bernie Boscoe, Jack Singal, and Evan Jones. Improving Generalization and Uncertainty Quantification of Photometric Redshift Models.Astron. J., 171(2):114, 2026. doi:10.3847/1538-3881/ae2ffe

    work page doi:10.3847/1538-3881/ae2ffe 2026

  19. [19]

    Uncertainty Estimation in Classification of Massive Stars Spectra using Conformal Predictions

    Raquel Pezoa, Luis Salinas, Claudio Torres, Felipe Ortiz, Michel Cure, and Ignacio Araya. Uncertainty Estimation in Classification of Massive Stars Spectra using Conformal Predictions. In2024 43rd Interna- tional Conference of the Chilean Computer Science Society (SCCC), page 43. IEEE, October 2024. doi: 10.1109/SCCC63879.2024.10767640

    work page doi:10.1109/sccc63879.2024.10767640 2024

  20. [20]

    Uncertainty quan- tification of the virial black hole mass with conformal prediction.Mon

    Suk Yee Yong and Cheng Soon Ong. Uncertainty quan- tification of the virial black hole mass with conformal prediction.Mon. Not. Roy. Astron. Soc., 524(2):3116– 3129, 2023. doi:10.1093/mnras/stad2080

    work page doi:10.1093/mnras/stad2080 2023

  21. [21]

    Marlon M. S. Mendes, Roberta Duarte Pereira, Mariana Dutra da Rosa Louren, and César H. Lenzi. Certified Uncertainty for Surrogate Models of Neutron Star Equa- tions of State via Mondrian Conformal Prediction. 2 2026

    work page 2026

  22. [22]

    Williams, and Sujit Ghosh

    Naomi Singer, Jonathan P. Williams, and Sujit Ghosh. Conformal prediction for astronomy data with measurement error.Monthly Notices Royal Astro- nomical Society, 539(2):1372–1380, May 2025. doi: 10.1093/mnras/staf515

    work page doi:10.1093/mnras/staf515 2025

  23. [23]

    Araz and Michael Spannowsky

    Jack Y. Araz and Michael Spannowsky. Another Fit Bites the Dust: Conformal Prediction as a Calibration Standard for Machine Learning in High-Energy Physics. 12 2025

    work page 2025

  24. [24]

    Classification uncertainty for transient gravitational- wave noise artifacts with optimized conformal pre- diction.Phys

    Ann-Kristin Malz, Gregory Ashton, and Nicolo Colombo. Classification uncertainty for transient gravitational- wave noise artifacts with optimized conformal pre- diction.Phys. Rev. D, 111(8):084078, 2025. doi: 10.1103/PhysRevD.111.084078

    work page doi:10.1103/physrevd.111.084078 2025

  25. [25]

    Abbott et al

    R. Abbott et al. GWTC-2.1: Deep Extended Catalog of Compact Binary Coalescences Observed by LIGO and Virgo During the First Half of the Third Observing Run. 8 2021

    work page 2021

  26. [27]

    A tutorial on conformal prediction

    GlennShaferandVladimirVovk. Atutorialonconformal prediction.Journal of Machine Learning Research, 9: 371–421, 2008. doi:10.48550/arXiv.0706.3188

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.0706.3188 2008

  27. [28]

    Gwtc-2.1: Deep extended catalog of compact binary coalescences observed by ligo and virgo during the first half of the third observing run - candidate data release, July 2021

    LIGO Scientific Collaboration and Virgo Collaboration. Gwtc-2.1: Deep extended catalog of compact binary coalescences observed by ligo and virgo during the first half of the third observing run - candidate data release, July 2021. URL https://doi.org/10.5281/ zenodo.5759108

    work page 2021

  28. [29]

    Gwtc-3: Compact binary coa- lescences observed by ligo and virgo during the second part of the third observing run — candidate data re- lease, November 2021

    LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration. Gwtc-3: Compact binary coa- lescences observed by ligo and virgo during the second part of the third observing run — candidate data re- lease, November 2021. URLhttps://doi.org/10.5281/ zenodo.5546665

    work page 2021

  29. [30]

    Gwtc-4.1: Candidate data release, May 2026

    LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration. Gwtc-4.1: Candidate data release, May 2026. URL https://doi.org/10.5281/ zenodo.20276095

    work page 2026

  30. [31]

    Gwtc-5.0: Candidate data release, May 2026

    LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration. Gwtc-5.0: Candidate data release, May 2026. URL https://doi.org/10.5281/ zenodo.20348004

    work page 2026

  31. [32]

    Low-latency gravi- tational wave alert products and their performance at the time of the fourth LIGO-Virgo-KAGRA observing run.Proc

    Sushant Sharma Chaudhary et al. Low-latency gravi- tational wave alert products and their performance at the time of the fourth LIGO-Virgo-KAGRA observing run.Proc. Nat. Acad. Sci., 121(18):e2316474121, 2024. doi:10.1073/pnas.2316474121

    work page doi:10.1073/pnas.2316474121 2024

  32. [33]

    Low-latency gravita- tional wave alert infrastructure and new data products for multi-messenger astronomy during ligo-virgo-kagras fourth observing run

    Sushant Sharma Chaudhary et al. Low-latency gravita- tional wave alert infrastructure and new data products for multi-messenger astronomy during ligo-virgo-kagras fourth observing run. 2026. in preparation

    work page 2026

  33. [34]

    Learning representations by back-propagating errors.nature, 323(6088):533–536, 1986

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors.nature, 323(6088):533–536, 1986

    work page 1986

  34. [35]

    XGBoost: A Scalable Tree Boosting System

    Tianqi Chen and Carlos Guestrin. XGBoost: A Scalable Tree Boosting System. 3 2016. doi: 10.1145/2939672.2939785

    work page doi:10.1145/2939672.2939785 2016

  35. [36]

    Conditional validity of inductive confor- mal predictors.Machine Learning, 92, 2013

    Vladimir Vovk. Conditional validity of inductive confor- mal predictors.Machine Learning, 92, 2013

    work page 2013

  36. [37]

    A unified approach to interpreting model predictions

    Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 30. Curran As- sociates, Inc., 2017. URL https://proceedings. neurips.cc/paper_files/paper/2017/file/ 8a20a...

    work page 2017

  37. [38]

    The elements of statistical learning, 2009

    Trevor Hastie, Robert Tibshirani, Jerome Friedman, et al. The elements of statistical learning, 2009

    work page 2009

  38. [39]

    Soni et al

    S. Soni et al. LIGO Detector Characterization in the first half of the fourth Observing run.Class. Quant. Grav., 42(8):085016, 2025. doi:10.1088/1361-6382/adc4b6

    work page doi:10.1088/1361-6382/adc4b6 2025

  39. [40]

    Case studies with GPBilby of glitch- contaminated transient gravitational waves

    Mattia Emma, Ann-Kristin Malz, Adriana Dias, and Gregory Ashton. Case studies with GPBilby of glitch- contaminated transient gravitational waves. 4 2026

    work page 2026

  40. [41]

    Measuring the rate of glitches in interferometric gravitational wave detectors with a hierarchical Bayesian model

    Gregory Ashton, Colm Talbot, Andrew Lundgren, Ann- Kristin Malz, and Joseph Areeda. Measuring the rate of glitches in interferometric gravitational wave detectors with a hierarchical Bayesian model. 4 2026

    work page 2026

  41. [42]

    A. G. Abac et al. GWTC-4.0: Methods for Identifying and Characterizing Gravitational-wave Transients. 8 2025

    work page 2025

  42. [43]

    Data release: Combining gravitational wave search pipelines to find subthreshold signals in 16 gwtc-5.0, July 2026

    Ann-Kristin Malz, Samuel Russell, Gregory Ashton, and nicolo colombo. Data release: Combining gravitational wave search pipelines to find subthreshold signals in 16 gwtc-5.0, July 2026. URLhttps://doi.org/10.5281/ zenodo.21258511

    work page 2026

  43. [44]

    BILBY: A user-friendly Bayesian inference library for gravitational-wave astronomy.As- trophys

    Gregory Ashton et al. BILBY: A user-friendly Bayesian inference library for gravitational-wave astronomy.As- trophys. J. Suppl., 241(2):27, 2019. doi:10.3847/1538- 4365/ab06fc

    work page doi:10.3847/1538- 2019

  44. [45]

    Computationally efficient models for the dominant and subdominant harmonic modes of precessing binary black holes.Phys

    Geraint Pratten et al. Computationally efficient models for the dominant and subdominant harmonic modes of precessing binary black holes.Phys. Rev. D, 103(10): 104056, 2021. doi:10.1103/PhysRevD.103.104056

    work page doi:10.1103/physrevd.103.104056 2021

  45. [46]

    Ramis Vidal, Cecilio García- Quirós, Sarp Akçay, and Sayantani Bera

    Marta Colleoni, Felip A. Ramis Vidal, Cecilio García- Quirós, Sarp Akçay, and Sayantani Bera. Fast frequency- domain gravitational waveforms for precessing binaries with a new twist.Phys. Rev. D, 111(10):104019, 2025. doi:10.1103/PhysRevD.111.104019

    work page doi:10.1103/physrevd.111.104019 2025

  46. [47]

    Joshua S. Speagle. dynesty: a dynamic nested sampling package for estimating Bayesian posteriors and evidences. Mon. Not. Roy. Astron. Soc., 493(3):3132–3158, 2020. doi:10.1093/mnras/staa278

    work page doi:10.1093/mnras/staa278 2020

  47. [48]

    A. G. Abac et al. GWTC-4.0: An Introduction to Version 4.0 of the Gravitational-Wave Transient Catalog. Astrophys. J. Lett., 995(1):L18, 2025. doi:10.3847/2041- 8213/ae0c06

    work page doi:10.3847/2041- 2025

  48. [49]

    GW231109_235456: A Sub-threshold Binary Neutron Star Merger in the LIGO-Virgo-KAGRA O4a Observing Run? 9 2025

    Wanting Niu et al. GW231109_235456: A Sub-threshold Binary Neutron Star Merger in the LIGO-Virgo-KAGRA O4a Observing Run? 9 2025

    work page 2025

  49. [50]

    A. G. Abac et al. GW231123: A Binary Black Hole Merger with Total Mass 190–265 M⊙.Astrophys. J. Lett., 993(1):L25, 2025. doi:10.3847/2041-8213/ae0c9c

    work page doi:10.3847/2041-8213/ae0c9c 2025

  50. [51]

    API design for machine learning software: experiences from the scikit-learn project

    Lars Buitinck et al. API design for machine learning software: experiences from the scikit-learn project. 9 2013

    work page 2013

  51. [52]

    Array programming with numpy.Nature, 585(7825):357–362, 2020

    Charles R Harris, K Jarrod Millman, Stéfan J van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al. Array programming with numpy.Nature, 585(7825):357–362, 2020

    work page 2020

  52. [53]

    pandas-dev/pandas: Pandas, February 2020

    The pandas development team. pandas-dev/pandas: Pandas, February 2020. URL https://doi.org/10. 5281/zenodo.3509134

    work page 2020

  53. [54]

    John D. Hunter. Matplotlib: A 2d graphics environment. Computing In Science & Engineering, 9(3):90–95, May- Jun 2007

    work page 2007

  54. [55]

    Klimenko, V

    Vaibhav Tiwari, S. Klimenko, V. Necula, and G. Mit- selmakher. Reconstruction of chirp mass in searches for gravitational wave transients.Class. Quant. Grav., 33 (1):01LT01, 2016. doi:10.1088/0264-9381/33/1/01LT01

    work page internal anchor Pith review doi:10.1088/0264-9381/33/1/01lt01 2016

  55. [56]

    The regression analysis of binary se- quences.Journal of the Royal Statistical Society Series B: Statistical Methodology, 20(2):215–232, 1958

    David R Cox. The regression analysis of binary se- quences.Journal of the Royal Statistical Society Series B: Statistical Methodology, 20(2):215–232, 1958

    work page 1958

  56. [57]

    Byrd, Peihuang Lu, and Jorge Nocedal

    Ciyou Zhu, Richard H. Byrd, Peihuang Lu, and Jorge Nocedal. Algorithm 778: L-BFGS-B.ACM Trans. Math. Software, 23(4):550–560, 1997. doi: 10.1145/279232.279236

    work page doi:10.1145/279232.279236 1997

  57. [58]

    Rectified linear units improve restricted boltzmann machines

    Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. InProceedings of the 27th international conference on machine learning (ICML-10), pages 807–814, 2010

    work page 2010

  58. [59]

    Deep Learning using Rectified Lin- ear Units (ReLU)

    Abien Fred Agarap. Deep Learning using Rectified Lin- ear Units (ReLU). 3 2018

    work page 2018

  59. [60]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. 12 2014

    work page 2014

  60. [61]

    Nearest neighbor pattern classification.IEEE transactions on information theory, 13(1):21–27, 1967

    Thomas Cover and Peter Hart. Nearest neighbor pattern classification.IEEE transactions on information theory, 13(1):21–27, 1967

    work page 1967

  61. [62]

    Class-Conditional Conformal Prediction with Many Classes

    Tiffany Ding, Anastasios N. Angelopoulos, Stephen Bates, Michael I. Jordan, and Ryan J. Tibshi- rani. Class-conditional conformal prediction with many classes.CoRR, abs/2306.09335, 2023. doi: 10.48550/arXiv.2306.09335. URL https://doi.org/10. 48550/arXiv.2306.09335

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2306.09335 2023

  62. [63]

    The use of hypermod- els to understand binary neutron star collisions.Nature Astron., 6(8):961–967, 2022

    Gregory Ashton and Tim Dietrich. The use of hypermod- els to understand binary neutron star collisions.Nature Astron., 6(8):961–967, 2022. doi:10.1038/s41550-022- 01707-x

    work page doi:10.1038/s41550-022- 2022

  63. [64]

    Abbott et al

    R. Abbott et al. GWTC-2: Compact Binary Coales- cences Observed by LIGO and Virgo During the First Half of the Third Observing Run.Phys. Rev. X, 11: 021053, 2021. doi:10.1103/PhysRevX.11.021053

    work page doi:10.1103/physrevx.11.021053 2021

  64. [65]

    GW190426_152155: a merger of neutron star-black hole or low mass binary black holes? 12 2020

    Yin-Jie Li, Ming-Zhe Han, Shao-Peng Tang, Yuan-Zhu Wang, Yi-Ming Hu, Qiang Yuan, Yi-Zhong Fan, and Da-Ming Wei. GW190426_152155: a merger of neutron star-black hole or low mass binary black holes? 12 2020

    work page 2020

  65. [66]

    Schäfer and Alexander H

    Marlin B. Schäfer and Alexander H. Nitz. From one to many: A deep learning coincident gravitational- wave search.Phys. Rev. D, 105(4):043003, 2022. doi: 10.1103/PhysRevD.105.043003

    work page doi:10.1103/physrevd.105.043003 2022

  66. [67]

    The relationship be- tween precision-recall and roc curves

    Jesse Davis and Mark Goadrich. The relationship be- tween precision-recall and roc curves. InProceedings of the 23rd international conference on Machine learning, pages 233–240, 2006

    work page 2006

  67. [68]

    The precision-recall plot is more informative than the roc plot when evaluat- ing binary classifiers on imbalanced datasets.PloS one, 10(3):e0118432, 2015

    Takaya Saito and Marc Rehmsmeier. The precision-recall plot is more informative than the roc plot when evaluat- ing binary classifiers on imbalanced datasets.PloS one, 10(3):e0118432, 2015

    work page 2015

  68. [69]

    The in- consistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve.American journal of epidemiology, 163(7):670–675, 2006

    Neil J Perkins and Enrique F Schisterman. The in- consistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve.American journal of epidemiology, 163(7):670–675, 2006

    work page 2006

  69. [70]

    From local explanations to global understanding with explainable ai for trees.Nature machine intelligence, 2 (1):56–67, 2020

    Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable ai for trees.Nature machine intelligence, 2 (1):56–67, 2020. Appendix A: Method details

    work page 2020

  70. [71]

    From the template- based pipelines, we use the reportedFAR, SNR, and χ2 (a matched-filter signal-consistency test across frequency bands [11, 25])

    Data preparation The pipeline outputs include both significance statis- tics and source parameter estimates. From the template- based pipelines, we use the reportedFAR, SNR, and χ2 (a matched-filter signal-consistency test across frequency bands [11, 25]). These pipelines also provide estimates of source parameters from the nearest waveform tem- plate. We...

    work page

  71. [72]

    Here, we explore addi- tional classifiers and compare the performance ofLR, MLP, K-nearest neighbours (KNN), and XGBoost, a tree-based method

    Machine learning classifiers In our previous work [44], we focused primarily on LR due to its interpretability. Here, we explore addi- tional classifiers and compare the performance ofLR, MLP, K-nearest neighbours (KNN), and XGBoost, a tree-based method. We implement the first three using scikit-learn [82], and use theXGBoost package [66] for the latter. ...

    work page

  72. [73]

    Thus,CP requires no assumptions about the model or data dis- tributions

    Conformal prediction CP [45, 46] is a framework developed to provide sta- tistically rigorous and well-calibrated uncertainties for any point prediction.CP does not modify the underlying algorithm but instead uses its predictions on a labelled calibration dataset to learn the uncertainty. Thus,CP requires no assumptions about the model or data dis- tribut...

    work page

  73. [74]

    analysing the event under the assumption that it is an astrophysical signal, its properties are highly consistent with that of a binary neutron star merger

    Model comparison The rawML classifier output lacks a statistically rig- orous significance measure, unlike theFARprovided by the maximum-log10IFAR approach, an essential compo- nent for assessing the significance of individual candidate events and motivating the use ofCP. We therefore apply CP to the ML classifier predictions and use the condi- tional con...

    work page

  74. [75]

    Sensitivity and precision In theGW literature, the sensitive volume [23, 97] is a key metric used to compare pipeline performance on mock data. It is estimated from injections as the fraction of simulated signals recovered above a givenFARthreshold (equivalently, the sensitivity at that threshold) scaled by the maximum survey volume [23, 97]. Since sensit...

    work page

  75. [76]

    If the main goal is catalogue purity, one approach to selecting a confidence threshold would be to choose the point at which precision reaches unity, ensuring no false positives

    Choosing the conditional confidence threshold For our analysis so far, we have used the arbitrary confidence threshold of0.5to distinguish between signal and noise; we now briefly explore alternative thresholds. If the main goal is catalogue purity, one approach to selecting a confidence threshold would be to choose the point at which precision reaches un...

    work page

  76. [77]

    This allows us to move beyond black-box predictions and understand the physical reasoning behind each classification

    Feature importance Having established XGBoost as our preferred classi- fier, we now examine the decision-making process of all classifiers throughSHAP analysis [68], using XGBoost as the primary example.SHAP analysis quantifies the con- tribution of each input feature to individual predictions, revealing which pipeline outputs drove a given prediction by ...

    work page