REVIEW 2 major objections 5 minor 1 cited by
Searching for stellar-origin binary black holes in LISA Data Challenge 1b: Yorsh
T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A semi-coherent search recovers all five stellar-origin black hole binaries in LISA Data Challenge 1b with injected SNR at least 12.94, indicating a lower detection threshold.
desk verdict A useful, honest first benchmark of SoBBH searching on an official LISA Data Challenge, but the 'confident detection' label leans on an unvalidated background from an earlier paper. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the semi-coherent detection statistic $\Upsilon$: a matched-filter score built by splitting the data into segments, comparing eccentric post-Newtonian template waveforms (TaylorF2Ecc, with aligned-spin terms up to 2.5 PN order) against the A/E/T time-delay-interferometry channels, and maximizing over parameters. The search uses a particle-swarm optimizer to explore each narrow tile in chirp-mass and starting-frequency space, and a fast frequency-domain model of the LISA response (TDI-1.5, a rigid rotating-constellation version of time-delay interferometry) rather than the full TDI-2 response used in the injections. Candidates with $\Upsilon \geq 100$ are declared confidently detected based on a noise background imported from an earlier search, then followed up with a fast ensemble MCMC for parameter estimation.
What would settle it
Generate many Yorsh-like datasets containing only the same instrumental noise and no injected signals, run the identical search (SC search-1.5 with the same tiles and threshold), and count how often a pure-noise candidate reaches $\Upsilon \geq 100$; if the false-alarm rate is materially higher than the imported background predicts, the confident-detection labels are not calibrated for Yorsh.
Extended reading notes
Core claim
The central claim is that a semi-coherent matched-filter search can find the five loudest stellar-origin binary black holes hidden in the Yorsh LISA Data Challenge 1b dataset, even though the search's waveform and instrument-response models differ from those used to generate the data. In the best-performing version (SC search-1.5, using the TDI-1.5 response and the Yorsh noise power spectral density), all five sources with injected SNR of 12.94 or higher are confidently detected; chirp masses are recovered within about $0.002\,M_\odot$ and merger times within hours, with the two sources that merge during the mission recovered to within minutes. The paper interprets these detections as evidence that the minimum SNR needed to detect stellar-origin binary black holes in LISA is lower than previous estimates. Rapid MCMC parameter estimation confirms that the detections correspond to the injected sources, with the expected small biases from waveform and response modeling.
Load-bearing premise
The significance threshold that decides which candidates count as confidently detected was calibrated on a background of noise triggers measured in a different, earlier search on different synthetic data, and that same threshold is applied to every search tile here without re-measuring the background for Yorsh.
Editorial extensions
If this is right
- If the threshold SNR for detecting stellar-origin binary black holes in LISA is around 12 rather than higher, the expected number of such sources in real LISA data increases, since source counts depend steeply on this threshold.
- Simplified LISA response models (TDI-1 and TDI-1.5) are sufficient for detecting some sources, especially at lower frequencies, so parts of a global fit may be able to use cheaper response models without losing the sources.
- The automated rapid parameter estimation after each detection delivers posteriors narrow enough to initialize more detailed global-fit parameter estimation.
- For the two sources that merge within the LISA mission lifetime, merger time is recovered to within minutes, enabling multi-band follow-up planning.
- The per-tile computational cost of about one to three days on a GPU makes a full survey of roughly 100 to 1000 search tiles over the stellar-origin binary black hole parameter space feasible.
Reading between the lines
- Because the search used one hand-picked tile per injected source rather than covering the whole parameter space automatically, the demonstrated sensitivity applies to sources whose approximate location in chirp-mass and frequency space is already known; a fully blind search still needs automated tiling and per-tile background calibration.
- The success with mismatched models suggests the search is robust to waveform and response inaccuracies for the loudest sources, but it also means the reported parameter biases, such as underestimated distances and slightly nonzero eccentricities, are partly systematic offsets from model mismatch rather than purely statistical errors.
- If the lower threshold holds in realistic data with gaps, noise uncertainties, and confusion from other source classes, previous forecasts of LISA's stellar-origin binary black hole detection yield may need to be revised upward; this is directly testable in future LISA Data Challenges that include these sources and more realistic noise.
- A natural next step would be to re-run the same pipeline on the same Yorsh data with the full TDI-2 response to separate the response-model contribution to parameter biases from waveform-model effects.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper applies the authors' semi-coherent hierarchical search (SC search) to the LISA Data Challenge 1b Yorsh dataset, using two approximate LISA response models. The more accurate model, SC search-1.5, returns candidate detections for the five injected SoBBH sources with SNR ≥ 12.94, recovering chirp mass to within ~0.002 M_sun and merger time to within hours; rapid MCMC parameter estimation gives posteriors consistent with the injections. The paper interprets this as evidence that the threshold SNR for detecting SoBBHs in LISA data is lower than some previous estimates. The main caveats are that detection significance is assigned using a background distribution imported from Ref. [17] rather than a Yorsh-specific noise background, and that search tiles were hand-placed around known injections.
Significance. If the results are taken at face value, they constitute a useful validation of a hierarchical semi-coherent search on a community LDC dataset, with public code and data products (Refs. [30, 46]) that aid reproducibility. The recovery accuracy reported in Tables II and III is internally consistent and the rapid parameter estimation is a practical contribution. However, the headline threshold-SNR conclusion is not fully supported: the false-alarm calibration is imported from a different search on different synthetic data, and the hand-placed tiles mean the exercise is not a blind search. These caveats temper the significance but do not destroy the value of the recovery results.
major comments (2)
- [III] The threshold used to classify detections as confident is imported from Fig. 5 of Ref. [17] and applied unchanged to all Yorsh tiles, with no noise-only background computed for the Yorsh PSD, TDI-2 injections, or the SC search-1.5 response. Since the false-alarm rate of the Υ statistic is a property of the search and the data, the statement that source #5 at injected SNR 12.94 is 'confidently detected' is not calibrated, and the threshold-SNR conclusion in Sec. VI rests on this unvalidated input. The authors should either compute Yorsh-specific background distributions for at least representative tiles or explicitly weaken the significance language and the threshold-SNR claim.
- [III] The search tiles were chosen by hand around each individual injection, with the flow width set so that the prior on the derived parameter tc brackets the actual merger time (Fig. 2). This makes the search non-blind: the reported detection efficiencies and the threshold-SNR interpretation apply only to a search already directed to the correct region of parameter space. The text notes that Yorsh is not a blind challenge, but the abstract and Sec. VI do not carry this qualifier; this limitation should be stated explicitly wherever the headline results are summarized.
minor comments (5)
- [Abstract and Sec. IV] The phrase 'all five sources in the data challenge with injected signal-to-noise ratios ≳ 12' is ambiguous because Yorsh contains eight SoBBH injections, three of which are not found by either search; the paper should say 'the five loudest injections' or 'all sources with injected SNR ≥ 12.94'.
- [Table II] The table layout contains stray punctuation and spacing, for example 'δMc, [M⊙]' in the SC search-1.5 header and entries such as '40689 .' and '11.60'; these should be cleaned for readability.
- [III] The semi-coherent statistic Υ_{N=1} is not defined in this paper; a one-sentence definition or an explicit equation reference to Ref. [17] would make the methods section more self-contained.
- [IV, Table II] Reporting the maximum Υ value for every candidate, including the non-detections, would let readers see how far each candidate is from the adopted threshold; currently only the found/not-found status and matched-filter SNR are given.
- [Appendix A and Sec. IV] Appendix A correctly states that no detailed convergence checks were performed for the rapid parameter estimation, but the main text says the parameter estimation 'confirms' the detections; 'is consistent with' would be more proportionate given the absence of convergence checks.
Circularity Check
The 'confidently detected' threshold is imported unchanged from the authors' earlier work (Fig. 5 of Ref. [17]) and applied to all Yorsh tiles, so the detection-significance claim rests on unvalidated self-citation; the recovered parameter values are, however, independent measurements.
-
self citation load bearing
[Sec. III (Methods), significance paragraph; used to define confident detections in Sec. IV (Table II).]
"The background should be generated by running a large number of identical searches on simulated LISA datasets which do not contain SoBBHs. For more details, see the demonstration of this process in Ref. [17]. This process is computationally expensive and must be repeated for each tile to account for variations in the background distribution across parameter space. Therefore, the background noise distribution from Fig. 5 in Ref. [17] was used for all tiles in this study. Here, sources with ΥN=1 ≥ 100 are deemed to be confidently detected."
The 'confident detection' threshold is not computed for Yorsh; it is taken from Fig. 5 of Ref. [17], the authors' own prior search on different synthetic data. Every confident detection in Table II, including source #5 (SNR 12.94) that drives the threshold-SNR conclusion, inherits its false-alarm calibration from that self-citation. The Yorsh background is never re-estimated, so the stated significance is an imported assumption, not a measured property of this dataset. Hence the headline 'confidently identifies five sources' is load-bearing on an unverified self-citation; the recovered parameter values themselves are independent measurements.
full rationale
The central parameter-recovery content is not circular: the search computes a semi-coherent statistic against Yorsh data, and the reported chirp-mass and time-to-merger accuracies are measured outputs, not inputs. The waveform and response models are cited from independent sources (TaylorF2Ecc, BBHx, LISAbeta/LISACode), and the injection/recovery mismatch is an external consistency check. The main circularity-sensitive step is the significance calibration: the Υ≥100 threshold is imported unchanged from Fig. 5 of Ref. [17] instead of being recomputed for Yorsh tiles, so the 'confident detection' claim for the five sources is supported only by a self-citation whose transferability is unverified. The paper is candid about this ('the background noise distribution from Fig. 5 in Ref. [17] was used for all tiles') and even concedes that the threshold SNR 'will not be accurately known until a search pipeline has been fully developed and run on realistic LISA data.' This warrants a moderate score rather than a high one, because the recovery of parameters remains independent content. The paper also explicitly states 'Yorsh is not a blind data challenge' and describes hand-placed search tiles around the known injections; that limits the search as a blind benchmark, but it does not by itself make the detections equivalent to the inputs, since the detection statistic still has to exceed threshold and the reported parameters come from the search and MCMC, not from the tile boundaries. No equation in the paper reduces the predicted detections to the injected values by construction, so the result is not forced circularity; the issue is the unvalidated self-cited false-alarm calibration.
Assumptions & free parameters
free parameters (2)
- Search tile boundaries in (Mc, flow) per source =
Tile width ~5 Msun in Mc; flow chosen so the prior width on tc approximately equals the source's true time to merger…
- Detection threshold Υ>=100 =
100
assumptions (3)
- domain assumption The background distribution of the semi-coherent statistic Υ from Ref. [17] is representative for all Yorsh search tiles.
- domain assumption The approximate LISA response models TDI-1 and TDI-1.5 are accurate enough to detect the injected TDI-2 signals for the sources claimed.
- domain assumption The particle swarm optimization and semi-coherent statistic converge to the loudest template in each hand-placed tile.
Cite this review
Pith. "Pith review of Searching for stellar-origin binary black holes in LISA Data Challenge 1b: Yorsh." pith.science (2026). https://pith.science/paper/7E2O474Z
@misc{pith2026241210501,
author = {Pith},
title = {Pith review of: Searching for stellar-origin binary black holes in LISA Data Challenge 1b: Yorsh},
year = {2026},
howpublished = {\url{https://pith.science/paper/7E2O474Z}},
note = {Machine review of arXiv:2412.10501}
}
abstract
This paper reports the first search for stellar-origin binary black holes within the LISA Data Challenges (LDC). The search algorithm and the \Yorsh{} LDC datasets, both previously described elsewhere, are only summarized briefly; the primary focus here is to present the results of applying the search to the challenge of data. The search employs a hierarchical approach, leveraging semi-coherent matching of template waveforms to the data using a variable number of segments, combined with a particle swarm algorithm for parameter space exploration. The computational pipeline is accelerated using graphical processing unit (GPU) hardware. The results of two searches using different models of the LISA response are presented. The most effective search finds all five sources in the data challenge with injected signal-to-noise ratios $\gtrsim 12$. Rapid parameter estimation is performed for these sources.
Figures
Forward citations
Cited by 1 Pith paper
-
Multiband parameter estimation with phase coherence and extrinsic marginalization: Extracting more information from low-SNR CBC signals in LISA data
A coherent multiband Bayesian parameter estimation method with extrinsic-parameter marginalization extracts useful information from LISA observations of stellar-mass binary black holes down to LISA SNR 3, nearly doubl...
Reference graph
Works this paper leans on
- [17]
-
[1]
Colpi et al., arXiv e-prints , arXiv:2402.07571 (2024), arXiv:2402.07571
M. Colpi et al., arXiv e-prints , arXiv:2402.07571 (2024), arXiv:2402.07571
arXiv 2024
- [2]
-
[3]
Sesana, PRL 116, 231102 (2016), arXiv:1602.06951 [gr-qc]
A. Sesana, PRL 116, 231102 (2016), arXiv:1602.06951 [gr-qc]
arXiv 2016
-
[4]
The Ligo Scientific Collaboration, VIRGO and Kagra Collaborations, Physical Review X 13, 041039 (2023), arXiv:2111.03606 [gr-qc]
arXiv 2023
- [5]
- [6]
-
[7]
A. Toubiana, S. Marsat, S. Babak, J. Baker, and T. Dal Canton, PRD 102, 124037 (2020), arXiv:2007.08544 [gr- qc]
arXiv 2020
Show all 46 references
- [8]
-
[9]
M. C. Digman and N. J. Cornish, PRD 108, 023022 (2023), arXiv:2212.04600 [gr-qc]
2023 arXiv
-
[10]
Klein et al., arXiv e-prints , arXiv:2204.03423 (2022), arXiv:2204.03423
A. Klein et al., arXiv e-prints , arXiv:2204.03423 (2022), arXiv:2204.03423
2022 arXiv
-
[11]
Y. Fu, Y. Wang, and S. D. Mohanty, arXiv e-prints , arXiv:2407.10797 (2024)
2024 arXiv
-
[12]
Zhang, N
X.-T. Zhang, N. Korsakova, M. L. Chan, C. Messenger, and Y.-M. Hu, arXiv e-prints , arXiv:2406.07336 (2024), arXiv:2406.07336
2024 arXiv
-
[13]
Zhang, C
X.-T. Zhang, C. Messenger, N. Korsakova, M. L. Chan, Y.-M. Hu, and J.-d. Zhang, PRD 105, 123027 (2022), arXiv:2202.07158
2022 arXiv
-
[14]
K. W. K. Wong, E. D. Kovetz, C. Cutler, and E. Berti, PRL 121, 251102 (2018), arXiv:1808.08247
2018 arXiv
-
[15]
Ewing, S
B. Ewing, S. Sachdev, S. Borhanian, and B. S. Sathyaprakash, PRD 103, 023025 (2021), arXiv:2011.03036 [gr-qc]
2021 arXiv
-
[16]
Bandopadhyay and C
D. Bandopadhyay and C. J. Moore, PRD 108, 084014 (2023), arXiv:2305.18048 [gr-qc]
2023 arXiv
-
[18]
LISA Data Challenges,
LISA Data Processing Group, “LISA Data Challenges,” https://lisa-ldc.lal.in2p3.fr (accessed Nov 2024)
2024
-
[19]
Karamanis, F
M. Karamanis, F. Beutler, and J. A. Peacock, arXiv preprint arXiv:2105.03468 (2021)
2021 arXiv
-
[20]
T. B. Littenberg and N. J. Cornish, PRD 107, 063004 (2023), arXiv:2301.03673 [gr-qc]
2023 arXiv
-
[21]
M. L. Katz et al., arXiv e-prints , arXiv:2405.04690 (2024)
2024 arXiv
-
[22]
T. A. Prince, M. Tinto, S. L. Larson, and J. W. Arm- strong, PRD 66, 122002 (2002), arXiv:gr-qc/0209039 [gr- qc]
2002 arXiv
-
[23]
LDC Data Chal- lenge 2a: Sangria,
LISA Data Processing Group, “LDC Data Chal- lenge 2a: Sangria,” https://lisa-ldc.lal.in2p3.fr/ static/data/pdf/LDC-manual-Sangria.pdf (2020)
2020
-
[24]
Babak, M
S. Babak, M. Hewitson, and A. Petiteau, arXiv e-prints , arXiv:2108.01167 (2021)
2021 arXiv
-
[25]
Karnesis, S
N. Karnesis, S. Babak, M. Pieroni, N. Cornish, and T. Littenberg, PRD 104, 043019 (2021), arXiv:2103.14598
2021 arXiv
-
[26]
Boileau, A
G. Boileau, A. Lamberts, N. Christensen, N. J. Cor- nish, and R. Meyer, MNRAS 508, 803 (2021), arXiv:2105.04283 [gr-qc]
2021 arXiv
-
[27]
Khan et al., PRD 93, 044007 (2016), arXiv:1508.07253 [gr-qc]
S. Khan et al., PRD 93, 044007 (2016), arXiv:1508.07253 [gr-qc]
2016 arXiv
-
[28]
LISAbeta,
S. Marsat and J. G. Baker, “LISAbeta,” https://pypi. org/project/lisabeta/1.0.2/ (2024)
2024
-
[29]
Petiteau, G
A. Petiteau, G. Auger, H. Halloin, O. Jeannin, E. Plagnol, S. Pireaux, T. Regimbau, and J.-Y. Vinet, PRD 77, 023002 (2008), arXiv:0802.2023 [gr-qc]
2008 arXiv
-
[30]
LDC-Yorsch- stellar-mass-search,
D. Bandopadhyay and C. J. Moore, “LDC-Yorsch- stellar-mass-search,” https://github.com/dig07/ LDC-Yorsh-stellar-mass-search (2024)
2024
-
[31]
Moore, M
B. Moore, M. Favata, K. G. Arun, and C. K. Mishra, PRD 93, 124061 (2016), arXiv:1605.00304 [gr-qc]
2016 arXiv
-
[32]
Fumagalli, I
G. Fumagalli, I. Romero-Shaw, D. Gerosa, V. De Ren- zis, K. Kritos, and A. Olejak, PRD 110, 063012 (2024), arXiv:2405.14945
2024 arXiv
-
[33]
Favata, C
M. Favata, C. Kim, K. G. Arun, J. Kim, and H. W. Lee, PRD 105, 023003 (2022), arXiv:2108.05861 [gr-qc]
2022 arXiv
-
[34]
Marsat, J
S. Marsat, J. G. Baker, and T. D. Canton, PRD 103, 083011 (2021), arXiv:2003.00357 [gr-qc]
2021 arXiv
- [35]
-
[36]
M. L. Katz, S. Marsat, A. J. K. Chua, S. Babak, and S. L. Larson, PRD 102, 023033 (2020), arXiv:2005.01827 [gr-qc]
2020 arXiv
-
[37]
Hartwig and M
O. Hartwig and M. Muratore, PRD 105, 062006 (2022), arXiv:2111.00975 [gr-qc]
2022 arXiv
-
[38]
Bayle, O
J.-B. Bayle, O. Hartwig, and M. Staab, PRD 104, 023006 (2021), arXiv:2103.06976 [gr-qc]. 7
2021 arXiv
-
[39]
C. J. Moore, D. Gerosa, and A. Klein, MNRAS 488, L94 (2019), arXiv:1905.11998
2019
-
[40]
Katz, “BBHx,” https://github.com/mikekatz04/ BBHx/blob/master/src/Response.cu (2024)
M. Katz, “BBHx,” https://github.com/mikekatz04/ BBHx/blob/master/src/Response.cu (2024)
2024
-
[41]
Okuta, Y
R. Okuta, Y. Unno, D. Nishino, S. Hido, and C. Loomis, in Proceedings (2017)
2017
-
[42]
S. K. Lam, A. Pitrou, and S. Seibert, in Proceedings (2015) pp. 1–6
2015
-
[43]
C. R. Harris et al., Nature 585, 357 (2020)
2020
-
[44]
J. D. Hunter, Computing in Science & Engineering 9, 90 (2007)
2007
-
[45]
mikekatz04/bbhx: New re- lease!
M. Katz and J. Roberts, “mikekatz04/bbhx: New re- lease!” (2023)
2023
-
[46]
SC Search code,
D. Bandopadhyay and C. J. Moore, “ SC Search code,” https://github.com/dig07/SC_Search (2024). Appendix A: Posterior plots Included here is an example of the posteriors produced by the rapid parameter estimation performed at the end of the search. Fig. 3 shows the posterior fo...
2024
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.