{"id":"89e09875-10fa-4c26-94bd-44f33f7a28c2","arxiv_id":"2607.15913","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A CNN trained on Q-transform images separates microlensed from unlensed precessing and non-spinning black-hole merger signals with 80–95% accuracy in simulated noise.","lead":"This paper trains neural networks to tell apart gravitational-wave signals magnified by microlensing from signals whose shape looks similar because of black-hole spin precession. It reports up to 95% accuracy in simulated Gaussian noise and 80–82% in real detector noise, but the real-event validation is indirect.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-noise test accuracies may be inflated by noise-segment leakage: 60,000 injections into ~8,000 overlapping noise segments with a signal-level train/test split.","rationale":"The reader's verdict of CONDITIONAL and the identified weaknesses (unverified simulation priors, circular real-data validation, missing code/data, inconsistent Q-transform settings) are reasonable. My stress-test identifies a more specific, load-bearing technical flaw in the real-noise evaluation: the construction of training and test images from a limited pool of overlapping noise segments without an explicit segment-disjoint split. This directly affects the headline accuracies claimed for real detector noise, which are the most important quantitative results for the paper's stated goal of a low-latency pipeline. The Gaussian-noise accuracies are less suspect because independent noise realizations are generated per signal, but the paper's strongest generalization claim—'82% in real noise'—is potentially inflated. The concern is concrete and testable: re-split by segment and re-evaluate. Since the issue can be addressed with a revised experimental setup, a conditional acceptance requiring this validation is appropriate, aligning with the reader's verdict but adding a sharper reason for the condition.","tokens_in":12404,"tokens_out":4161,"duration_ms":48511,"concrete_test":"Obtain the dataset generation metadata (or rerun the pipeline) and compute, for each real-noise image, the identifier of the underlying O4a noise segment. Split the data by segment ID so that no segment (or overlapping segment) contributes to both training and test sets—e.g., assign all injections sharing a segment start time to the same fold. Then retrain the real-noise CNN-2 models and evaluate on the disjoint test set. If the ML-vs-UP and ML-vs-UN accuracies drop materially below 82% and 80%, respectively, the original numbers were inflated by noise-segment leakage. Even a simpler diagnostic—checking the maximum Jaccard overlap of noise segment time windows between train and test images—will reveal whether leakage exists.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantitative claim—82% and 80% accuracy on real O4a noise—rests on a dataset construction that appears to permit leakage between training and test images. In §II A, ~32,000 s of H1 data are divided into 8-s segments with a 4-s overlap, yielding roughly 8,000 segments. The paper states that 20,000 signals per class (60,000 total) are injected into these segments, so each noise segment must be reused ~7–8 times. The train/validation/test split (12k/4k/4k per class) is described only as a per-signal split, not a per-segment split. If the same or overlapping noise segment appears in both the training and test sets, the CNN can memorize noise features rather than the injected signal morphology, artificially inflating the reported accuracies. This is especially acute for the real-noise model, where the same noise realization could appear with different injected signals in train and test. The Gaussian-noise model is less affected because noise is regenerated per signal, but the real-noise numbers (82% ML vs UP, 80% ML vs UN) are central to the claim that the pipeline works on real detector data. Without a segment-disjoint split, the headline accuracies are not trustworthy. The paper also provides no code or data to check this, and the real-event validation in §III B assumes all 124 O4a events are unlensed, so it cannot independently confirm the simulated accuracies.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates the degeneracy between gravitational-wave (GW) signals modulated by microlensing and those modulated by spin precession. The authors generate 20,000 simulated signals each for unlensed non-spinning (UN), unlensed precessing (UP), and microlensed non-spinning (ML) BBH systems, using IMRPhenomXPHM waveforms, wave-optics lensing through GWMAT, and GWTC-3-based population priors. They train convolutional neural networks on frequency-series and Q-transform spectrograms, with signals injected into either Gaussian noise or real O4a H1 detector noise. They report up to 95% accuracy in Gaussian noise and 82% in real noise for the ML-vs-UP task, and up to 80% accuracy in real noise for ML-vs-UN classification. They also find that UN-vs-UP classification remains difficult. The ML-vs-UN network is applied to 124 real O4a events, with the Gaussian-trained model classifying more events as UN than the real-noise-trained model. The paper claims to present the first low-latency machine-learning pipeline for distinguishing microlensed from unlensed non-spinning GW signals.","tokens_in":12723,"tokens_out":6378,"duration_ms":70603,"significance":"If the reported accuracies hold, this work would be a meaningful step toward rapid screening of GW events for microlensing signatures, addressing a real astrophysical degeneracy that complicates template-based searches and parameter estimation. The simulation setup is physically motivated: it incorporates wave-optics effects, realistic binary populations, and a network SNR threshold of 20. The misclassification corner plots (Figs. 6 and 7) are a useful diagnostic, identifying high impact parameter/low lens mass for ML signals and high total mass/extreme mass ratio for UN signals as the challenging regions. However, the central real-noise results are undermined by a likely dataset-construction problem (noise-segment leakage between train and test) and by inconsistent reported numbers. Because the paper does not ship code or data and does not report confidence intervals or repeated-seed variance, the robustness of the headline accuracies cannot currently be verified. The novelty claim—first wave-optics-based ML classification of microlensed vs precessing GW signals—is plausible given the cited prior work.","major_comments":[{"comment":"Dataset-construction leakage in the real-noise split. With ~32,000 s of H1 data divided into 8-s segments with 4-s overlap, there are ~8,000 noise segments. The 60,000 injections mean each segment is reused roughly 7–8 times. The train/validation/test split (12k/4k/4k per class) is described per signal, not per noise segment. This permits the same or overlapping noise realization to appear in both training and test images, allowing the CNN to memorize noise features rather than the injected signal morphology. This would artificially inflate the reported 82% (ML vs UP) and 80% (ML vs UN) real-noise accuracies. The authors must re-run the real-noise experiments with a segment-disjoint split (all injections sharing a segment assigned to the same split, or use non-overlapping segments) and report whether the accuracies change. Without this, the headline real-noise numbers are not trustworthy","section":"§II A"},{"comment":"The real-event evaluation assumes all 124 O4a events are unlensed, so any ML classification is counted as an error by construction. This test is consistent with the prior (no microlensing has been confirmed) but cannot validate classifier accuracy. Moreover, the two models were trained on different Q-transform representations (Gaussian: q-range 4–64, whiten=False; real: q-range 4–15, whiten=True), and the text does not specify which Q-transform settings were used to prepare the real events for each model. The comparison between the Gaussian-trained and real-noise-trained models (84 vs 66 events classified as UN) is therefore confounded by input representation. Please specify the exact preprocessing applied to real events, and either restrict the conclusions to what the test can support or conduct the dedicated background study already acknowledged in the text.","section":"§III B"},{"comment":"The reported real-noise accuracies are inconsistent: the abstract states '82% in real detector noise' (presumably for ML vs UP), while §IV states '80% for both these classifications' (ML vs UP and ML vs UN). The exact test-set accuracies and AUCs are not reported in §III; only ROC curves are shown. Please provide the numerical accuracy and AUC for each experiment and noise condition, reconcile the 82% vs 80% discrepancy, and include confidence intervals or standard deviations across training runs.","section":"Abstract / §IV"},{"comment":"The Gaussian and real-noise datasets use different Q-transform hyperparameters (q-range 4–64 vs 4–15, whiten=False vs True, highpass) and different strain amplitude windows (10^-26–10^-20 vs 10^-24–10^-20). Consequently, the reported performance gap between Gaussian (95%) and real noise (82%) does not isolate the effect of noise; the input representations differ in multiple ways. This should be controlled (identical preprocessing, changing only the noise) or explicitly discussed as a limitation when interpreting the headline comparison. As written, a reader cannot tell how much of the degradation is due to noise versus preprocessing.","section":"§II A"}],"minor_comments":[{"comment":"The sentence 'the signal peak occurs at t= sec' is missing a value; presumably t=0 s.","section":"§II A"},{"comment":"The table formatting is broken: 'ρ34.72' is missing a delimiter, 'ml' and 'yl' should be subscripted consistently (e.g., m_l, y_l).","section":"Table II"},{"comment":"'Fraction of misclassifications as a function CNN threshold' should be 'as a function of the CNN threshold'.","section":"Figure 8 caption"},{"comment":"The phrase 'ML signals (without precession' is confusing; clarify that the microlensed signals in this work are non-spinning and non-precessing.","section":"§IV"},{"comment":"No code or data repository is provided; the data availability statement says 'available upon reasonable request.' For a machine-learning paper, shipping code and trained models would greatly enhance reproducibility, especially given the complexity of the simulation pipeline.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The dataset-leakage issue in the real-noise experiments is the main technical concern; if the authors cannot demonstrate that a segment-disjoint split preserves the reported accuracies, the real-noise claims would be substantially weakened. The paper is otherwise within scope for the journal, but the authors should be encouraged to provide code/data and to clarify the inconsistencies in reported numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick heads-up: this paper trains a CNN on Q-transform images to separate wave-optics microlensed BBH signals from unlensed precessing and non-spinning ones. On simulated test sets it reports up to 95% accuracy in Gaussian noise and 82% in real O4a noise. The part to scrutinize before believing the real-noise numbers is the train/test split.\n\nThe paper describes ~32,000 s of H1 data cut into 8-s segments with 4-s overlap, then 60,000 injections — so each segment is reused about seven or eight times. The train/validation/test split is described as per-signal, not per-segment. If the same or overlapping noise segment lands in both training and test, the CNN can memorize noise features and the reported accuracies are inflated. This is the one load-bearing question for the paper's headline claim. It's fixable, but the paper ships no code or data, so I can't verify it either way.\n\nWhat is actually good: the wave-optics treatment of lensing instead of geometric optics, realistic GWTC-3-based population priors, and testing on real H1 O4a data. The authors also report negatives honestly — frequency-series inputs failed, UN vs UP separation is poor even in Gaussian noise, and the real-noise-trained model generalizes worse to real events than the Gaussian-noise-trained one. The corner plots showing where misclassifications concentrate (low lens mass, high impact parameter for ML; high mass and extreme mass ratio for UN) are genuinely informative.\n\nSofter spots in proportion: no error bars or repeated-seed variance on the accuracies; different Q-transform settings for Gaussian and real data make the two numbers not directly comparable; the real-event check assumes all 124 O4a events are unlensed, so it can't validate the classifier, only check consistency with prior — the authors mostly acknowledge this. The \"first pipeline\" claim should be scoped against Kim et al. 2021 and Kim & Liu 2023. Minor: the abstract says 82% real noise and the conclusion says 80% for both; reconcile.\n\nNone of this is fatal. The Gaussian-noise results are plausibly solid, the real-noise results are plausible but provisional. This deserves a serious referee, not a desk reject. I'd ask the authors to redo or confirm a segment-disjoint split for the real-noise data, release code and data, and report variance. A microlensing-search person will get real value from this paper even in its current form.","headline":"Useful ML application with a plausible central result, but the real-noise accuracies may be inflated by a non-disjoint noise-segment split; fix that and it deserves a serious referee.","tokens_in":13206,"tokens_out":3144,"would_cite":true,"duration_ms":33306,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A convolutional neural network trained on Q-transform time-frequency images can separate microlensed gravitational-wave signals from unlensed precessing and non-spinning ones, achieving up to 95% accuracy in Gaussian noise and about 80–82%","keywords":["gravitational waves","microlensing","spin precession","convolutional neural network","Q-transform","wave optics","LIGO","low-latency classification"],"falsifier":"A blinded injection campaign: bury simulated microlensed signals with known parameters in real H1 O4a noise, run the RaSMUNN pipeline at its optimal threshold, and check whether about 80% are recovered; if recovery falls well short, the claimed real-noise generalization fails. Alternatively, a real event with independent Bayesian evidence for wave-optics microlensing that the CNN classifies as unlensed would falsify the classifier's practical utility.","tokens_in":12283,"feed_emoji":"🌌","tokens_out":5871,"duration_ms":79360,"temperature":0.7,"pith_summary":"Microlensing by compact objects of 10–10^5 solar masses imprints a frequency-dependent beating on gravitational-wave chirps that can resemble the modulations from spin precession. This paper asks whether a machine-learning classifier can tell the two apart, and answers yes on simulated data: a convolutional network fed Q-transform spectrograms separates microlensed from unlensed non-spinning and precessing signals with up to 95% accuracy in Gaussian noise and roughly 80–82% in real O4a detector noise. The authors build a low-latency pipeline, RaSMUNN, to apply the microlensed-versus-unlensed classifier to real events, and find that the model trained on Gaussian noise generalizes better than the one trained on real noise when tested on 124 O4a events. If the result holds, it offers a fast screen for microlensing candidates before expensive Bayesian parameter estimation, and it highlights a degeneracy that template-based searches must handle.","feed_headline":"CNN separates microlensed from unlensed GW signals, up to 95%","feed_subtitle":"Time-frequency images let a classifier tell wave-optics microlensing from spin precession in simulated LIGO noise.","key_machinery":"The load-bearing object is the Q-transform spectrogram, a two-dimensional time-frequency image that preserves the characteristic beating and modulation patterns of each signal class. A six-block convolutional neural network with max-pooling, dropout, and a softmax output reads these images and produces a probability for the unlensed class; the pipeline RaSMUNN wraps this trained classifier for direct use on detector data. Signals are simulated with a precessing-binary waveform model, wave-optics microlensing transfer functions for lens masses 10–10^5 solar masses, population priors fitted to observed binary-black-hole mergers, projection onto a single detector, and a network SNR threshold of","core_discovery":"On the paper's own terms, the central discovery is that wave-optics microlensing features in the time-frequency plane are learnable and distinguishable from spin-precession features and from ordinary non-spinning chirps, even when signals are buried in non-stationary real detector noise. The authors show this by training a six-block convolutional neural network on Q-transform images of 20,000 simulated signals per class, with realistic population priors and SNR above 20, and reporting accuracies of up to 95% in Gaussian noise and about 80–82% in real noise for microlensed versus unlensed classes. They also map the failure regions—mild lensing with high impact parameter and low lens mass, and","pith_inferences":["Because all 124 real O4a events are treated as unlensed by construction, the real-data evaluation measures threshold behavior and consistency, not true sensitivity; a dedicated background study with hundreds of known-unlensed events and blind injections is needed before the reported accuracies can be quoted for real data.","The misclassification maps suggest a natural follow-up: train a version of the network to label the uncertain regions—high impact parameter, low lens mass, high total mass, extreme mass ratio—as low-confidence rather than forcing a binary choice, guiding human review toward those candidates.","The same Q-transform CNN approach could be extended to eccentric binaries, which also produce amplitude and phase modulations, or to a multi-class setup that includes signals affected by both precession and microlensing simultaneously.","A testable extension is to use the network as a trigger in low-latency searches and compare its candidates against Bayesian evidence ratios for wave-optics lensing; a statistically significant overlap would independently validate the features the network has learned."],"forward_implications":["RaSMUNN can act as a low-latency screen, flagging microlensing candidates within seconds of a detection and reducing the need for full Bayesian parameter estimation on every event.","The claimed degeneracy implies that template-based searches and parameter-estimation pipelines that ignore either precession or microlensing risk biased inferences; a classifier of this kind could serve as a pre-filter.","If the roughly 80–82% real-noise accuracy transfers to the growing O4/O5 event catalogs, the pipeline could help identify the first microlensed gravitational-wave event or tighten constraints on compact-object lens populations.","The poor UN-versus-UP classification marks the method's limit: precession imprints are too subtle for this architecture, so separating precessing from non-spinning signals will need different representations or models.","The better generalization of the Gaussian-noise-trained model over the real-noise-trained model on real events suggests that training on simulated stationary noise can be more robust than training on a finite segment of real noise."],"fun_headline_variants":["CNN sniffs out microlensed gravitational waves","Neural net tells lensed from unlensed GW signals","Machine learning catches microlensing in LIGO noise","AI spots lensed gravitational waves in real noise","Deep learning separates GW microlensing signals"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing assumption is that the simulated training family—wave-optics microlensing with lens masses 10–10^5 solar masses, time delays under 0.15 seconds, realistic population priors, single-detector projection, and SNR above 20—matches the shapes of real microlensed signals in real detector noise; this is untested by the real-event evaluation because all 124 O4a events are treated as unlensed.","fun_headline_variants_meta":{"raw":{"variants":["CNN sniffs out microlensed gravitational waves","Neural net tells lensed from unlensed GW signals","Machine learning catches microlensing in LIGO noise","AI spots lensed gravitational waves in real noise","Deep learning separates GW microlensing signals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000241,"raw_usage":{"total_tokens":1400,"prompt_tokens":825,"completion_tokens":575,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":501}},"tokens_in":569,"tokens_out":575,"duration_ms":5713,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T21:56:12.580424+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A blinded injection campaign: bury simulated microlensed signals with known parameters in real H1 O4a noise, run the RaSMUNN pipeline at its optimal threshold, and check whether about 80% are recovered; if recovery falls well short, the claimed real-noise generalization fails. Alternatively, a real event with independent Bayesian evidence for wave-optics microlensing that the CNN classifies as unlensed would falsify the classifier's practical utility.","supporting_citations":[],"review_version":1}