{"id":"1e2bd568-fdba-4587-b088-42262151b9c7","arxiv_id":"1908.04996","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"The LLAMA pipeline searched for coincidence between gravitational waves and neutrinos during LIGO/Virgo's O3 run and found no significant joint signals.","lead":"This paper reports the status of a real-time search that looks for high-energy neutrinos arriving at the same time and from the same direction as gravitational-wave events detected by LIGO and Virgo. It finds no significant coincident events among the first 14 gravitational-wave alerts of the third observing run.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Null result is not fully supported: p-values for the three BNS O3 candidates are omitted and all quoted p-values are preliminary, so the 'no p<1%' claim cannot be checked for those events.","rationale":"The paper is a proceedings that reports a null result. The authors are transparent that BNS p-values are omitted and that BBH p-values are preliminary. That transparency is a point in favor of the paper, but it also means the central claim cannot be independently checked from the text. A conference proceeding does not need to rederive every prior, and the Gaussian assumption may be acceptable for this purpose, but the strongest claim ('no p<1%') is exactly the part that depends on the one set of numbers not shown. The concrete check is obtainable from the collaboration, so the concern is resolvable. Because the existing verdict is already CONDITIONAL, this stress-test does not move it; it reinforces the need to treat the null result as provisional.","tokens_in":5746,"tokens_out":5267,"duration_ms":49438,"concrete_test":"Request or recompute the BNS p-values for S190425z, S190426c, and S190510g using the published LLAMA/TS calibration from refs. [11,13] with the revised BNS priors (or with the O3-era priors if the revision is not final). If any of the three p-values is below 0.01, the headline null result is invalid for the BNS class. If all three are above 0.01, the conclusion survives but Table 1 should be updated to include the numbers. A secondary check: re-run the BBH p-values replacing the Gaussian neutrino directional PDF in Sec. 3 with the empirical IceCube reconstruction PDF; if any p-value shifts from above 0.01 to below 0.01, the assertion in Sec. 4 needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—no GW+HEN coincidence with p<1% in the first 14 O3 alerts (Sec. 4, Sec. 7)—is only partially supported by Table 1. Three BNS candidates (S190425z, S190426c, S190510g) have no p-value reported; the table note says BNS p-values are omitted because the BNS distributions are being revisited, and the same note labels the BBH p-values preliminary. Since Eq. 3.1 and the TS calibration in Sec. 3 depend on source priors (uniform distance, log-uniform energy), a change in the BNS distance/energy distribution can shift TS thresholds and p-values for exactly those events. The reader therefore cannot verify the null claim for the BNS subset, and if any of these three events has p<1% under the revised priors, the conclusion in Sec. 4/7 would be false. The Gaussian directional-error assumption in Sec. 3 is a separate, more speculative issue; it is not the primary gap because the reported BBH p-values (0.12–0.83) are far from threshold, while the missing BNS p-values are an unquantified hole in the stated result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript, a proceedings contribution from the IceCube Collaboration, describes the Low-Latency Algorithm for Multi-messenger Astrophysics (LLAMA) pipeline for searching for coincident gravitational-wave (GW) and high-energy neutrino (HEN) events. The pipeline combines LIGO/Virgo public alerts with IceCube GFU data, uses a Bayesian Bayes-factor statistic introduced in prior work, and has been running in realtime since the start of O3. For the first 14 GW alerts, the paper reports p-values for 11 BBH-type candidates, with three BNS-type candidates (S190425z, S190426c, S190510g) omitted because the BNS distributions are under revision. The paper claims that no event with p-value < 1% has been found, and provides a discovery potential estimate: a fluence of 0.049 GeV/cm^2 is detectable with 50% probability at the 3-sigma level.","tokens_in":6117,"tokens_out":3769,"duration_ms":37003,"significance":"If the null result is taken at face value, it is a useful realtime constraint on joint GW+HEN emission and demonstrates the operational readiness of the LLAMA pipeline. The paper is transparent about the preliminary nature of the analysis, references the detailed derivation of the statistical method, and presents the data in a compact table. It also highlights the importance of choked-jet scenarios for BNS mergers, where neutrinos may be the only escaping messenger. However, the strength of the central claim is weakened by the omission of the BNS p-values and by the absence of systematic uncertainties on the discovery potential; these are directly relevant to the conclusions drawn in Sections 4 and 7.","major_comments":[{"comment":"The central claim that 'no event with a p-value < 1% have been found to date' cannot be fully checked for the three BNS candidates (S190425z, S190426c, S190510g) because their p-values are omitted from Table 1. The table note states that BNS p-values are not reported because the BNS distributions are being revisited, but the Bayes factor in Eq. (3.1) depends on source priors such as distance (uniform in volume) and energy (log-uniform), so a change in BNS priors could shift the TS distribution and the p-values for exactly these events. The conclusion in Section 7 should be explicitly restricted to the BBH candidates, or the BNS p-values should be reported even if marked preliminary, so that the reader can independently verify the null claim.","section":"Section 4, Table 1"},{"comment":"The p-values quoted in Table 1 are conditional on the prior assumptions adopted in the Bayesian calculation, including the Gaussian directional-error model for neutrinos and the uniform-distance/log-uniform-energy source priors. The paper does not quantify how sensitive the p-values are to these assumptions. This is not just an academic concern, because the manuscript itself states that the BNS distributions are being revisited; if the revised priors change the TS calibration, previously borderline p-values could cross the 1% threshold. The authors should report the p-values under the current priors and also state explicitly which systematic variations are under consideration, ideally with a quantitative reassessment of the three BNS candidates.","section":"Section 3, Eq. (3.1) and Section 4"},{"comment":"The discovery potential of 0.049 GeV/cm^2 is reported as a single-point estimate with no statistical or systematic uncertainty. The value depends on the assumed E^-2 spectrum, the population of injected GW sources, the background scrambling, and the prior choices, none of which are varied or discussed. The curve in Figure 2 would be much more informative if the authors added a systematic band (e.g., varying the spectral index or the source-distance prior) or at least explicitly stated the dominant sources of uncertainty. Without this, the reader cannot judge how robust the quoted sensitivity is.","section":"Section 5, Figure 2"}],"minor_comments":[{"comment":"The phrase 'searches for HENs in a time window of 1000 s of the GW detected time' is grammatically awkward; it should read 'within 1000 s of the GW trigger time'.","section":"Section 2"},{"comment":"The sentence 'For the case of GW public alerts, H0 is zero' is ambiguous. It should be made explicit that the probability of the null hypothesis P(H0) is set to zero because the GW alert is a confirmed astrophysical event, rather than a background fluctuation.","section":"Section 3"},{"comment":"The caption states 'None of the neutrinos made a significant coincidence' — the subject-verb agreement is incorrect; it should be 'No neutrino made a significant coincidence'.","section":"Figure 1 caption"},{"comment":"The column heading 'p-value (binary merger)' is misleading; it refers to the p-value from the GW+HEN search, not to a property of the binary merger itself. A heading such as 'GW+HEN p-value (preliminary)' would be clearer.","section":"Table 1"},{"comment":"The definition of discovery potential is given, but the x-axis of Figure 2 is not described in the text. The reader has to infer the units and range from the figure; a brief description in the caption or text would improve clarity.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"This is a conference proceedings paper, so the bar for statistical completeness is lower than for a full journal article. However, the central null claim is made in the abstract and conclusions and should be defensible as stated. The omission of the three BNS p-values creates a hole in that claim, and the lack of systematic uncertainties on the discovery potential is a weakness that can be addressed with modest additions. I recommend major revision rather than rejection, as the issues are fixable within the scope of the manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a conference proceedings status report, not a discovery paper. The genuinely new content is that the LLAMA pipeline ran on the first 14 O3 LIGO/Virgo alerts and found no GW+neutrino coincidence with p<1%; the table lists p-values for the 11 BBH-type events; the updated discovery potential is a fluence of 0.049 GeV/cm2 at 50% detection probability. The Bayesian search formalism is not new—it is taken from the authors' earlier papers, chiefly refs. [11] and [13], and this paper doesn't re-derive it.\n\nThe paper does its job honestly. It labels the p-values 'preliminary,' says the BNS p-values are omitted because those distributions are being revisited, and it makes clear the analysis is an operational report. The realtime pipeline exists, ran, and the null result is the result. The discovery-potential curve is a useful benchmark for the field, and the prose about future prospects is measured.\n\nThe soft spots are proportionate. The most concrete one is that the headline claim 'no event with p<1%' cannot be checked for the three BNS candidates, because their p-values are missing. The authors note why, and in a proceedings that may be acceptable, but Sec. 4 and the conclusion repeat the claim without that caveat. If a revised BNS prior moves one of those three events below 1%, the sentence would be wrong. That is a genuine reporting gap, not a fatal flaw. The other caveat is that the quoted significances and the TS calibration rest on assumptions—Gaussian directional errors, uniform-in-distance and log-uniform-in-energy priors—that are only referenced, not validated here. The BBH p-values are all far from threshold, so this does not change the practical conclusion, but it does mean the numbers should be treated as provisional.\n\nWho is this for? Groups doing multimessenger follow-up, people building low-latency pipelines, and anyone who wants the current O3 joint-search sensitivity number. It is not a complete derivation and it is not a major discovery. It deserves a serious referee if submitted to an archival journal—the caveats should be addressed, but the operational null result over 14 alerts is a legitimate, useful data point.","headline":"A short, honest proceedings status report: real operational null result for 14 O3 alerts, but the BNS p-values are missing, so the headline claim is under-supported.","tokens_in":6507,"tokens_out":2865,"would_cite":false,"duration_ms":28910,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The LLAMA pipeline found no statistically significant joint gravitational-wave and high-energy neutrino event in the first 14 LIGO/Virgo O3 alerts, with all reported p-values above 1%.","keywords":["multi-messenger astronomy","gravitational waves","high-energy neutrinos","LIGO","Virgo","IceCube","LLAMA pipeline","binary neutron star mergers"],"falsifier":"Recompute the joint p-values for the 14 O3 alerts once the revised binary-neutron-star source distributions are adopted. If any of the withheld BNS candidates (S190425z, S190426c, S190510g) then yields a combined p-value below 1%, or if any currently reported p-value shifts below 1% under the updated priors, the paper's claim of no significant joint detection would be overturned.","tokens_in":5563,"feed_emoji":"🔭","tokens_out":7771,"duration_ms":64425,"temperature":0.7,"pith_summary":"This paper reports on the LLAMA pipeline, which searches in real time for high-energy neutrinos from IceCube that are coincident with LIGO/Virgo gravitational-wave candidates, and presents results from the first 14 alerts of the third observing run (O3). The search found no joint event with a p-value below 1%, so no statistically significant gravitational-wave plus high-energy neutrino detection has been made to date. The paper also quantifies the search's sensitivity: a neutrino fluence of about $0.049\\ \\mathrm{GeV/cm^2}$ is detected with 50% probability when averaged over the sky. If the approach works, joint detections could localize sources to sub-degree precision and probe the physics of relativistic outflows from neutron star mergers even when electromagnetic emission is hidden.","feed_headline":"No joint GW–neutrino event found in 14 O3 alerts","feed_subtitle":"Bayesian search across LIGO/Virgo and IceCube data gives p-values above 1% for every candidate; BNS results await revised priors.","key_machinery":"The load-bearing object is the LLAMA pipeline and its Bayesian test statistic, the odds ratio $O_{gw+\\nu}$. The pipeline pulls public gravitational-wave alerts and IceCube's low-latency neutrino stream, searches a $\\pm 500$ s window for coincident neutrinos, and computes the odds ratio using priors of uniform sky position, uniform-in-volume distance, and log-uniform energy for both messengers, together with a Gaussian model for neutrino directional errors. The odds ratio is converted to a p-value by comparing it with a background distribution generated from scrambled neutrino data, and the significance threshold for a public alert is a p-value below 1%.","core_discovery":"Using a Bayesian framework that incorporates astrophysical priors for both messengers, the LLAMA pipeline computes an odds ratio $O_{gw+\\nu}$ comparing the hypothesis that a gravitational wave and at least one high-energy neutrino come from the same astrophysical source against the null and chance-coincidence hypotheses. For each of the 14 O3 gravitational-wave alerts, this odds ratio is converted to a p-value through a background test-statistic distribution built from scrambled neutrino data. None of the resulting p-values fell below 1%, so the search reports no significant joint detection. The p-values for the three binary-neutron-star candidates are not reported because the BNS source distributions are being revised, and the p-values for the black-hole candidates are described as preliminary.","pith_inferences":["Because most of the 14 O3 alerts are binary black holes, which are not expected to be strong neutrino emitters, this non-detection is not yet a strong test of neutron-star jet physics; the withheld BNS p-values carry that test.","If the revised BNS priors shift any of the three BNS-candidate p-values below 1%, the paper's headline conclusion of no significant detection could reverse, a check that becomes available as soon as the new distributions are adopted.","The sub-degree neutrino localization advantage suggests that the most promising near-term chance for a first joint detection is a BNS or neutron-star–black-hole merger in the northern sky, where IceCube's effective area is larger and atmospheric background is lower.","Applying the same Bayesian search to the full completed O3 dataset, rather than only the real-time alerts, would provide a more sensitive all-sky search and could supersede the preliminary numbers reported here."],"forward_implications":["With no event below a p-value of 1% in the first 14 O3 alerts, the search currently yields upper limits on high-energy neutrino fluence for these gravitational-wave sources.","A future alert with a p-value below 1% will trigger GCN circulars reporting neutrino directions, angular uncertainties, time offsets, and p-values from both the LLAMA and companion maximum-likelihood searches, enabling prompt electromagnetic follow-up with sub-degree localization.","For binary neutron star mergers, joint searches can reveal choked-jet physics through neutrinos that escape the ejecta even when electromagnetic emission is absorbed, and non-detections can place limits on choked-jet model parameters.","The quoted discovery potential of about $0.049\\ \\mathrm{GeV/cm^2}$ corresponds to a 50% detection probability averaged over the sky, indicating the search is sensitive to nearby, strongly emitting events."],"supporting_citations":[{"why":"Supplies the Bayesian formalism and odds-ratio derivation used to compute joint significance.","marker":"[13]"},{"why":"Describes the LLAMA pipeline and its analysis steps, including prior choices and background calibration.","marker":"[11]"},{"why":"Provides the original GW+HEN search method that LLAMA was adapted from.","marker":"[12]"},{"why":"Supplies the IceCube gamma-ray follow-up real-time neutrino stream used in the search.","marker":"[16]"},{"why":"The companion maximum-likelihood search whose p-values are reported alongside LLAMA in GCN circulars.","marker":"[15]"},{"why":"Provides the choked-jet neutrino emission models that motivate the BNS search prospects.","marker":"[10]"}],"fun_headline_variants":["No joint GW–neutrino source found in O3 alerts","O3 multi-messenger search yields no coincident events","All 14 O3 alerts show no GW-neutrino coincidence","LLAMA null result: no shared GW-neutrino sources","Bayesian search finds no joint GW-neutrino in O3"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quoted significances rest on the Bayesian model and background calibration developed in the two earlier papers, which assume Gaussian neutrino direction errors, a uniform-in-volume distance prior, and log-uniform energy priors; the paper itself notes that the binary-neutron-star distributions are being revisited, so the BNS p-values could change.","fun_headline_variants_meta":{"raw":{"variants":["No joint GW–neutrino source found in O3 alerts","O3 multi-messenger search yields no coincident events","All 14 O3 alerts show no GW-neutrino coincidence","LLAMA null result: no shared GW-neutrino sources","Bayesian search finds no joint GW-neutrino in O3"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000732,"raw_usage":{"total_tokens":3245,"prompt_tokens":888,"completion_tokens":2357,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":2269}},"tokens_in":504,"tokens_out":2357,"duration_ms":16734,"temperature":1.0,"reasoning_tokens":2269,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:25:24.775666+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the joint p-values for the 14 O3 alerts once the revised binary-neutron-star source distributions are adopted. If any of the withheld BNS candidates (S190425z, S190426c, S190510g) then yields a combined p-value below 1%, or if any currently reported p-value shifts below 1% under the updated priors, the paper's claim of no significant joint detection would be overturned.","supporting_citations":[{"cited_title":"Baret et al., Phys","cited_arxiv_id":null,"evidence_quote":"Provides the original GW+HEN search method that LLAMA was adapted from."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the IceCube gamma-ray follow-up real-time neutrino stream used in the search."},{"cited_title":"Hussain et al., PoS(ICRC2019)859 (these proceedings)","cited_arxiv_id":null,"evidence_quote":"The companion maximum-likelihood search whose p-values are reported alongside LLAMA in GCN circulars."},{"cited_title":"Kimura et al., Phys","cited_arxiv_id":null,"evidence_quote":"Provides the choked-jet neutrino emission models that motivate the BNS search prospects."}],"review_version":1}