{"id":"e0af0fd6-f593-428a-a5f1-2c564d4ceda5","arxiv_id":"1908.05526","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"A five-year search for neutrino flares from 1,023 blazars finds no significant population of neutrino-emitting blazars; only the previously known TXS 0506+056 stands out.","lead":"IceCube searched five years of neutrino data for flares from over a thousand blazars and found no statistically significant excess beyond random background. The only blazar with detected neutrino activity remains TXS 0506+056, so emitting blazars appear to be rare.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified","rationale":"The reader identified the validity of scrambled data as the weakest assumption. I agree that this is the most subtle part of the analysis, but I find that the paper handles it properly by calibrating the final p-value against scrambled data. The final p-value is an empirical quantile of the data's test statistic in the scrambled distribution, so residual artifacts in the binomial test or decorrelation are absorbed as long as the scrambles are representative of the null. I searched for specific flaws, such as the heuristic correlation criterion or the arbitrary choice of k scan range, but these are applied uniformly to data and background, and the calibration ensures the result is not overstated. The analysis also reports results with and without the motivating source, strengthening the conclusion. Thus, the central claim is supported and the verdict should remain ACCEPT.","tokens_in":164,"tokens_out":8184,"duration_ms":107238,"concrete_test":"Run a Monte Carlo validation: generate many scrambled data sets from the actual event sample. For each scrambled set, treat it as a pseudo-data set, run the full analysis pipeline (per-direction p-value computation, binomial test, decorrelation, and final calibration against a separate ensemble of scrambles) to obtain a final p-value. Check that the distribution of these final p-values is uniform on (0,1). If the distribution deviates significantly from uniformity, the calibration is biased and the reported p-values would be unreliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that no significant excess of neutrino flares from 3LAC blazars is observed, with calibrated p-values of 0.114 (with TXS) and 0.244 (excluding TXS). The most plausible vulnerability is the validity of scrambled data as the null distribution. However, the paper does not rely solely on the raw binomial test; it calibrates the final p-value by comparing the data's decorrelated binomial p-value to the distribution obtained from the same analysis on many scrambled data sets. This is a standard Monte Carlo calibration that absorbs any residual non-uniformity caused by correlations or trial factors. The decorrelation procedure, while ad hoc, is applied identically to data and scrambles, so the final p-value remains a valid frequentist p-value under the null hypothesis that scrambling faithfully represents background. No specific failure of the scrambling assumption was identified, and the conclusion is robust to the choice of including or excluding TXS 0506+056 and to the FSRQ/BLLac sub-catalogs. Therefore, I do not find a load-bearing flaw in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a search for time-dependent neutrino emission from blazars in the Fermi 3LAC catalog, using 5 years of IceCube muon-neutrino data. For each of 1023 catalog directions, the best-fit neutrino flare is found with an unbinned maximum-likelihood method, and a p-value is assigned by comparing the test statistic to scrambled data. A binomial test is then applied to the set of p-values to look for an excess of low p-values beyond the background expectation. Correlated search directions are removed with a heuristic prescription, and the final binomial p-value is calibrated by repeating the entire procedure on scrambled data. The resulting final p-values are 0.114 with TXS 0506+056 included and 0.244 with it excluded, consistent with the background-only hypothesis. Sub-catalog tests for FSRQ and BL Lac objects also show no significant excess. The most significant fitted flare is TXS 0506+056, with a local p-value of 3.47e-5.","tokens_in":5717,"tokens_out":9756,"duration_ms":92440,"significance":"If the result is correct, it places a meaningful constraint on the fraction of 3LAC blazars with flaring neutrino emission: the data show no evidence for a population of neutrino-bright blazars beyond the known TXS 0506+056. The analysis is a clear, well-calibrated null result. Its main strength is the use of scrambled data not only to assign individual p-values but also to calibrate the final binomial p-value, which absorbs the trial factor from scanning over k and the effects of correlations. The paper is transparent in reporting results with and without TXS 0506+056 and for the two optical sub-classes. The individual fit parameters in Table 1 are useful for follow-up. The central claim is defensible and appropriately limited.","major_comments":[],"minor_comments":[{"comment":"The algorithm for determining how many AGNs in the top k need to be removed is not fully specified; please describe the exact removal procedure (for example, greedy removal or maximum independent set) so that the analysis is reproducible.","section":"2.3"},{"comment":"The threshold for a 'significant neutrino contributor' is defined as the top ns×3 neutrinos by Si/Bi; the factor of 3 appears arbitrary. Please provide a justification or demonstrate that the final p-value is insensitive to this choice.","section":"2.3"},{"comment":"The scan over the 100 lowest p-values is an arbitrary upper bound; please state whether the final result changes if the scan is restricted to, for example, k=1..50 or extended to k=1..200, since the calibration only absorbs the scan range actually used.","section":"2.2"},{"comment":"The scrambling procedure is described only as 'randomized positions and times'; please specify which components of the data are preserved (for example, declination, energy, livetime) or cite a reference that does so, because the null distribution depends on this.","section":"2.1"},{"comment":"The raw binomial p-value for k=11 before decorrelation is not stated; reporting it would make the effect of the correlation-removal step explicit and would help the reader interpret the size of the correction.","section":"3.1"},{"comment":"In the sentence describing the signal PDF, 'spacial' should be 'spatial'.","section":"2.1"},{"comment":"The axes in Figure 3 are not labeled in the manuscript; adding x- and y-axis labels would aid interpretation.","section":"Figure 3"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a well-behaved null result from the first systematic search for time-dependent neutrino flares across the northern-sky 3LAC catalog. The likelihood machinery is taken from earlier IceCube papers, so methodological novelty is low, but the survey itself is new: they fit the most significant flare in each of 1023 directions, apply a binomial test to the p-values, decorrelate nearby directions that share events, and calibrate the final p-value against scrambled data. The headline number is p=0.244 excluding TXS 0506+056 (0.114 including), consistent with background. That is a defensible statement that the 5-year muon-neutrino sample contains no significant population of blazar neutrino flares.\n\nWhat the paper does well: the calibration is careful. Rather than quoting the raw binomial p-value, they compare it to the distribution of the same decorrelated statistic on scrambled data, which absorbs residual non-uniformities from the decorrelation procedure. They report with and without the motivating source, and they separately test FSRQ and BL Lac sub-catalogs. The per-object fit table is useful.\n\nSoft spots: the decorrelation recipe (top ns*3 neutrinos, reduce k by the number of correlated directions) is ad hoc, though applying it identically to data and scrambles makes the final p-value valid under the null. The scan over k=1..100 is a choice, not derived from a prior on population size. More substantively, no detection sensitivity is quoted, so the null result is not as sharp as it could be—readers can't tell what fraction of a putative blazar population would have been excluded. The analysis code and data are not public, which limits reproducibility, though the method is documented in prior papers. The authors themselves flag the non-blazar counterpart observation as post hoc, which is the right call.\n\nNone of this undermines the central claim. The scrambled-data null is the main vulnerability, but no specific failure of that assumption surfaces here, and the final calibration is a standard Monte Carlo move. This is a valid contribution for multimessenger astrophysics, especially as a companion to the TXS 0506+056 results. It deserves a serious referee; the main requests would be sensitivity numbers and more detail on the decorrelation. I'd cite it if I were working on blazar neutrino searches.\n\nRecommendation: send to peer review.","headline":"A careful, calibrated null result from the first systematic time-dependent neutrino flare search over the northern 3LAC catalog; method is borrowed but the survey and its ensemble calibration are new.","tokens_in":6275,"tokens_out":2379,"would_cite":true,"duration_ms":22073,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper finds no statistically significant population of blazar neutrino flares in five years of IceCube muon-neutrino data, with final p=0.244 excluding TXS 0506+056 and p=0.114 including it.","keywords":["blazars","neutrino flares","muon neutrinos","IceCube","time-dependent analysis","unbinned maximum likelihood","binomial test","3LAC catalog"],"falsifier":"Run the identical pipeline on simulated data in which ten of the 1023 blazar directions each inject a five-event Gaussian neutrino flare: if the decorrelated binomial p-value is not systematically smaller than the scrambled-background distribution, then the analysis cannot detect the population it appears to exclude, and the reported null result would not constrain small populations.","tokens_in":5368,"feed_emoji":"🔭","tokens_out":12143,"duration_ms":103799,"temperature":0.7,"pith_summary":"After the 2017 coincidence between a high-energy neutrino and the flaring blazar TXS 0506+056, the question is whether other blazars also produce neutrino flares. This paper applies the same time-dependent flare search used to find the 2014-15 neutrino flare from TXS 0506+056 to every one of the 1023 northern-sky blazar and AGN directions in the Fermi 3LAC catalog, fitting the most significant neutrino flare in each direction without reference to gamma-ray light curves. It then asks whether the distribution of the 1023 best-fit p-values contains more extremely small values than background would produce, using a binomial test designed to catch a small population of neutrino emitters. The test finds no such excess: after removing correlated directions, the final significance is $p=0.244$ when TXS 0506+056 is excluded and $p=0.114$ when it is included, both consistent with atmospheric background. If correct, the result means that five years of IceCube muon-neutrino data show no evidence that blazars as a class are episodic neutrino emitters beyond TXS 0506+056.","feed_headline":"Blazar flare hunt finds no neutrino excess in five years","feed_subtitle":"A binomial test over all 1023 catalogued blazar directions gives p=0.244 without TXS 0506+056.","key_machinery":"The engine of the search is the unbinned maximum-likelihood flare fit of equations (2.1)-(2.4): a likelihood in which a signal component with spatial, energy, and Gaussian time terms (equation 2.2) is mixed with an atmospheric-background term (equation 2.3), and the test statistic $D$ penalizes short time windows through the livetime-to-window ratio $\\mathrm{TL}/\\hat{T}_\\sigma$ to account for the increased trial factor. Its output is a best-fit number of signal events, spectral index, flare time, and flare width, which is converted to a local p-value against scrambled data. The second piece of machinery is the binomial test of equation (2.5), which scans the 100 smallest ordered p-values and asks whether $k$ or more blazars have p-values at or below $p_k$. The third piece is a correlation-removal prescription: neutrino events are ranked by signal-to-background ratio $S_i/B_i$, two search directions are called correlated if they share any of the top $3\\hat{n}_s$ neutrinos, and $k$ is reduced by the number of redundant directions before the p-value is recomputed. Finally, the entire decorrelated-binomial procedure is run on scrambled data to produce the null distribution against which the data's decorrelated p-value is judged.","core_discovery":"On the paper's own terms: for each of the 1023 search directions, the best-fit neutrino flare is found by maximizing an unbinned likelihood in which the signal is a spatially Gaussian, energy-power-law, time-Gaussian burst and the background is atmospheric muon neutrinos; the test statistic penalizes short time windows for their larger trial factor. The p-value of each direction's best fit is calibrated against scrambled-data ensembles. The smallest binomial p-value over the 100 lowest p-values occurs at $k=11$, with local p-value $3.27\\times10^{-3}$. Two correlation clusters are identified in those 11 directions (three directions around B2 1126+37 and two around GB6 J0929+5013) and removed by lowering $k$ to 8, giving a decorrelated binomial p-value of $2.09\\times10^{-2}$ including TXS 0506+056 and $5.33\\times10^{-2}$ without it. When these decorrelated values are compared with the same quantity from scrambled data, the final p-values are $0.114$ and $0.244$, respectively, so the data are consistent with background. The same test applied separately to FSRQ and BLLac sub-catalogs gives final p-values of $0.50$ and $0.60$.","pith_inferences":["A natural extension the paper leaves implicit: the same catalog scan with time windows fixed by gamma-ray flares could be more sensitive to blazars that emit neutrinos only while gamma-bright, since the present search deliberately fits time without any electromagnetic trigger.","Because the final p-value comes from comparing the data's decorrelated binomial p-value with scrambled skies, the claim is about the population of detectable flares: a population whose flares are too weak to enter the top 100 p-value list would be invisible to this test.","One testable improvement would be to run the same binomial test on an independent half of the data to check that the reported p=0.244 reproduces."],"forward_implications":["At p=0.244 excluding TXS 0506+056, the observed set of per-direction flare p-values is consistent with atmospheric-neutrino background alone.","The eight uncorrelated directions that survive decorrelation in the top 11 do not constitute a significant population; the analysis places no limit on individual sources but bounds the collective excess.","FSRQ-only and BLLac-only sub-catalogs each return final p-values consistent with background (0.50 and 0.60).","The three non-blazar objects among the top 11 are a small a posteriori fraction, and the paper treats them as motivation for follow-up rather than evidence."],"supporting_citations":[{"why":"reports the 2017 high-energy neutrino coincident with TXS 0506+056 that motivates the catalog search.","marker":"[1]"},{"why":"reports the 3.5-sigma archival 2014-15 neutrino flare from TXS 0506+056, the signal the same algorithm previously found and the source excluded in one version of the final p-value.","marker":"[2]"},{"why":"supplies the Fermi 3LAC catalog of blazar directions used as the 1023 search positions.","marker":"[5]"},{"why":"supplies the five-year muon-neutrino track sample analyzed here.","marker":"[6]"},{"why":"introduces the unbinned maximum-likelihood time-dependent flare search whose likelihood ratio forms the test statistic.","marker":"[7]"},{"why":"extends that method with energy information and the trial-factor-corrected test statistic used to obtain per-direction p-values.","marker":"[8]"}],"fun_headline_variants":["IceCube blazar flare search finds no neutrino excess","No time-dependent neutrino signal from blazar catalog","Blazar neutrino flare hunt yields only background noise","IceCube survey of 1023 blazars sees no neutrino flares","Null result: no neutrino flares from Fermi blazar list"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on scrambled events faithfully reproducing the atmospheric-neutrino background: if scrambling distorts detector acceptance, energy response, or the spatial-time correlation structure, every reported p-value is biased.","fun_headline_variants_meta":{"raw":{"variants":["IceCube blazar flare search finds no neutrino excess","No time-dependent neutrino signal from blazar catalog","Blazar neutrino flare hunt yields only background noise","IceCube survey of 1023 blazars sees no neutrino flares","Null result: no neutrino flares from Fermi blazar list"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000148,"raw_usage":{"total_tokens":1239,"prompt_tokens":1044,"completion_tokens":195,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":116}},"tokens_in":660,"tokens_out":195,"duration_ms":2821,"temperature":1.0,"reasoning_tokens":116,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:10:37.965464+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical pipeline on simulated data in which ten of the 1023 blazar directions each inject a five-event Gaussian neutrino flare: if the decorrelated binomial p-value is not systematically smaller than the scrambled-background distribution, then the analysis cannot detect the population it appears to exclude, and the reported null result would not constrain small populations.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"reports the 3.5-sigma archival 2014-15 neutrino flare from TXS 0506+056, the signal the same algorithm previously found and the source excluded in one version of the final p-value."},{"cited_title":"Williams, PoS(ICRC2019)16 (these proceedings)","cited_arxiv_id":null,"evidence_quote":"supplies the Fermi 3LAC catalog of blazar directions used as the 1023 search positions."},{"cited_title":"Ackermann et al., Astrophys","cited_arxiv_id":null,"evidence_quote":"supplies the five-year muon-neutrino track sample analyzed here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces the unbinned maximum-likelihood time-dependent flare search whose likelihood ratio forms the test statistic."},{"cited_title":"Braun , J","cited_arxiv_id":null,"evidence_quote":"extends that method with energy information and the trial-factor-corrected test statistic used to obtain per-direction p-values."}],"review_version":1}