REVIEW 3 major objections 3 minor 35 references
SPORE: An Event-Level Sampling Pipeline for Multi-Telescope Neutrino Astronomy
T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read spore reads tabulated telescope responses and outputs simulated neutrino events, reproducing a public ten-year track sample's declination distribution to 10-15%.
desk verdict A useful and honest infrastructure paper; the low-energy validation gap is an unambiguous ambiguity in the control, not a fatal flaw, and it deserves refereeing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the standardized HDF5 instrument-response file: for each event morphology it stores the effective area $A_{\mathrm{eff}}(\zeta,E)$, the PSF inverse CDF $\psi(E,u)$ mapping a uniform quantile to an angular deflection, and the energy-resolution inverse CDF $\Delta(u)$ for the log-ratio $\ln(E_{\mathrm{reco}}/E_{\mathrm{true}})$, or optionally a joint five-dimensional smearing histogram $P(E_{\mathrm{reco}}, \psi, \sigma_{\psi} | E_{\mathrm{true}}, \delta)$. The sampling engine is a hierarchical inverse-CDF algorithm that precomputes a weight grid $w(i,j,k) = A_{\mathrm{eff}}(\zeta_{ij}, E_k) \times \Phi(\delta_i, \alpha_j, E_k)$, factors it as $p(\sin\delta)\, p(\ln E|\sin\delta)\, p(\alpha|\sin\delta,\ln E)$, draws three uniform variates per event, and jitters each coordinate within its cell. This replaces rejection or MCMC sampling with direct inverse-CDF draws, gives errors linear in the cell size, and lets each detector independently Poisson-sample its count for joint analyses.
What would settle it
Take the same detector response but replace the smearing matrix with a fine-grained version, e.g. true-energy bins of width 0.1 in $\log_{10}(E/\mathrm{GeV})$ in ten declination bands, and re-run the energy round-trip: if the sampled energies still over-predict the observed rate below 600 GeV by about a factor of three, then the paper's attribution of the discrepancy to the released binning would be wrong and the sampler or the preprocessing would be implicated.
Extended reading notes
Core claim
The paper's central claim is that the full event-level response of a neutrino telescope—direction smearing, energy resolution, and event morphology—can be captured by three tabulated functions, stored in a standardized HDF5 format, and used to Monte Carlo sample individual reconstructed events without any proprietary simulation chain. The sampler draws true directions and energies from the source flux weighted by the effective area, then applies the point-spread-function and energy-resolution inverse CDFs to produce reconstructed directions and energies, including per-event angular uncertainties when the response supplies them. A hierarchical inverse-CDF factorization over $\sin\delta$, right ascension, and $\ln E$ handles extended sources with arbitrary RA- and declination-dependent flux maps, and multiple detectors can be passed to the same sampler to produce a joint, detector-tagged event list. The validation demonstrates that the sampled declination distribution of northern-sky tracks matches the observed public data to 10-15%, that the high-energy starting-event sample's deposited-energy spectrum tracks the published best-fit expectation, and that the point-source $\psi^2$ background shape is recovered. The reported energy-proxy discrepancy below 600 GeV is presented as an input-resolution artifact, not a sampler failure, because a direct forward fold of the same IRFs reproduces it.
Load-bearing premise
The energy round-trip conclusion assumes that the factor-of-three over-prediction below 600 GeV comes from the coarse true-energy binning of the public smearing matrix (0.5 in $\log_{10}(E/\mathrm{GeV})$, three declination bands, kernels averaged over the release simulation spectrum) rather than from the sampler itself or from the default effective-area smoothing and trimming.
Editorial extensions
If this is right
- Any neutrino telescope that publishes effective area, point spread function, and energy resolution in the standardized HDF5 format can be simulated at event level with no code changes, so future detectors can enter joint studies as soon as their response tables exist.
- Joint multi-detector analyses reduce to passing a list of detectors to one sampler: each detector Poisson-samples its own count, and the combined event list is tagged by detector, sharing a common source flux normalization.
- Extended sources with full RA- and declination-dependent flux maps are sampled natively by the same inverse-CDF engine, avoiding MCMC burn-in and multimodal convergence problems.
- The validation sets expectations for public-response-based simulations: declination distributions can be trusted at the 10-15% level, and reconstructed-energy predictions are reliable to about a third over the bulk of the sample, with a known low-energy bias traceable to coarsely binned public smearing matrices.
- Both Poisson and fixed-N sampling modes, plus transient and steady-state effective-area averaging, cover the standard pseudo-experiment workflows used in point-source searches.
Reading between the lines
- A natural stress test would be to apply the package to a second telescope with published response tables and public event data; if its sky and energy distributions are reproduced at similar accuracy, the selection-agnostic claim would be considerably stronger.
- The identified low-energy bias is an actionable message for data releases: publishing smearing matrices with finer true-energy bins, or with kernels unweighted by the release simulation spectrum, would remove the dominant systematic this paper had to work around.
- The same inverse-CDF factorisation could be extended to time-dependent effective areas or to detectors with non-separable responses (such as radio arrays) by generalising the HDF5 schema, opening event-level joint fitting across different technologies.
- If adopted widely, the package would let sensitivity projections for proposed detectors be produced from response tables alone, making detector-design comparisons and multi-telescope forecasts reproducible by people outside the collaborations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents SPORE, an open-source Python package that samples individual neutrino events from tabulated instrument response functions (IRFs) in a standardized HDF5 format. The package supports point and extended sources, multi-detector joint simulations, and provides reconstructed directions, energies, times, and morphologies. The authors validate SPORE with three tests: a comparison to the IceCube HESE 7.5-year public data release, a round-trip consistency test against the IceCube 10-year point-source data release, and an angular-distribution test for a point-source analysis near NGC-1068. The central claim is that any neutrino telescope with published response tables can be plugged into SPORE without code modification, and that the sampler reproduces observed data to the stated accuracy. The paper is accompanied by public code, IRF files, and scripts that reproduce all figures.
Significance. If the central claim holds, SPORE fills a genuine gap in the neutrino simulation ecosystem: it produces final-level reconstructed events directly from public IRFs, complementing Prometheus's detector-level simulation and toise's aggregate sensitivity calculations. The inverse-CDF sampling approach is fast and avoids MCMC burn-in, and the joint smearing-matrix support is a useful feature for realistic event generation. The open-source release with IRF files and figure scripts is a notable strength. However, the validation is largely round-trip in nature, and the energy-proxy comparison shows a factor-of-three excess at low energies whose attribution to the public IRFs rather than to SPORE's own preprocessing is not fully demonstrated. The quantitative validation claims therefore need strengthening before the plug-and-play assertion is fully supported.
major comments (3)
- [§6.2, App. B] The claim that the sub-600 GeV overprediction is 'traced to the coarse true-energy binning of the public smearing matrix rather than to the sampling' (Sec. 6.2) is not established by the forward-fold control as described, because the forward fold uses the same default effective-area preprocessing (trim_isolated and Gaussian smoothing with smoothing_sigma=1.5 energy bins, App. B) that is applied before any sampling. If that preprocessing broadens the low-energy acceptance, the forward fold would overpredict regardless of the smearing-matrix binning, and the discrepancy would be a property of SPORE's defaults rather than of the public IRFs. Please re-run the forward fold with smoothing_sigma=0 and trim_isolated disabled, or otherwise demonstrate that the excess persists when the effective-area table is unmodified. Without this, the abstract's statement that the discrepancy is 'traced to the coarse true-energy binning' is not supported.
- [§6.2, Fig. 5] The ratio panels in Fig. 5 show no uncertainty bands, so the quantitative agreement claims ('10–15% level' in declination, 'within about a third' in energy proxy) cannot be assessed against statistical or systematic scatter. Given that the atmospheric normalization is fit to the same data and only 40 pseudo-experiments are averaged, adding Poisson/ensemble bands or per-bin error bars to the Obs./Pred. panels is necessary to support these claims.
- [§6.1, Fig. 4] The HESE goodness-of-fit test is circular in the sense stated in the text: the same best-fit Monte Carlo defines both the reference expectation and the test-statistic distribution, so the agreement primarily demonstrates internal consistency of the sampler. The paper acknowledges this, but the sentence in Sec. 6.1 that the test 'indicates that spore reproduces both the mean expectation and the size of the Poisson scatter' overstates what can be concluded. I recommend either rephrasing to emphasize the consistency-check nature, or adding an independent validation against an analytic expectation for a simple case.
minor comments (3)
- [§3.2, Eq. (3.3)] The definition PSF(ψ|E) = d/dψ P(ψ'≤ψ|E) is a probability density, not a cumulative distribution; the text immediately says the PSF is stored as an inverse CDF. Please clarify the notation to avoid ambiguity between the density and the stored quantile function.
- [Fig. 2] The declination labels in Fig. 2 appear as boxes (δ = □45◦, δ = □90◦), likely a rendering or encoding issue. Please ensure the minus signs are displayed correctly.
- [App. B] The sentence about the spurious loader warning when the smearing subgroup is present is confusing; consider suppressing the warning automatically when the joint smearing subgroup is detected, or rewording the sentence to state clearly that the warning is expected and harmless in that case.
Circularity Check
No significant circularity: the validations are explicitly labeled round-trip/self-consistency tests, the one fitted normalization is disclosed before the energy-shape prediction, and no central result reduces to its own inputs.
full rationale
The paper's validation strategy is transparent about what is and is not independent. The 10-year tracks comparison is explicitly called a 'round-trip consistency test' (Sec. 6.2): public IRFs are ingested, events are sampled, and the resulting distributions are compared to observed IceCube data, not to a prediction derived from the same sampled events. The atmospheric normalization is fit to the sin-delta distribution and this fit is disclosed before the energy-proxy comparison, so the energy-proxy shape is a genuine conditional prediction rather than a fitted input renamed as a prediction. The HESE test is likewise framed honestly: the paper states that comparing to the best-fit Monte Carlo 'is not an independent test of the underlying flux model' and that the observed data sitting on the well-fitting side is 'expected rather than anomalous' because the expectation was itself fit to those data (Sec. 6.1). The low-energy discrepancy attribution to the public smearing-matrix binning, supported by a no-sampling forward fold, is a methodological argument about the public IRFs rather than a circular derivation; the paper also discloses in Sec. 7 and App. B that default effective-area smoothing and trimming modify the tabulated response before sampling, which is a possible confound but not a circularity. Citations to prior IceCube releases and to Prometheus provide context and comparison, not load-bearing justification for the sampler's correctness. No equation or fitted parameter is reused as its own prediction, and no self-citation chain forces the central claim. The paper is best read as presenting an internal-consistency-validated software tool, with the external-data comparisons serving as sanity checks whose limitations are acknowledged.
Assumptions & free parameters
free parameters (3)
- atmospheric_normalization =
1.30 relative to MCEq nominal
- smoothing_sigma =
1.5 energy bins (default)
- sampling_grid_resolution =
n_delta = n_alpha = n_E = 40 (default)
assumptions (5)
- domain assumption The tabulated instrument response functions provided in public IceCube releases are faithful enough representations of detector response and event selection for validation purposes.
- domain assumption The low-energy over-prediction below 600 GeV arises from the coarse true-energy binning and simulation-spectrum averaging of the public smearing matrix, not from the sampling algorithm or the default preprocessing.
- ad hoc to paper Default effective-area preprocessing (trimming isolated bins and Gaussian smoothing with width 1.5 energy bins) does not materially bias the validation comparisons.
- domain assumption For each detector, event counts are independent Poisson draws given the source flux, so no inter-detector correlation beyond the shared source normalization needs to be modeled.
- standard math Standard probability theory, inverse-CDF sampling, and PCHIP or spline interpolation are assumed correct.
Cite this review
Pith. "Pith review of SPORE: An Event-Level Sampling Pipeline for Multi-Telescope Neutrino Astronomy." pith.science (2026). https://pith.science/paper/RK6RZKP4
@misc{pith2026260811862,
author = {Pith},
title = {Pith review of: SPORE: An Event-Level Sampling Pipeline for Multi-Telescope Neutrino Astronomy},
year = {2026},
howpublished = {\url{https://pith.science/paper/RK6RZKP4}},
note = {Machine review of arXiv:2608.11862}
}
read the original abstract
We present an open-source Python package for simulating neutrino events from astrophysical point and extended sources using tabulated instrument response functions (IRFs). The package encodes three detector response components - effective area, point spread function, and energy resolution - in a selection-agnostic HDF5 format, which can represent a neutrino telescope whose response is supplied in that form, whether from a public release or a private study. Sampling algorithms cover point sources (inverse-CDF with Poisson or fixed-count modes), extended sky distributions (hierarchical inverse-CDF sampling, including full RA- and declination-dependent flux maps), and multi-detector joint analyses. We validate the framework via a round-trip consistency test using the publicly available IceCube 10-year tracks data release: the released IRFs are ingested into the package and used to generate a synthetic event set, whose declination distribution reproduces the observed one to 10-15% across the northern sky. The reconstructed-energy distribution agrees to within about a third over the bulk of the sample but exceeds the data by up to a factor of three below 600 GeV, a discrepancy we trace to the coarse true-energy binning of the public smearing matrix rather than to the sampling: an independent forward fold of the same IRFs reproduces it. We further compare against the IceCube HESE 7.5-year public data release: the sampled deposited-energy spectrum tracks the published best-fit expectation, and the observed data fall within the goodness-of-fit distribution built from 1,000 sampled pseudo-experiments, though on its well-fitting side, as expected for an expectation that was itself fit to those data.
Reference graph
Works this paper leans on
-
[1]
P. Mészáros, D. B. Fox, C. Hanna, K. Murase, Multi-Messenger Astrophysics, Nature Rev. Phys. 1 (2019) 585–599.arXiv:1906.10212,doi:10.1038/s42254-019-0101-z
arXiv 2019
-
[2]
M. G. Aartsen, et al., Evidence for High-Energy Extraterrestrial Neutrinos at the IceCube Detector, Science 342 (2013) 1242856.arXiv:1311.5238,doi:10.1126/science.1242856
arXiv 2013
-
[3]
R. Abbasi, et al., The IceCube high-energy starting event sample: Description and flux characterization with 7.5 years of data, Phys. Rev. D 104 (2021) 022002.arXiv:2011.03545, doi:10.1103/PhysRevD.104.022002. – 14 –
arXiv 2021
-
[4]
R. Abbasi, et al., Evidence for a Spectral Break or Curvature in the Spectrum of Astrophysical Neutrinos from 5 TeV–10 PeV, Phys. Rev. Lett. 136 (2026) 121002.arXiv:2507.22233, doi:10.1103/2gh9-d4q7
arXiv 2026
-
[5]
M. G. Aartsen, et al., Neutrino emission from the direction of the blazar TXS 0506+056 prior to the IceCube-170922A alert, Science 361 (6398) (2018) 147–151.arXiv:1807.08794, doi:10.1126/science.aat2890
arXiv 2018
-
[6]
R. Abbasi, et al., Evidence for neutrino emission from the nearby active galaxy NGC 1068, Science 378 (2022) 538–543.arXiv:2211.09972,doi:10.1126/science.abg3395
arXiv 2022
-
[7]
Adrian-Martinez, et al., Letter of intent for KM3NeT 2.0, J
S. Adrian-Martinez, et al., Letter of intent for KM3NeT 2.0, J. Phys. G 43 (8) (2016) 084001. arXiv:1601.07459,doi:10.1088/0954-3899/43/8/084001
arXiv 2016
-
[8]
Agostini, et al., The Pacific Ocean Neutrino Experiment, Nature Astron
M. Agostini, et al., The Pacific Ocean Neutrino Experiment, Nature Astron. 4 (2020) 913–915. arXiv:2005.09493,doi:10.1038/s41550-020-1182-4
arXiv 2020
Show all 35 references
-
[9]
Z. P. Ye, et al., A multi-cubic-kilometre neutrino telescope in the western Pacific Ocean, Nature Astron. 7 (2023) 1497–1505.arXiv:2207.04519,doi:10.1038/s41550-023-02087-6
2023
-
[10]
M. G. Aartsen, et al., IceCube-Gen2: the window to the extreme Universe, J. Phys. G 48 (2021) 060501.arXiv:2008.04323,doi:10.1088/1361-6471/abbd48
2021
-
[11]
W. G. Thompson, TAMBO: Searching for Tau Neutrinos in the Peruvian Andes, in: 38th International Cosmic Ray Conference, 2023.arXiv:2308.09753
2023 arXiv
-
[12]
C. A. Argüelles, et al., Measuring the high-energy neutrino sky using the deep-valley neutrino observatory TAMBO, Nature Astron. 10 (7) (2026) 947–951.arXiv:2507.08070, doi:10.1038/s41550-026-02916-4
2026 arXiv
-
[13]
A. N. Otte, et al., Trinity: An Air-Shower Imaging System for the Detection of Ultrahigh Energy Neutrinos, PoS ICRC2019 (2020) 976.arXiv:1907.08732,doi:10.22323/1.358.0976
2020 arXiv
-
[14]
Álvarez-Muñiz, et al., The Giant Radio Array for Neutrino Detection (GRAND): Science and Design, Sci
J. Álvarez-Muñiz, et al., The Giant Radio Array for Neutrino Detection (GRAND): Science and Design, Sci. China Phys. Mech. Astron. 63 (2020) 219501.arXiv:1810.09994, doi:10.1007/s11433-018-9385-7
2020 arXiv
-
[15]
Kotera, et al., The Hybrid Elevated Radio Observatory for Neutrinos (HERON) Project, PoS ICRC2025 (2025) 1078.arXiv:2507.04382,doi:10.22323/1.501.1078
K. Kotera, et al., The Hybrid Elevated Radio Observatory for Neutrinos (HERON) Project, PoS ICRC2025 (2025) 1078.arXiv:2507.04382,doi:10.22323/1.501.1078
2025 arXiv
-
[16]
Lazar, et al., Prometheus: An Open-Source Neutrino Telescope Simulation, Comput
J. Lazar, et al., Prometheus: An Open-Source Neutrino Telescope Simulation, Comput. Phys. Commun. 302 (2024) 109247.arXiv:2304.14526,doi:10.1016/j.cpc.2024.109247
2024
-
[17]
van Santen, B
J. van Santen, B. A. Clark, R. Halliday, S. Hallmann, A. Nelles, toise: a framework to describe the performance of high-energy neutrino detectors, JINST 17 (08) (2022) T08009. arXiv:2202.11120,doi:10.1088/1748-0221/17/08/T08009
2022 arXiv
-
[18]
Abbasi, et al., IceCube Data for Neutrino Point-Source Searches: Years 2008–2018, Astrophys
R. Abbasi, et al., IceCube Data for Neutrino Point-Source Searches: Years 2008–2018, Astrophys. J. Lett. 923 (2021) L3.arXiv:2101.09836,doi:10.3847/2041-8213/ac2c7b
2021
-
[19]
J. H. Koehne, K. Frantzen, M. Schmitz, T. Fuchs, W. Rhode, D. Chirkin, J. Becker Tjus, PROPOSAL: A tool for propagation of charged leptons, Comput. Phys. Commun. 184 (2013) 2070–2090.doi:10.1016/j.cpc.2013.04.001
2013 doi
-
[20]
Dunsch, J
M. Dunsch, J. Soedingrekso, A. Sandrock, M. Meier, T. Menne, W. Rhode, Recent Improvements for the Lepton Propagator PROPOSAL, Comput. Phys. Commun. 242 (2019) 132–144.arXiv:1809.07740,doi:10.1016/j.cpc.2019.03.021
2019 arXiv
-
[21]
D. E. Groom, N. V. Mokhov, S. I. Striganov, Muon stopping power and range tables 10-MeV to 100-TeV, Atom. Data Nucl. Data Tabl. 78 (2001) 183–356.doi:10.1006/adnd.2001.0861
2001
-
[22]
URLhttps://github.com/icecube/nuflux – 15 –
IceCube Collaboration, nuflux: Atmospheric neutrino flux library (2022). URLhttps://github.com/icecube/nuflux – 15 –
2022
-
[23]
Fedynitch, R
A. Fedynitch, R. Engel, T. K. Gaisser, F. Riehn, T. Stanev, Calculation of conventional and prompt lepton fluxes at very high energy, EPJ Web Conf. 99 (2015) 08001. arXiv:1503.00544,doi:10.1051/epjconf/20159908001
2015 arXiv
-
[24]
Fedynitch, F
A. Fedynitch, F. Riehn, R. Engel, T. K. Gaisser, T. Stanev, Hadronic interaction model sibyll 2.3c and inclusive lepton fluxes, Phys. Rev. D 100 (10) (2019) 103018.arXiv:1806.04140, doi:10.1103/PhysRevD.100.103018
2019 arXiv
-
[25]
H. P. Dembinski, R. Engel, A. Fedynitch, T. Gaisser, F. Riehn, T. Stanev, Data-driven model of the cosmic-ray flux and mass composition from 10 GeV to1011 GeV, PoS ICRC2017 (2018) 533.arXiv:1711.11432,doi:10.22323/1.301.0533
2018 arXiv
-
[26]
Riehn, R
F. Riehn, R. Engel, A. Fedynitch, T. K. Gaisser, T. Stanev, Hadronic interaction model Sibyll 2.3d and extensive air showers, Phys. Rev. D 102 (6) (2020) 063002.arXiv:1912.03300, doi:10.1103/PhysRevD.102.063002
2020 arXiv
-
[27]
J. P. Yañez, A. Fedynitch, Data-driven muon-calibrated neutrino flux, Phys. Rev. D 107 (12) (2023) 123037.arXiv:2303.00022,doi:10.1103/PhysRevD.107.123037
2023 arXiv
-
[28]
Braun, J
J. Braun, J. Dumm, F. De Palma, C. Finley, A. Karle, T. Montaruli, Methods for point source analysis in high energy neutrino telescopes, Astropart. Phys. 29 (2008) 299–305. arXiv:0801.1604,doi:10.1016/j.astropartphys.2008.02.007
2008 arXiv
-
[29]
M. G. Aartsen, et al., Time-Integrated Neutrino Source Searches with 10 Years of IceCube Data, Phys. Rev. Lett. 124 (5) (2020) 051103.arXiv:1910.08488, doi:10.1103/PhysRevLett.124.051103
2020
-
[30]
T. P. Robitaille, et al., Astropy: A community Python package for astronomy, Astron. Astrophys. 558 (2013) A33.arXiv:1307.6212,doi:10.1051/0004-6361/201322068
2013 arXiv
-
[31]
A. M. Price-Whelan, et al., The Astropy Project: Building an Open-science Project and Status of the v2.0 Core Package, Astron. J. 156 (2018) 123.arXiv:1801.02634, doi:10.3847/1538-3881/aabc4f
2018 arXiv
-
[32]
H. E. Grecco, et al., pint: Physical quantities in Python (2023). URLhttps://pint.readthedocs.io
2023
-
[33]
Collette, et al., h5py: A Pythonic interface to the HDF5 binary data format (2023)
A. Collette, et al., h5py: A Pythonic interface to the HDF5 binary data format (2023). URLhttps://www.h5py.org
2023
-
[34]
Virtanen, et al., SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nature Meth
P. Virtanen, et al., SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nature Meth. 17 (2020) 261–272.arXiv:1907.10121,doi:10.1038/s41592-019-0686-2
2020 arXiv
-
[35]
C. R. Harris, et al., Array programming with NumPy, Nature 585 (2020) 357–362. arXiv:2006.10256,doi:10.1038/s41586-020-2649-2. – 16 – A Software Architecture sporeis organized into five top-level sub-packages. spore.conventionsDefines the coordinate and unit primitives used th...
2020 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.