Pith. sign in

REVIEW 4 major objections 6 minor 23 references

Enhancements to the IceCube Extremely High Energy Neutrino Selection using Graph & Transformer Based Neural Networks

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that combining a graph neural network for muon-bundle rejection with a multitask transformer for direction reconstruction can roughly quintuple the number of IceCube EHE neutrino candidates reconstructed within 5 degrees…

desk verdict A useful methods paper whose performance numbers are not yet trustworthy: the evaluation appears to be on the training sample, so the claimed gains could be largely overfitting. read the letter →

arxiv 2507.11774 v1 pith:NC6PVSK4 submitted 2025-07-15 astro-ph.HE

Maxwell Nakos , Aske Rosted , Lu Lu (for the IceCube Collaboration) This is my paper · ORCID
classification astro-ph.HE
keywords IceCubeextremelyhighenergyneutrinosgraphneuralnetworktransformeratmosphericmuonbackgrounddirectionalreconstructionlateralspreadmultitasklearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that two neural networks can substantially improve IceCube's search for neutrinos above 10 PeV (the extremely-high-energy range), where a recent detection by another telescope has renewed interest. A graph neural network separates neutrino-induced tracks from cosmic-ray muon bundles by using the lateral spread of light deposits, without any prior on the event direction. A multitask transformer then reconstructs each event's direction without assuming it is a single straight track, which is the assumption baked into the current SplineMPE likelihood method. On a simulated 15-year exposure, the combination yields about five times more neutrino candidates reconstructed within 5 degrees of the true direction at the same atmospheric-muon background rate. If the performance carries over to real data, the selection would improve diffuse flux measurements above 10 PeV and increase the chance of catching an IceCube event comparable to the recent very-high-energy detection.

What carries the argument

The load-bearing machinery has two parts. First, a graph neural network (GNN) with one node per DOM, using position, first-hit time, and total charge as features, trained with a focal cross-entropy loss to separate neutrino events from cosmic-ray muon bundles; its discriminative power tracks the charge-weighted RMS of energy-deposit distances from the leading trajectory, i.e., the lateral spread. Second, a multitask transformer-based neural network (MTNN) that summarizes pulses at the DOM level (total charge, charge-weighted mean time, timing and charge percentiles, and closest times at fixed charge fractions), then uses a classifier token and supporting tasks (visible length, trackness, position, calorimetric energy) before final tasks (direction and energy) to achieve inductive transfer. Both networks process events above a 27,500-photoelectron charge threshold; the comparison baseline is SplineMPE, a likelihood reconstruction that assumes an infinite single muon track.

What would settle it

Run the trained GNN and MTNN on real IceCube events with independently known directions (for example, muons whose direction is fixed by IceTop or the Moon shadow) and compare the classifier score distribution and the opening-angle residuals to the simulation-based predictions; a significant mismatch in either would directly invalidate the claimed background rejection and reconstruction gains.

Watch

Extended reading notes

Core claim

The central claim is that the dominant background in IceCube's extremely-high-energy neutrino search—dense muon bundles from cosmic-ray showers—can be rejected using the lateral spread of their light deposits, and that a topology-agnostic neural network can reconstruct neutrino directions better than the standard likelihood method for most events. The graph network classifies each event using each DOM's position, first-hit time, and total charge; its score correlates strongly with the charge-weighted RMS of deposit distances from the event's leading trajectory, so thin single-muon-like events score as neutrino-like and laterally spread bundles are rejected. The multitask transformer reconstructs direction through supporting tasks (trackness, interaction position, visible length, calorimetric energy) that feed final tasks (direction and energy) and make no prior assumption about event topology, unlike SplineMPE's single-infinite-track model. On a simulated 15-year exposure with a bundle-rejection cut of 0.9, the transformer gives 20.8 neutrino events within 1 degree and 88.6 within 5 degrees of the true direction, compared with 12.5 and 15.7 for SplineMPE, at the same atmospheric-muon rate of roughly 85 events; for neutrinos above 10 PeV the 5-degree yield rises from 2.62 to 5.39.

Load-bearing premise

The networks are trained and tested only on simulated events, so the claimed gains hinge on the simulation matching how real cosmic-ray muon bundles deposit and spread light in the detector.

Editorial extensions

If this is right

  • A future EHE selection built on these networks would recover roughly 5–6 times more neutrino candidates within 5 degrees of the true direction than the SplineMPE-based selection at the same muon-bundle background rate.
  • The 15-year sample of about 89 well-reconstructed candidates and about 85 atmospheric muons at a score cut of 0.9 is well matched to a diffuse flux measurement above 10 PeV.
  • Because the transformer is trained on all flavors and topologies, cascade and starting-track events—where SplineMPE's single-track prior fails—gain the most reconstruction accuracy.
  • The lateral-spread classifier can be combined with existing overburden and stochastic energy-loss cuts to suppress downgoing-region background, where atmospheric muons are hardest to remove.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the GNN score correlates with lateral RMS, a simple analytic lateral-spread cut could capture much of the classifier's power and would be far easier to validate on real data; the network likely learns this physical observable rather than an uninterpretable pattern.
  • The same topology-agnostic multitask transformer design could transfer to other neutrino telescopes or to IceCube-Gen2, where higher pulse densities make DOM-level summarization and prior-free reconstruction more valuable.
  • The reported yields rest entirely on simulation; a calibration with real through-going muons of known direction would both validate the claimed gains and, if discrepancies appear, provide the data-driven corrections needed to retain them.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This proceedings paper describes two machine-learning tools for IceCube extremely-high-energy (EHE) neutrino searches. A graph neural network (GNN), built with the DYNEDGE architecture in GraphNeT, classifies neutrino-induced events against atmospheric muon bundles by exploiting the lateral spread of energy deposits. A multitask transformer (MTNN) performs directional reconstruction without assuming a single-track topology. The authors report improved muon background rejection in ROC curves, better angular resolution than the SplineMPE algorithm for most events, and expected 15-year neutrino event rates below various opening-angle thresholds (Table 1), weighted to astrophysical and atmospheric flux models. The work is framed as a proof-of-concept for future EHE selections.

Significance. If the reported gains survive a proper held-out evaluation, this is a useful methodological contribution for the next generation of IceCube EHE analyses. The paper has concrete strengths: it relies on publicly available software (GraphNeT/DYNEDGE), gives detailed descriptions of training weights and loss choices, compares against an existing IceCube algorithm, and provides falsifiable 15-year rate predictions with bootstrap uncertainties. However, the central claims currently rest on two load-bearing points that need work: (i) no held-out test set is described, so the reported improvements may be in-sample artifacts; and (ii) the SplineMPE baseline is used across all event topologies even though it is designed for single muon tracks, so part of the reported improvement may be an artifact of an unsuitable baseline. These issues are fixable and do not by themselves invalidate the approach, but they are essential for the paper's conclusions.

major comments (4)
  1. [Sections 2.2–2.3 and 3.1–3.2] No held-out test set is described anywhere in the paper. The GNN is trained on events passing a minimum charge cut of 27,500 PE (§2.2) and then evaluated on 'the analysis sample' with the same cut (§2.3); the MTNN is trained on events above 1,000 PE homogenized charge (§3.1) and evaluated on a subset with ≥27,500 PE (§3.2). The ROC curves (Fig. 3a), angular resolution distributions (Fig. 7), and Table 1 rates therefore appear to be in-sample. Hyperparameter choices such as the focal-loss γ=1.5 are also made from the same events. This can inflate all reported gains through memorization rather than generalization. Please state the exact train/validation/test split used for each network and recompute all headline numbers on a held-out sample; if a split was used but omitted, it must be documented.
  2. [Section 3.2, Figure 8, Table 1] The comparison with SplineMPE is not apples-to-apples. SplineMPE is a single-track likelihood reconstruction, while the MTNN is explicitly designed to handle cascades and other topologies. The paper states that 'a large part of the difference is driven by cascade-like events,' so the factor-of-several gains in Table 1 and the per-flavor medians in Fig. 8 may largely reflect SplineMPE's known poor performance on non-track topologies rather than genuinely better direction reconstruction. Please report the comparison separately for track-like and cascade-like events, and/or include a non-track-capable reference algorithm (e.g., a cascade reconstruction or a second ML baseline) so the claimed improvement is demonstrated on comparable footing.
  3. [Figure 3a and Figure 7] The central performance curves are shown without uncertainty bands. Figure 3a presents ROC curves for different focal-loss parameters with no confidence intervals, and Figure 7 presents opening-angle histograms without statistical errors. Since Table 1 does provide bootstrap uncertainties, the omission is not a question of means; the reader cannot tell whether the differences between γ values or between MTNN and SplineMPE are statistically significant. Please add bootstrap or equivalent uncertainties to these figures, or provide the relevant intervals in the text.
  4. [Sections 2.3 and 3.2 (data/MC validation)] All results are derived exclusively from CORSIKA/SIBYLL/H4a Monte Carlo. For a selection that is intended to be used on real data, the absence of any data/MC closure check is a concern. At minimum, the paper should state explicitly that the results are simulation-only, and ideally show a comparison of the classifier score distribution or the MTNN-vs-SplineMPE agreement on a high-charge atmospheric-muon data sample (e.g., through-going muons) where the simulation can be validated. This would make the central 'improved background rejection' claim robust to the assumption that the simulated lateral light distribution is accurate.
minor comments (6)
  1. [Abstract and throughout] The text is full of missing spaces (e.g., 'KM3NeThasrecentlyreported' in the abstract). This makes the paper hard to read and should be fixed in the published version.
  2. [Section 2.2 / Figure 1b] The term 'homogenized charge' is used in the body (§2.2, §3.2) but only defined in the caption to Fig. 1b; please define it at first use in the main text.
  3. [Table 1] The column headers use 'NN' without specifying that this denotes the MTNN. Since the paper presents both a GNN and an MTNN, clarify the notation.
  4. [Section 2.3] The model notation is inconsistent: 'GaisserH4a' appears alongside 'H4a' and 'GaisserH4a model'. Please unify the naming of the hadronic interaction model.
  5. [Figure 3a] The figure includes the focal-loss formula in the inset, but the caption does not identify the meaning of the inset or define the variables; please annotate the equation clearly.
  6. [Section 4.1] The paper describes applying the GNN and MTNN 'in conjunction' with existing selection methods but does not specify how the classifier score and the MTNN direction are combined into a single event selection; a flow diagram or a concise algorithm description would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are empirical supervised-learning results evaluated on Monte Carlo simulation, not quantities forced by construction.

full rationale

The paper's central claims are empirical evaluations of two neural networks trained on Monte Carlo simulation. The GNN classifier is trained with event labels (neutrino vs. atmospheric muon bundle) using DOM-level input features (position, first hit time, total charge) and is then evaluated via ROC curves and score distributions; the reported background rejection is a measured classifier performance, not a quantity defined by the training inputs. The MTNN is trained to regress MC-truth HEP direction as a final task and is evaluated by the opening angle between the reconstructed and true directions; the target quantity (direction) is not used as an input feature or as a selection weight, so the angular-resolution comparison to SplineMPE does not reduce to the training setup by construction. Choices such as the 27,500 PE charge threshold, the focal-loss gamma, and the 0.9 score cut are selection or tuning decisions, not definitions of the reported outcomes. Self-citations to prior IceCube work (DYNEDGE architecture, pulse summarization, Kaggle-inspired transformer blocks) supply methodological components but are not load-bearing justifications of the performance claims. The absence of a described held-out test set is a legitimate statistical concern about possible overfitting and optimistic bias, but it is an evaluation-validity issue, not circularity: an in-sample evaluation can be biased without making the result definitionally equal to its inputs. No equation, fitted parameter, or citation chain in the paper forces the claimed improvements by construction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claims rest on choices of analysis thresholds, training compositions, and simulation models. The 27,500 PE threshold and focal loss gamma are hand-picked from simulated data, and the physical separation relies on the assumption that simulated lateral spreads match reality. No new physical entities are introduced.

free parameters (4)
  • Charge threshold = 27,500 PE
    Chosen based on Fig. 1b to filter out single muon showers; affects both training and analysis samples.
  • Focal loss gamma = 1.5
    Selected as best performing on ROC curve, affecting classifier behavior.
  • Class composition = 90% nu_mu, 10% nu_tau
    Training mix chosen to represent EHE neutrino flavors; may bias classification.
  • Muon bundle rejection cut = 0.9
    Chosen to balance astrophysical neutrino and atmospheric muon rates in Table 1.
assumptions (3)
  • domain assumption CORSIKA with SIBYLL 2.3d accurately models cosmic ray air showers and the IceCube detector simulation is accurate.
    All training and evaluation relies on simulated events; if simulation is wrong, the reported performance may not hold in real data.
  • domain assumption Muon bundles from cosmic rays have greater lateral spread than neutrino-induced single muons.
    This physical premise underpins the GNN's discriminative power; it is demonstrated in simulation but not yet confirmed on data.
  • domain assumption The H4a cosmic ray model and the best-fit starting track neutrino flux are appropriate weighting models.
    These models determine the relative weighting of signal and background in ROC curves and rate estimates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancements to the IceCube Extremely High Energy Neutrino Selection using Graph & Transformer Based Neural Networks." pith.science (2026). https://pith.science/paper/NC6PVSK4

@misc{pith2026250711774,
  author       = {Pith},
  title        = {Pith review of: Enhancements to the IceCube Extremely High Energy Neutrino Selection using Graph & Transformer Based Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NC6PVSK4}},
  note         = {Machine review of arXiv:2507.11774}
}
read the original abstract

KM3NeT has recently reported the detection of a very high-energy neutrino event, while IceCube has previously set upper limits on the differential neutrino flux above 100 PeV but has yet to observe a neutrino event with an energy comparable to that of the KM3NeT detection. To improve diffuse measurements above 10 PeV, we apply machine learning techniques to enhance atmospheric muon background rejection and directional reconstruction. We utilize a Graph Neural Network (GNN) to perform a classification task that distinguishes neutrinos from high-energy atmospheric muons. The method allows for the rejection of early hits from laterally spread, lower-energy muons in cosmic ray showers without relying on directional reconstruction as a prior. Additionally, a Transformer-based Neural Network is implemented for directional reconstruction. Unlike previous likelihood-based rapid reconstruction algorithms that assume a single muon track, this method makes no prior assumptions about event topology of the particle inside the detector. We demonstrate improved background rejection and reconstruction performance using machine learning techniques. Applications to the development of future Extremely High Energy (EHE) selections are also discussed.

Figures

Figures reproduced from arXiv: 2507.11774 by the authors.

Figure 1
Figure 1. (a) Energy deposit locations of events in IceCube, shown in cylindrical detector coordinates and weighted by the number of photoelectrons detected. Left: a cosmic ray shower with a bundle of muons exhibiting lateral spread from the trajectory of the initial cosmic ray. Right: a muon produced from a 𝜈 𝐶𝐶 𝜇 interaction is shown traversing through the detector. (b) Number of muons in cosmic ray showers with a primary e… view at source ↗
Figure 2
Figure 2. (a) Lateral spread distribution function of a high-energy muon bundle. The RMS (root mean square) is computed by the equation shown in the plot, where 𝑤𝑖 is the number of photo electrons detected from an energy deposit and 𝑟𝑖 is the radial distance of the energy deposit from the highest energy muon in the event. (b) (left) RMS over an ensemble of cosmic ray showers and 𝜈𝜇 events. The astrophysical spectrum is weight… view at source ↗
Figure 3
Figure 3. (a) ROC curve for classifiers trained with different focal loss 𝛾 parameters described by [9]. The focus is on the neutrino signal dominated region of the curve. Astrophysical neutrinos are weighted with the best fit starting track single power law fit and the cosmic rays are weighted with H4a. (b) The score distribution for cosmic rays and neutrinos. Neutrino events have a target of 1, and cosmic ray events have a … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Energy deposit locations of cosmic ray showers in IceCube, shown in cylindrical detector coordinates and weighted by the number of photoelectrons detected. The top row shows relatively horizontal cosmic ray showers, while the bottom rows shows more downgoing cosmic ray…
Figure 5
Figure 5. Figure 5: Classifier score distribution as a function of the RMS of cosmic ray showers. Percentiles are derived from unweighted Monte Carlo statistics. This analysis includes a wide range of event topologies such as through-going tracks, starting tracks, cascades. Therefore, we …
Figure 6
Figure 6. Figure 6: (a) Schematic illustrating how pulses are summarized at the DOM level. (b) Overview of the model architecture, the Transformer blocks as well as the bias and encoding are inspired by the second place solution of the IceCube Kaggle competition [18]. Another complication…
Figure 7
Figure 7. Figure 7: Weighted histogram distribution of the opening angle between reconstructed and true directions. (a) Neutrino events (all flavor) (b) Atmospheric muon background events of events, these events are likely to be the ones fitting the prior used to generate the PDF splines.…
Figure 8
Figure 8. Figure 8: (a) MTNN medians and 25/75th percentiles for different neutrino flavors. (b) SplineMPE medians and 25/75th percentiles for different neutrino flavors. 4. Discussion 4.1 Application to future Extremely high energy neutrino selections [PITH_FULL_IMAGE:figures/full_fig_p…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 21 canonical work pages

  1. [1]

    Greisen,Phys

    K. Greisen,Phys. Rev. Lett.16(Apr, 1966) 748–750

    work page 1966

  2. [2]

    Aartsenet al.,Journal of Instrumentation12 no

    IceCubeCollaboration, M. Aartsenet al.,Journal of Instrumentation12 no. 03, (Mar, 2017) P03012

    work page 2017

  3. [3]

    Aartsenet al.,Nature551no

    IceCubeCollaboration, M. Aartsenet al.,Nature551no. 7682, (Nov., 2017) 596–600

    work page 2017

  4. [4]

    IceCubeCollaboration, M. G. Aartsenet al.,Phys. Rev. Lett.111 (Jul, 2013) 021103

    work page 2013

  5. [5]

    IceCubeCollaboration, M. G. Aartsenet al.,Phys. Rev. D88(Dec, 2013) 112008

    work page 2013

  6. [6]

    IceCube Collaboration, M. G. Aartsenet al.,Phys. Rev. Lett.117 (Dec, 2016) 241101

    work page 2016

  7. [7]

    IceCube Collaboration, M. G. Aartsenet al.,Phys. Rev. D98(Sep, 2018) 062003

    work page 2018

  8. [8]

    Abbasiet al., 2025

    IceCube Collaboration, R. Abbasiet al., 2025. https://arxiv.org/abs/2502.01963

    arXiv 2025

Show all 23 references
  1. [9]

    Focal loss for dense object detection,

    T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in2017 IEEE International Conference on Computer Vision (ICCV), pp. 2999–3007. 2017

    2017

  2. [10]

    Abbasiet al.,Phys

    IceCube Collaboration, R. Abbasiet al.,Phys. Rev. D110 (Jul, 2024) 022001

    2024

  3. [11]

    T. K. Gaisser, T. Stanev, and S. Tilav,Frontiers of Physics8 no. 6, (Dec, 2013) 748–758

    2013

  4. [12]

    Abbasiet al.,Journal of Instrumentation17no

    IceCube Collaboration, R. Abbasiet al.,Journal of Instrumentation17no. 11, (Nov, 2022) P11003

    2022

  5. [13]

    Søgaardet al.,Journal of Open Source Software8 no

    A. Søgaardet al.,Journal of Open Source Software8 no. 85, (2023) 4971

    2023

  6. [14]

    CORSIKA: A Monte Carlo code to simulate extensive air showers,

    D. Hecket al., “CORSIKA: A Monte Carlo code to simulate extensive air showers,” 1998

    1998

  7. [15]

    Riehn, R

    F. Riehn, R. Engel, A. Fedynitch, T. K. Gaisser, and T. Stanev,Phys. Rev. D102(Sep, 2020) 063002

    2020

  8. [16]

    Bradascio and T

    F. Bradascio and T. Glüsenkamp,EPJ Web of Conferences207 (2019) 05002

    2019

  9. [17]

    Caruana,Machine Learning28(07, 1997)

    R. Caruana,Machine Learning28(07, 1997)

    1997

  10. [18]

    Bukhariet al.,Eur

    H. Bukhariet al.,Eur. Phys. J. C84no. 6, (2024) 646

    2024

  11. [19]

    Abbasiet al.,Journal of Instrumentation16no

    IceCube Collaboration, R. Abbasiet al.,Journal of Instrumentation16no. 07, (July, 2021) P07041

    2021

  12. [20]

    Cauchy loss function: Robustness under gaussian and cauchy noise,

    T. Mlotshwa, H. van Deventer, and A. S. Bosman, “Cauchy loss function: Robustness under gaussian and cauchy noise,” 2023.https://arxiv.org/abs/2302.07238

    2023 arXiv

  13. [21]

    Aartsenet al.,Physical Review Letters125no

    M. Aartsenet al.,Physical Review Letters125no. 12, (Sept., 2020)

    2020

  14. [22]

    R. Naab, E. Ganster, Z. Zhang,et al.,PoSICRC2023(2023) 1064

    2023

  15. [23]

    H. P. Dembinskiet al., PoSICRC2017(2018) 533. 8 Enhancing the IceCube EHE Neutrino Selection Maxwell Nakos, Aske Rosted, and Lu Lu Full Author List: IceCube Collaboration R. Abbasi16, M. Ackermann63, J. Adams17, S. K. Agarwalla39, a, J. A. Aguilar10, M. Ahlers21, J.M. Alameddi...

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.