Advancements in Computing and Simulation Techniques for the HIBEAM-NNBAR Experiment

Adam Kozela; Alexander Burgman; Anders Oskarsson; Andr\'e Nepomuceno; Bernhard Meirose; Blahoslav Rataj; David Milstead; Ernesto Kemp; Gabriel Silva; Jorge Amaral

REVIEW 3 major objections 5 minor 18 references

Advancements in Computing and Simulation Techniques for the HIBEAM-NNBAR Experiment

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Machine-learning models trained on simulation can reject 100% of cosmic-ray background while keeping up to 98.71% of neutron-annihilation signal in the HIBEAM-NNBAR test set, and fast parametric simulations of the calorimeter and TPC are…

desk verdict Useful status report with an overclaimed ML headline; the detector studies are fine, but the 100% rejection claim needs proper counting and a metric that does what it says. read the letter →

arxiv 2507.02810 v1 pith:JQHGGIOO submitted 2025-07-03 physics.ins-det

Bernhard Meirose , Jorge Amaral , Alexander Burgman , Matthias Holl , Ernesto Kemp , Adam Kozela , David Milstead , André Nepomuceno

show 9 more authors

Anders Oskarsson Krzysztof Pysz Valentina Santoro Tiago Quirino Blahoslav Rataj Gabriel Silva Samuel Silverstein Magnus Wolke Lucas {AA}strand

This is my paper

classification physics.ins-det

keywords neutron-antineutronoscillationbaryonnumberviolationmachinelearningeventselectionbackgroundrejectionfastdetectorsimulationtimeprojectionchamberreadoutelectronicscosmic-ray

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports on practical computing tools for the proposed HIBEAM-NNBAR neutron-antineutron search: machine learning for event selection, fast parametrized detector simulation for design studies, and a readout-electronics simulator. Its central quantitative claim is that, on a simulated sample of 369,569 events, gradient-boosted tree models and a random forest trained with a false-positive-penalizing metric achieve 100% background rejection while keeping up to 98.71% of signal. The same approach reduces the feature set from 49 high-level variables to 26 with essentially no loss in efficiency. If these results transfer to real data, the experiment could rely on essentially background-free event selection with most of the signal preserved, which matters because the expected annihilation rate is extremely small. The remaining sections give faster approximations of calorimeter response and a detailed simulation chain for the time projection chamber and readout electronics to support detector development.

What carries the argument

The central object is the custom asymmetric metric $C_{\mathrm{metric}} = \mathrm{FN} + (\beta-1)\mathrm{FP}/\beta$ with $\beta=1000$, which makes a single false positive 999 times more costly than a single false negative; models are tuned and thresholded on this metric, and that is why they reach 100% background rejection. For the simulation sections, the carrying objects are parametric resolution functions such as $\sigma_E/E = 0.05/\sqrt{E[\mathrm{GeV}]}$ for photon energy deposits, a Gaussian-smearing wrapper that turns particle kinetic energy into calorimeter signals, and a modular readout chain combining Poisson event timing, energy deposition, convolution with an electronic impulse response, and Gaussian noise.

What would settle it

Take the trained classifiers and expose them to background events generated by a second, independent simulation or by real cosmic-ray calibration data; any event that passes the chosen threshold as a false positive would show that the 100% rejection is tied to the specific training simulation rather than to the background itself.

Watch

Extended reading notes

Core claim

The authors claim that a custom cost metric of the form $C_{\mathrm{metric}} = \mathrm{FN} + (\beta-1)\mathrm{FP}/\beta$ with $\beta=1000$, applied to tree-based classifiers, produces a decision threshold with zero false positives on the test set for several model families, with signal efficiencies of 95.81%, 98.71%, and 98.69%. Correlation-based feature reduction to 26 variables keeps 100% rejection at 98% efficiency, while PCA reduction to 14 variables drops efficiency to 74%. The paper further claims that a fast simulation that smears energy deposits with the measured calorimeter resolution reproduces real data from a lead-glass calorimeter, and that a detailed gas-detector simulation reconstructs simulated muon tracks with residuals at the level of a few tenths of a millimeter or better.

Load-bearing premise

The 100% background rejection is a statement about the simulated test set: if the simulated cosmic-ray events do not match the real background the detector will see, the rejection rate will be lower on actual data.

Editorial extensions

If this is right

Event selection for HIBEAM-NNBAR could use machine-learned scores instead of hand-crafted cuts, with zero simulated background accepted at thresholds that keep roughly 98% of annihilation signal.
The 26-variable subset performs nearly as well as the full 49-variable set, suggesting the analysis can be simplified or made more interpretable without much loss.
The fast calorimeter simulation is accurate enough for design studies, predicting about 73% acceptance for neutral pions from annihilation at around 150 MeV, which shortens detector-optimization cycles.
TPC pad geometry can be tuned in simulation: more pads and steeper zigzag angles improve z-resolution, while y-resolution is set by the number of readout rows.
The readout simulation framework can show how electronic noise and pulse shaping affect physics variables, allowing electronics specifications to be evaluated before construction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: because the background model is a single cosmic-ray generator, the claimed 100% rejection is a proof-of-principle for the custom metric rather than a guarantee about real running conditions; a different background source, such as beam-related neutrons, could break it.
Editorial inference: the custom metric with $\beta=1000$ encodes the physics priority of never letting background through, so the same training recipe should transfer to other experiments with extreme background-suppression requirements.
Editorial inference: the fast simulation ignores position smearing and shower overlap, so its reconstructed pion-mass peak is likely narrower than what a full simulation or real data would show; testing overlap effects is a natural next step.
Editorial inference: if the readout simulation is faithful, it could be used to train classifiers on digitized waveforms rather than high-level variables, potentially improving rejection and making selection robust to electronics variations.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a collaboration status report, and its most striking claim—100% background rejection with up to 98.7% efficiency—does not hold up as written. The rest of the paper is a reasonable, if preliminary, set of detector studies.

What is genuinely useful: the TPC simulation work is concrete, with three pad geometries compared and residuals reported; the framework combining Matlab, COMSOL, and Garfield++ is sensible, though 21 muons is a very small sample. The fast calorimeter simulation uses published resolution parametrizations from WASA, KEK, and proton data, and the check against WASA pi0 data is a real cross-check, not a circular fit to the same dataset. The readout simulation framework is straightforward and clearly described. These parts are solid enough for a proceedings paper.

The soft spot is the ML section. Table 1 reports 'Rejection 100' for every model, but the paper never states how many background events are in the 50% test split or gives any confidence interval. With a handful of background events, 0/N rejection is compatible with a large true false-positive rate. The custom metric in Eq. 1 is also mis-specified: with beta = 1000, (beta-1)/beta is 0.999, so false positives and false negatives have nearly equal weight. The text says the metric penalizes false positives, but numerically it barely does. On top of that, thresholds are tuned on the validation set, so the test numbers are a fit, not an independent prediction. The ML claim needs test-set counts, error bars, and a metric that actually implements the stated penalty.

There are also smaller issues: the fast simulation treats photons as well-separated by construction, so shower overlaps are ignored; the TPC sample of 21 muons is too small for the residual comparisons to mean much; and no code or data are provided, which limits reproducibility. None of these are fatal for a status report, but they reinforce that the paper is a progress update, not a definitive result.

I would send this to peer review, but with a clear expectation of heavy revision. The collaboration is doing serious work for a proposed experiment, and a referee can push them to quantify the ML claim properly. As is, the headline result is unsupported, but the underlying detector studies are worth engaging with.

Referee Report

3 major / 5 minor

Summary. The paper describes recent progress in computing and simulation techniques for the HIBEAM-NNBAR experiment at the European Spallation Source. Section 2 presents machine-learning models (Random Forest, XGBoost, LightGBM) trained on a simulated dataset of neutron-antineutron annihilation signals and CRY-generated cosmic-ray backgrounds, with a custom metric (Eq. 1) and threshold optimization on a validation set; Table 1 reports 100% background rejection with efficiencies up to 98.71% on a 50% test split. Section 3 introduces fast parametric simulations for the electromagnetic calorimeter, with resolution parametrizations for photons, charged pions, and protons, validated against WASA SEC data. Section 4 describes a TPC simulation framework using Garfield++, COMSOL, and Matlab, with a preliminary study of three zigzag pad geometries using 21 simulated muons. Section 5 outlines a readout simulation framework that models Poisson-distributed event timing, Geant4 energy deposition, electronics shaping, and Gaussian noise.

Significance. The paper reports a set of computational tools under development for the HIBEAM-NNBAR experiment. The strongest contribution is the ML event-selection study, which, if the perfect-separation claim could be substantiated with proper statistics, would be valuable for the experiment's background rejection strategy. The fast calorimeter simulation is a practical speed-up for detector design, but its predictive value depends on independent validation. The TPC and readout simulations are early-stage frameworks that will likely evolve. The paper is honest about several limitations (e.g., no shower development in fast simulation), which is a credit, but the current reporting does not yet support the headline claims.

major comments (3)

[§2, Table 1 and Eq. (1)] The headline claim that XGBoost, LGBM, and Random Forest models achieve 100% background rejection on the test set is not statistically supported as reported. Table 1 gives no class counts for the 50% test split, so a perfect rejection may be vacuous if the background test sample is small; the authors should report the number of signal and background events in the test set and provide a confidence interval (e.g., Wilson) for the zero false-positive count. In addition, the custom metric in Eq. (1), with β=1000, assigns a false-positive weight of (β−1)/β = 0.999, essentially identical to the false-negative weight of 1, so the stated objective of 'penalizing false positives' is not realized by the equation as written; either the equation or the text needs correction.
[§3] The validation of the fast simulation is circular for the photon parametrization: Eq. (2) is taken from reference [10], the WASA SEC detector paper, and the validation compares the fast simulation with real data from that same detector, so agreement is largely by construction. The charged-pion and proton parametrizations, Eqs. (3) and (4), are fitted to data from other detectors ([12] and [13]) and are not validated at all against the WASA SEC. The authors should either validate the fast simulation against independent data (e.g., a different calorimeter) or explicitly frame the model as an unvalidated interpolation tool rather than claiming 'good agreement' as evidence of fidelity.
[§4, Table 2] The TPC geometry study is based on only 21 simulated muons, with no statement about how these are distributed across the three pad designs. The residuals in Table 2 (e.g., real-path z residuals of 0.002±0.027, 0.003±0.014, and 0.005±0.017 cm for the prototype, double-pad, and double-angle designs) are all mutually consistent within their quoted uncertainties, so the conclusion that 'increasing pad count improves z-resolution' is not justified by the data. The authors should increase the sample size, report the per-configuration track counts, and apply a statistical test before drawing conclusions about pad geometry.

minor comments (5)

[§2, Table 1] The text states that PCA reduced the input variables to 14, but Table 1 lists 17 variables for the LGBM (Optuna) + PCA model; this inconsistency should be corrected.
[§2] The phrase 'to find he best hyperparameters' contains a typo and should read 'to find the best hyperparameters'.
[§3] The fast simulation does not simulate shower development or position smearing; this limitation is stated in the text but should be more prominent in the abstract or conclusion so that the model's scope is not misrepresented.
[§4, Figures 3 and 5] The captions of Figures 3 and 5 do not specify the number of simulated muons or the pad configuration; adding these details would aid reproducibility.
[§5] The readout simulation framework is described without any comparison to measured electronics signals; the authors should state the current validation status of this framework.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: the central ML claim is evaluated on a held-out test set, and the only mild validation loop is the WASA fast-simulation check, where the input energy resolution comes from the same detector used for comparison.

fitted input called prediction [Section 3, 'Fast Simulation of the HIBEAM Calorimeter', Eq. (2) and the validation paragraph]
"For photons, the parametrization of the signal of interest—energy deposits—was constructed based on the detector’s energy resolution for a two-photon signal, as given in reference [10]: σE/E = 0.05/sqrt(E[GeV]). ... The code was validated by simulating the decay of η mesons into photons and neutral pions (which subsequently decay into photons) and comparing the fast detector simulation results with real data recorded by the W ASA calorimeter. We found good agreement between the fast simulation and real data."

The Gaussian energy smearing in the fast simulation is set by the WASA SEC's measured energy resolution (Eq. 2, from reference [10]). The 'validation' then compares simulated events with real data from the same WASA calorimeter. Because the smearing width is taken from those same-detector data, the reconstructed mass resolution in the simulation is expected to match by construction; the comparison is a self-consistency check rather than an independent test of the simulation. The agreement does not provide independent evidence for the parametrization beyond the resolution value that was input. This loop affects only the WASA calorimeter fast-simulation validation, not the ML or TPC/readout results.

full rationale

The strongest claim, 100% background rejection in Table 1, is not circular: the thresholds are tuned on the validation set, but the reported metrics are for a held-out 50% test split, so the performance is an independent evaluation rather than a fit. The absence of test-set background counts and confidence intervals is a statistical reporting gap, not a circularity. The fast-simulation validation for the WASA calorimeter is mildly circular because the energy-resolution input (Eq. 2) comes from the same detector whose real data are used as the validation reference; the 'good agreement' is therefore a self-consistency check. However, this is a supporting validation, not a central derivation, and the TPC, readout, and ML sections stand on their own. Citations to previous HIBEAM-NNBAR papers are program references and are not load-bearing for the simulation or ML results. Overall, the paper is largely self-contained against external benchmarks, with one minor validation loop, so the circularity score is low.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claims rest on several fitted parameters and on the assumption that simulations accurately model the real detector. No new physical entities are introduced.

free parameters (6)

beta in Cmetric = 1000
Chosen by hand in Eq. (1) to define the event selection metric; the equation does not actually heavily penalize false positives.
ML decision thresholds = 0.933, 0.999, 0.999, 0.997, 0.999 (Table 1)
Optimized on the validation set for each model.
Proton resolution parameters a and b = a=0.00068, b=0.0036
Fitted to data from reference [13] and used in Eq. (4).
Charged pion resolution coefficient = 9.3 (and 0.16 special case)
Fitted to data from reference [12] and used in Eq. (3).
Photon resolution coefficient = 0.05
Taken from reference [10] in Eq. (2); still a parameter of the simulation.
TPC drift field and gas mixture = 287.3 V/cm, 80% Ar/20% CO2
Chosen for the study in Section 4.

assumptions (3)

domain assumption The simulated signal (neutron annihilation in carbon producing multi-pion events) and background (CRY cosmic-ray shower generator) are accurate representations of the experimental conditions.
Used to generate the ML training data in Section 2; if unrealistic, the 100% rejection claim does not transfer.
domain assumption Resolution parametrizations from historical detectors (WASA SEC [10], KEK CsI [12], and reference [13]) remain valid for the HIBEAM calorimeter configuration.
Used in the fast simulation in Section 3 without re-validation for the new geometry.
domain assumption Poisson event timing and Gaussian electronic noise are adequate models for the readout system.
Assumed in Section 5 to generate signals; no comparison to real readout data is shown.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Advancements in Computing and Simulation Techniques for the HIBEAM-NNBAR Experiment." pith.science (2026). https://pith.science/paper/JQHGGIOO

@misc{pith2026250702810,
  author       = {Pith},
  title        = {Pith review of: Advancements in Computing and Simulation Techniques for the HIBEAM-NNBAR Experiment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JQHGGIOO}},
  note         = {Machine review of arXiv:2507.02810}
}

read the original abstract

The HIBEAM-NNBAR program is a proposed two-stage experiment at the European Spallation Source focusing on searches for baryon number violation processes as well as ultralight dark matter. This paper presents recent advancements in computing and simulation, including machine learning for event selection, fast parametric simulations for detector studies, and detailed modeling of the time projection chamber and readout electronics.

Figures

Figures reproduced from arXiv: 2507.02810 by the authors.

**Figure 1.** (a) Reconstructed π 0 invariant mass and (b) SEC geometrical acceptance. ∼200 MeV, the pions punch through the detector unless nuclear reactions occur, resulting in an energy loss peak at around 140 MeV, which is nearly independent of the incident kinetic energy. The spectrum for ∼190 MeV pions includes contributions from both stopped and punched-through pions and is treated as a special case in the parametrization.… view at source ↗

**Figure 2.** (a) Energy loss resolution for charged pions in a CsI crystal, with fitted curve, and [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Simulated muon trajectories in the TPC. Track reconstruction depends on the zigzag pad geometry [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Zigzag pad designs: Double pads (left), Prototype (center), Double angle (right). [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Reconstructed tracks (dashed lines mark GEM positions). [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Flowchart of the simulation framework for TPC, HDR, or LEC readout channels. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Results of simulated signal operations in the framework. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 16 canonical work pages

[10]

Bargholtz et al., Nucl

C. Bargholtz et al., Nucl. Instrum. Methods A 594, 339–350 (2008)

work page 2008
[12]

Yamazaki et al., Nucl

H. Yamazaki et al., Nucl. Instrum. Meth. A 391, 427 (1997)

work page 1997
[13]

Merchez et al., Nucl

F. Merchez et al., Nucl. Instrum. Methods A 275, 133 (1989)

work page 1989
[1]

Addazi et al., J

A. Addazi et al., J. Phys. G 48, 070501 (2021), 2006.04907

arXiv 2021
[2]

Garoby et al., Physica Scripta 93, 014001 (2017)

R. Garoby et al., Physica Scripta 93, 014001 (2017)

work page 2017
[3]

The HIBEAM Instrument at the European Spallation Source

V . Santoro et al. (2023), 2311.08326

work page Pith review arXiv 2023
[4]

Santoro et al., J

V . Santoro et al., J. Neutron Res. 25, 315 (2024)

work page 2024
[5]

Baldo-Ceolin et al., Z

M. Baldo-Ceolin et al., Z. Phys. C 63, 409 (1994)

work page 1994

Show all 18 references

[6]

Barrow et al., EPJ Web Conf

J. Barrow et al., EPJ Web Conf. 251, 02062 (2021), 2106.15898

2021 arXiv
[7]

Hagmann et al., CRY: Cosmic-Ray Shower Generator , in IEEE Nucl

C. Hagmann et al., CRY: Cosmic-Ray Shower Generator , in IEEE Nucl. Sci. Symp. Conf. Rec. (2007), V ol. 2, pp. 1143–1146

2007
[8]

Akiba et al., Optuna: Hyperparameter Optimization Framework, in Proc

T. Akiba et al., Optuna: Hyperparameter Optimization Framework, in Proc. 25th ACM SIGKDD Conf. (2019), pp. 2623–2631

2019
[9]

Samek et al., Explainable AI: Interpreting and Visualizing Deep Learning , V ol

W. Samek et al., Explainable AI: Interpreting and Visualizing Deep Learning , V ol. 11700 (Springer, 2019)

2019
[11]

Agostinelli et al

S. Agostinelli et al. (GEANT4), Nucl. Instrum. Meth. A 506, 250 (2003)

2003
[14]

Schindler, Garfield++ (2010), https://garfieldpp.web.cern.ch/ garfieldpp/

H. Schindler, Garfield++ (2010), https://garfieldpp.web.cern.ch/ garfieldpp/

2010
[15]

mathworks.com

The MathWorks Inc., Matlab version: 9.13.0 (r2022b) (2022), https://www. mathworks.com

2022
[16]

COMSOL AB, COMSOL Multiphysics® v6.3, Stockholm, Sweden (2024), available at https://www.comsol.com

2024
[17]

Pfei ffer et al., Nucl

D. Pfei ffer et al., Nucl. Instrum. Meth. A 935, 121 (2019), 1806.05880

2019 arXiv
[18]

Luna et al., FPGA-Based Simulator for ATLAS TileCal Readout , in Proc

F.C. Luna et al., FPGA-Based Simulator for ATLAS TileCal Readout , in Proc. XXVII Encontro Nac. Modelagem Comput. (2024)

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [10]

Bargholtz et al., Nucl

C. Bargholtz et al., Nucl. Instrum. Methods A 594, 339–350 (2008)

work page 2008

[2] [12]

Yamazaki et al., Nucl

H. Yamazaki et al., Nucl. Instrum. Meth. A 391, 427 (1997)

work page 1997

[3] [13]

Merchez et al., Nucl

F. Merchez et al., Nucl. Instrum. Methods A 275, 133 (1989)

work page 1989

[4] [1]

Addazi et al., J

A. Addazi et al., J. Phys. G 48, 070501 (2021), 2006.04907

arXiv 2021

[5] [2]

Garoby et al., Physica Scripta 93, 014001 (2017)

R. Garoby et al., Physica Scripta 93, 014001 (2017)

work page 2017

[6] [3]

The HIBEAM Instrument at the European Spallation Source

V . Santoro et al. (2023), 2311.08326

work page Pith review arXiv 2023

[7] [4]

Santoro et al., J

V . Santoro et al., J. Neutron Res. 25, 315 (2024)

work page 2024

[8] [5]

Baldo-Ceolin et al., Z

M. Baldo-Ceolin et al., Z. Phys. C 63, 409 (1994)

work page 1994