REVIEW 2 major objections 4 minor 27 references
Machine Learning for the Cluster Reconstruction in the CALIFA Calorimeter at R3B
T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Neural edge detector lifts CALIFA cluster reconstruction 34 percent
desk verdict Solid engineering study with a promising ML-based cluster reconstruction method, but the missing validation split for threshold tuning makes the headline numbers somewhat optimistic. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is the hit-pair edge classifier: a fully connected network with one hidden layer of 1,000 nodes and two hidden layers of 100 nodes, taking a 12-dimensional input per hit pair—energies, polar angles, azimuthal angles, times, and their absolute differences—and returning a sigmoid score between 0 and 1 that a hit pair shares a true cluster, with the azimuthal difference computed to respect the angle's periodicity. Before the network runs, agglomerative hierarchical clustering groups hits in spherical coordinates whose radial dimension encodes shifted time, using a variance-minimizing linkage; this pre-clustering defines candidate cluster boundaries. The network then classifies the pairs, and connected classified pairs are merged into final clusters. The paper points out that this makes the edge classifier a special case of a graph neural network, with the agglomerative step acting as an inductive bias that improves on applying the network to raw hits alone.
What would settle it
Run the Agglo+Edge pipeline on the 7,000 simulated test events with both thresholds chosen exclusively on a held-out validation subset, then measure the event-level true-positive rate; if it does not reproduce roughly 81.3% and does not clearly beat the 60.6% geometric baseline, the central comparison collapses. An even stronger check is to repeat on real CALIFA events with known calibration and compare full-energy peak reconstruction efficiency.
Extended reading notes
Core claim
On simulated CALIFA events, the paper finds that a two-stage model called Agglo+Edge outperforms the standard geometrical R3B clustering on every reported metric. Event-level true positive rate rises from 60.6% to 81.3%; the false negative rate falls from 25.2% to 12.2%; false mixed events fall from 8.9% to 1.5%; false positives stay essentially flat at 5.1% versus 5.3%; and the fraction of well-reconstructed clusters rises from 80.4% to 91.0%. The decisive component is the edge-detection network used as a cleanup step after agglomerative pre-clustering: it reattaches hits that the geometric step leaves stranded, especially low-energy deposits around 0.5 MeV and 1.6 MeV produced by pair production and annihilation. Time-of-hit information is important: the time-aware combined model reaches 81.3%, while time-blind variants reach at most 74.7%, and even those outperform the geometric baseline. The paper presents the result as evidence that machine-learning clustering can improve reconstruction in highly granular calorimeters, while noting that the method cannot correct over-merging introduced by the pre-clustering stage.
Load-bearing premise
The 81.3% figure rests on the assumption that the thresholds used for the agglomerative and neural-network stages were fixed on a validation set separate from the 7,000 test events, which the paper does not explicitly document.
Editorial extensions
If this is right
- Time-of-hit information should be included in CALIFA cluster reconstruction whenever it is available, because every time-aware variant in the study outperforms its time-blind equivalent.
- The false-negative rate, the dominant failure of the geometric algorithm, is cut by more than half, so the method should make low-statistics and rare-reaction measurements more sensitive.
- The edge network can be appended to an existing geometric pipeline as a cleanup step, not only to the agglomerative pipeline, so adoption can be incremental.
- Because time-blind edge models also beat the geometric baseline, the neural pair-classification itself, not only the timing channel, is responsible for most of the improvement.
- The pipeline's known inability to split over-merged clusters points to cluster splitting as the next required stage; the paper identifies this as future work.
Reading between the lines
- The paper does not test transfer to the full detector: the simulation uses a partial 2024 geometry without the forward CEPA region, so the absolute percentages are specific to that configuration until full-geometry or real-data checks are done.
- The same recipe—unsupervised clustering to build a graph, then a small network to classify its edges—should transfer to other segmented calorimeters or tracking detectors whose showers produce sparse, split deposits; this is an extension the paper does not make.
- The 34% gain is a simulation-based estimate; a direct test would fix both the agglomerative threshold and the network threshold on a validation split, then report test-set numbers from those fixed thresholds, since the paper does not document that selection.
- A natural extension suggested by the failure mode is to weight false negatives and false positives asymmetrically in the loss, since the baseline failure pattern is dominated by under-merging rather than over-merging.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies cluster reconstruction for the CALIFA calorimeter at R3B using Geant4 simulations with three gamma rays per event. It compares the existing geometrical R3B clustering algorithm against agglomerative clustering, an edge-detection neural network, and hybrid combinations, with the central claim that the Agglo+Edge model improves the event-level true-positive rate from 60.6% to 81.3% while also improving the cluster-level well-reconstructed rate from 80.4% to 91.0% (Table 1). The paper also discusses the role of time-of-hit information, the limitations of the approach in correcting over-merging, and computational scaling.
Significance. If the reported improvement is unbiased, this is a useful practical contribution to high-granularity calorimeter reconstruction: it demonstrates that relatively simple machine-learning methods can substantially outperform a fixed-cone geometric baseline on simulated data, and it gives a concrete example where time information is important. The evaluation is based on external simulated ground truth with supervised labels, which avoids circularity, and the reported uncertainties from ten independent training runs for the NN variants are a strength. However, the evaluation protocol has a load-bearing gap: no validation split is documented for threshold selection, and the training procedure for the edge NN is not described at the level needed to exclude data leakage. The significance of the headline 81.3% figure therefore depends on whether these methodological issues are resolved.
major comments (2)
- [Sections 2.5, 2.6, and Table 1] The thresholds for agglomerative clustering and for the edge NN are described as optimized on the data, but no validation split is documented anywhere in the paper. Section 2.3 states only a single train/test split of 13,000/7,000 events, while Section 2.5 says the agglomerative threshold was optimized to yield the best TP and WR, and Section 2.6 says the NN threshold is tuned to optimize performance across all defined metrics. If these threshold scans used the 7,000 test events, then the Table 1 numbers, especially TP=81.3% for Agglo+Edge, are selected maxima rather than unbiased estimates, and the comparison against the fixed-cone baseline is inflated by an unknown amount. Please introduce a validation split for all threshold choices, report the thresholds selected on that validation set, and recompute the test-set metrics accordingly.
- [Section 2.6] The training procedure for the edge NN is described as training on false negative cases of the pre-clustering step, but the paper does not state how those cases are identified or whether they are drawn exclusively from the 13,000-event training subset. If false-negative events were selected using ground-truth labels from the combined dataset, information from test events could influence the training pairs and inflate the reported performance. Please clarify whether the false-negative cases used for training are derived only from training events, and confirm explicitly that no test-event information is used in any stage of model development, including feature normalization, early stopping, or threshold selection.
minor comments (4)
- [Table 1] The baseline and agglomerative rows are reported without uncertainties, while all NN variants include standard deviations from ten runs; adding uncertainties for the non-NN rows would help the reader judge whether the performance gaps are statistically significant.
- [Section 2.5] The radial coordinate r encodes time as r = t + 4.5 microseconds, but Ward linkage uses Euclidean distances in a space where angular coordinates are dimensionless while r has units of time; this scale mismatch can strongly affect the clustering and should be discussed or the features should be normalized.
- [Figure 2] The lower panel of Figure 2 shows a bin-by-bin count difference, but without statistical uncertainties or a pull distribution it is difficult to judge whether the excess near 6.3 MeV is significant; a small addition would make the comparison more quantitative.
- [References] Several references have incomplete or inconsistently formatted entries, for example reference [2] contains a stray typo and reference [5] lacks full bibliographic information; references [7] and [15] point to the same hierarchical clustering chapter and should be consolidated.
Circularity Check
No circularity found: the reported cluster-reconstruction improvements are empirical comparisons against external Geant4 ground truth.
full rationale
The paper's central claim is that the Agglo+Edge model reaches 81.3% event-level true positives versus 60.6% for the geometric R3B algorithm on simulated CALIFA data. The derivation chain is not circular: the Geant4 simulation provides ground-truth labels; the geometric, agglomerative, and neural-network clustering algorithms are applied to the simulated hits; and the custom metrics (TP, FP, FN, FM, WR) compare the resulting clusters to that external ground truth. The edge-detection neural network is trained with a supervised binary cross-entropy loss on the 13,000-event training split and evaluated on the separate 7,000-event test split described in Section 2.3. No equation in the paper defines an output in terms of an input, and no fitted parameter is renamed as a prediction. The agglomerative threshold and the neural-network decision threshold are said to be optimized to improve TP/WR and the full metric set, respectively, but the paper does not state that this optimization is performed on the test set; without such a statement one cannot exhibit a reduction of the reported 81.3% figure to a test-set fit. At most this is an evaluation-leakage risk, not a demonstrated circular step. The cited methods (SciPy agglomerative clustering, PyTorch, Geant4, standard references) are external and not load-bearing self-citations. The comparison is self-contained against simulated ground truth, so the score is 0.
Assumptions & free parameters
free parameters (4)
- Cone aperture for geometrical R3B clustering =
0.25 rad
- Agglomerative clustering distance threshold =
not reported
- Edge NN decision threshold =
not reported
- NN training hyperparameters =
learning rate 5e-3, 8e4 epochs, layers 10^3-10^2-10^2
assumptions (4)
- domain assumption Geant4 simulation accurately models photon interactions and detector response for CALIFA.
- domain assumption The simulated event topology (three gamma rays per event, uniform energies 0.3-10 MeV, all detected) is representative of R3B physics.
- ad hoc to paper The custom clustering metrics (TP, FP, FN, FM, WR) capture reconstruction quality.
- domain assumption The 4 microsecond event window and 200 ns Gaussian timing smear reflect real CALIFA timing.
Cite this review
Pith. "Pith review of Machine Learning for the Cluster Reconstruction in the CALIFA Calorimeter at R3B." pith.science (2026). https://pith.science/paper/QEYQT5AM
@misc{pith2026250609088,
author = {Pith},
title = {Pith review of: Machine Learning for the Cluster Reconstruction in the CALIFA Calorimeter at R3B},
year = {2026},
howpublished = {\url{https://pith.science/paper/QEYQT5AM}},
note = {Machine review of arXiv:2506.09088}
}
read the original abstract
The R3B experiment at FAIR studies nuclear reactions using high-energy radioactive beams. One key detector in R3B is the CALIFA calorimeter consisting of 2544 CsI(Tl) scintillator crystals designed to detect light charged particles and gamma rays with an energy resolution in the per cent range after Doppler correction. Precise cluster reconstruction from sparse hit patterns is a crucial requirement. Standard algorithms typically use fixed cluster sizes or geometric thresholds. To enhance performance, advanced machine learning techniques such as agglomerative clustering were implemented to use the full multi-dimensional parameter space including geometry, energy and time of individual interactions. An Edge Detection Neural Network exhibited significant differences. This study, based on Geant4 simulations, demonstrates improvements in cluster reconstruction efficiency of more than 30%, showcasing the potential of machine learning in nuclear physics experiments.
Figures
Reference graph
Works this paper leans on
-
[1]
Experiments at the Interface of Nuclear, Atomic, and Hadron Physics with FRS at GSI and Super-FRS at FAIR
-
[2]
Leifels, Status and physics perspectives of FAIR, IL NUOVO CIMENTO 100 (59) (2025) 48
Y . Leifels, Status and physics perspectives of FAIR, IL NUOVO CIMENTO 100 (59) (2025) 48
work page 2025
-
[3]
D. Cortina-Gil, H. Alvarez-Pol, T. Aumann, V . Avde- ichikov, M. Bendel, J. Benlliure, D. Bertini, A. Bezbakh, T. Bloch, M. Böhmer, et al., CALIFA, a Dedicated Calorimeter for the R3B/FAIR, Nuclear Data Sheets 120 (2014) 99–101. 5
work page 2014
-
[4]
H. Alvarez-Pol, N. Ashwood, T. Aumann, D. Bertini, P. Cabanelas, E. Casarejos, J. Cederkall, D. Cortina-Gil, P. D. Fernández, I. Durán, et al., Performance analysis for the CALIFA Barrel calorimeter of the R3B experi- ment, Nuclear Instruments and Methods in Physics Re- search Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 767 ...
work page 2014
-
[5]
T. Le Bleis, M. Bendel, R. Gernhäuser, W. Henning, P. Klenze, R. Krücken, M. Winkel, A Digital Readout for CALIFA
-
[6]
D. Bertini, R3BRoot, simulation and analysis framework for the R3B experiment at FAIR, in: Journal of Physics: Conference Series, V ol. 331, IOP Publishing, 2011, p. 032036
work page 2011
-
[7]
Nielsen, Hierarchical Clustering, Springer International Publishing, Cham, 2016, pp
F. Nielsen, Hierarchical Clustering, Springer International Publishing, Cham, 2016, pp. 195–211
work page 2016
-
[8]
M.-C. Popescu, V . E. Balas, L. Perescu-Popescu, N. Mas- torakis, Multilayer perceptron and neural networks, WSEAS Transactions on Circuits and Systems 8 (7) (2009) 579–588
work page 2009
Show all 27 references
-
[9]
Kolanoski, N
H. Kolanoski, N. Wermes, Teilchendetektoren, Springer, 2016, Ch. 13.5.2
2016
-
[10]
Seltzer, XCOM-Photon Cross Sections Database, NIST Standard Reference Database 8, http://www.nist.gov/pml/data/xcom/index.cfm, accessed: 2025-04-10 (2010)
S. Seltzer, XCOM-Photon Cross Sections Database, NIST Standard Reference Database 8, http://www.nist.gov/pml/data/xcom/index.cfm, accessed: 2025-04-10 (2010)
2010
-
[11]
Klein, Y
O. Klein, Y . Nishina, Über die Streuung von Strahlung durch freie Elektronen nach der neuen relativistischen Quantendynamik von Dirac, Zeitschrift für Physik 52 (11) (1929) 853–868
1929
-
[12]
Lipi ´nski, T
M. Lipi ´nski, T. Włostowski, J. Serrano, P. Alvarez, White rabbit: A PTP application for robust sub-nanosecond syn- chronization, in: 2011 IEEE International Symposium on Precision Clock Synchronization for Measurement, Con- trol and Communication, IEEE, 2011, pp. 25–30
2011
-
[13]
Agostinelli, J
S. Agostinelli, J. Allison, K. a. Amako, J. Apostolakis, H. Araujo, P. Arce, M. Asai, D. Axen, S. Banerjee, G. Barrand, et al., GEANT4—a simulation toolkit, Nu- clear instruments and methods in physics research section A: Accelerators, Spectrometers, Detectors and Associated E...
2003
-
[14]
Virtanen, R
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haber- land, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, et al., SciPy 1.0: fundamental al- gorithms for scientific computing in Python, Nature meth- ods 17 (3) (2020) 261–272
2020
-
[15]
Nielsen, F
F. Nielsen, F. Nielsen, Hierarchical clustering, Introduc- tion to HPC with MPI for Data Science (2016) 195–211
2016
-
[16]
Imambi, K
S. Imambi, K. B. Prakash, G. Kanagachidambaresan, Py- torch, Programming with TensorFlow: solution for edge computing applications (2021) 87–104
2021
-
[17]
A. F. Agarap, Deep learning using rectified linear units (relu), arXiv preprint arXiv:1803.08375 (2018)
2018 arXiv
-
[18]
Mannor, D
S. Mannor, D. Peleg, R. Rubinstein, The cross entropy method for classification, in: Proceedings of the 22nd international conference on Machine learning, 2005, pp. 561–568
2005
-
[19]
De Boer, D
P.-T. De Boer, D. P. Kroese, S. Mannor, R. Y . Rubinstein, A tutorial on the cross-entropy method, Annals of opera- tions research 134 (2005) 19–67
2005
-
[20]
Newton, R
D. Newton, R. Pasupathy, F. Yousefian, Recent trends in stochastic gradient descent for machine learning and Big Data, in: 2018 Winter Simulation Conference (WSC), IEEE, 2018, pp. 366–380
2018
-
[21]
P. W. Battaglia, J. B. Hamrick, V . Bapst, A. Sanchez- Gonzalez, V . Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, et al., Relational in- ductive biases, deep learning, and graph networks, arXiv preprint arXiv:1806.01261 (2018)
2018 arXiv
-
[22]
Vaswani, N
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30 (2017)
2017
-
[23]
Amatriain, A
X. Amatriain, A. Sankar, J. Bing, P. K. Bodigutla, T. J. Hazen, M. Kazi, Transformer models: an introduction and catalog, arXiv preprint arXiv:2302.07730 (2023)
2023 arXiv
-
[24]
DeZoort, S
G. DeZoort, S. Thais, J. Duarte, V . Razavimaleki, M. Atkinson, I. Ojalvo, M. Neubauer, P. Elmer, Charged particle tracking via edge-classifying interaction net- works, Computing and Software for Big Science 5 (2021) 1–13
2021
-
[25]
X. Ju, D. Murnane, P. Calafiura, N. Choma, S. Conlon, S. Farrell, Y . Xu, M. Spiropulu, J.-R. Vlimant, A. Au- risano, et al., Performance of a geometric deep learn- ing pipeline for HL-LHC particle tracking, The European Physical Journal C 81 (2021) 1–14
2021
-
[26]
Van Stroud, P
S. Van Stroud, P. Duckett, M. Hart, N. Pond, S. Rettie, G. Facini, T. Scanlon, Transformers for Charged Parti- cle Track Reconstruction in High Energy Physics, arXiv preprint arXiv:2411.07149 (2024)
2024
-
[27]
Cluster of Excellence ORIGINS, Cluster of excel- lence origins,https://www.origins-cluster.de, ac- cessed: 2025-05-03 (2025). 6
2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.