Pith. sign in

REVIEW 2 major objections 6 minor 19 references

Development and Explainability of Models for Machine-Learning-Based Reconstruction of Signals in Particle Detectors

T0 review · 2 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read A convolutional autoencoder with upsampled output reconstructs calorimeter pulse timing to 0.19 ns and amplitude to 23 mV on simulated scintillator signals.

desk verdict Occlusion sensitivity points to sub-bin timing information, but the UMAC-vs-MAC gain is overstated because the two models are compared under different post-processing. read the letter →

arxiv 2504.17272 v1 pith:5EBGA3GT submitted 2025-04-24 physics.ins-det hep-exphysics.comp-ph

classification physics.ins-dethep-exphysics.comp-ph
keywords machinelearningsignalreconstructionexplainableAIcalorimetersautoencoderupsamplingOcclusionSensitivitypulsearrivaltime
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a convolutional autoencoder trained on raw digitized waveforms can reconstruct the arrival time and amplitude of pulses in a scintillating-crystal calorimeter, and that an upsampled version of the model (UMAC) is substantially more accurate than the earlier modified autoencoder (MAC). On a simulated test set, UMAC lowers arrival-time mean absolute error from 0.77 ns to 0.19 ns and amplitude mean absolute error from 111.6 mV to 23.2 mV. The authors also use occlusion-sensitivity analysis to show that the signal front and maximum are the parts of the pulse the network relies on, and they use that observation to motivate finer arrival-time labels and a weighted arrival-time formula. If the comparison is correct, this is a practical route to better time and energy resolution in pile-up-prone calorimeters without changing the detector hardware.

What carries the argument

The load-bearing mechanism is the upsampled modified autoencoder (UMAC): a convolutional autoencoder whose decoder inserts two factor-2 upsampling layers after the first two transposed convolutions, expanding the 1024-sample input to a 4096-sample output so the pulse arrival-time label can be rounded at 0.25 ns instead of 1 ns. Around it sits a weighted arrival-time post-processing formula $t_{\mathrm{arrival}} = \frac{\sum_i A_i t_i}{\sum_i A_i}$, which uses the model's amplitude values on neighboring bins to estimate a sub-bin arrival time. The same architecture also serves as the inference target of the Occlusion Sensitivity analysis, which masks stretches of the waveform and watches the prediction loss to identify the signal rise and maximum as the features the network uses.

What would settle it

Re-run the MAC model on the same independent test set after applying the weighted arrival-time post-processing (Eq. 4) that UMAC uses, then recompute the arrival-time MAE and MSE in Table 1. If the MAC values move close to UMAC's $0.19$ ns / $0.16~\mathrm{ns}^2$, the fourfold timing improvement would be attributable to post-processing rather than to the upsampling architecture.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that finer time labels alone, realized by adding two factor-2 upsampling layers to the decoder, turn a moderate autoencoder reconstruction into one that reads pulse timing at a fraction of the digitizer's 1 ns bin. The reported numbers for the same independent test set are arrival-time MAE/MSE of $0.19$ ns / $0.16~\mathrm{ns}^2$ and amplitude MAE/MSE of $23.2$ mV / $988.6~\mathrm{mV}^2$ for UMAC, against $0.77$ ns / $0.86~\mathrm{ns}^2$ and $111.6$ mV / $17{,}045.8~\mathrm{mV}^2$ for MAC. The arrival-time error distribution for UMAC is centered at 0.02 ns with standard deviation 0.30 ns, while MAC sits at about 0.75 ns, and the amplitude error for UMAC is centered near $-2.7$ mV instead of $-111.8$ mV. From this the authors conclude that upsampling models reconstruct both arrival time and amplitude better, and that the amplitude result is centered enough to suggest less energy calibration will be needed when the model is applied to real detector data.

Load-bearing premise

The load-bearing premise is that MAC and UMAC were compared under identical post-processing; the paper never explicitly states that the weighted arrival-time formula (Eq. 4) was applied to MAC, and the 0.75 ns offset left in the MAC timing distribution suggests it may not have been.

Editorial extensions

If this is right

  • On the reported test set, UMAC improves arrival-time MAE by roughly a factor of 4 (0.77 to 0.19 ns) and amplitude MAE by roughly a factor of 5 (111.6 to 23.2 mV), so the reconstruction can be noticeably more accurate before any detector changes.
  • Because the arrival-time error for UMAC is centered near zero rather than offset by roughly 0.75 ns, the need for an empirical timing calibration of the model may be reduced compared with MAC.
  • The 0.25 ns label precision and the weighted post-processing give the model sub-bin timing information, which the authors identify as important for separating close-in-time pulses.
  • The xAI analysis suggests that the signal front and maximum carry most of the information used for recognition, which can guide where to spend model capacity or where to mask inputs in future architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A test the paper does not report: apply the weighted arrival-time formula of Eq. (4) to the MAC baseline on the same test set. If the MAC arrival-time MAE drops close to 0.19 ns, part of the claimed fourfold timing gain would be attributable to post-processing rather than to the upsampling architecture.
  • UMAC's added upsampling layers increase the parameter count and runtime, so a natural extension is to check whether the same sub-bin accuracy survives quantization or distillation for FPGA-based triggers.
  • The same upsampling-plus-weighted-arrival-time recipe could transfer to other binned waveform-extraction problems, such as drift-chamber timing or photodetector pulse-height analysis, wherever the label rounding is coarser than the signal's intrinsic arrival-time precision.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper presents two convolutional autoencoder models, MAC and UMAC, for reconstructing pulse arrival time and amplitude from simulated 1024-sample scintillator waveforms. The MAC is first analyzed with Occlusion Sensitivity; the xAI results are used to motivate a weighted arrival-time post-processing formula (Eq. 4). The UMAC modifies the decoder with two upsampling layers and uses 4096-element labels with 0.25 ns precision. On a common held-out simulated test set, the paper reports arrival-time MAE/MSE of 0.19 ns/0.16 ns^2 for UMAC versus 0.77 ns/0.86 ns^2 for MAC, and amplitude MAE/MSE of 23.2 mV/988.6 mV^2 versus 111.6 mV/17,045.8 mV^2, concluding that the upsampling model reconstructs both quantities more accurately.

Significance. If the reported improvement is real, the UMAC is a practically valuable step for calorimeter pulse reconstruction, and the use of xAI to motivate a concrete post-processing change is a nice illustration of explainability-driven development. The paper has real strengths: the comparison uses a large common held-out simulated dataset; the metrics and error distributions are presented transparently; and the central numbers are internally consistent on that dataset. Its main weakness is not the statistical basis but the lack of a controlled comparison: the UMAC and MAC differ in post-processing, label precision, and architecture, so the headline gain cannot yet be attributed to the upsampling design as claimed.

major comments (2)
  1. [Section 4, Eq. (4) and Table 1/Figure 8] The paper never states that the MAC baseline was re-processed with the weighted arrival-time formula, and the evidence suggests it was not. After introducing Eq. (4), the text says only that the UMAC results are 'post-processed using the weighted arrival time determination'; no analogous statement is made for MAC. Figure 8 shows the MAC arrival-time error distribution centered at 0.7521 ns, which the text attributes to the difference between the true arrival time and the rounded 1 ns label position—exactly the bias Eq. (4) is designed to remove—and Table 1 gives a MAC error-distribution mean of 0.75 ns. If MAC predictions were read out by unweighted argmax while UMAC used Eq. (4), the reported 4x timing improvement conflates the post-processing change with the model change. The authors should apply Eq. (4) to MAC outputs and report the resulting Table 1 and Figure 8, or explicitly document and justify any difference in post-processing.
  2. [Section 4, UMAC description and Section 5 conclusions] The UMAC design changes two factors simultaneously: the label precision is increased from 1 ns to 0.25 ns (4096 labels per event) and the decoder is augmented with two upsampling layers. The conclusion 'upsampling models do perform better for reconstructing both the arrival time and for the signal amplitude' is therefore ambiguous as a statement about architecture: the improvement could come entirely from the finer labels. A controlled comparison (for example, MAC trained on 0.25 ns labels, or UMAC with labels rounded to 1 ns) is needed to attribute the gain to the upsampling layers; otherwise the conclusions should be rephrased to refer to the combined UMAC design rather than to 'upsampling models.'
minor comments (6)
  1. [Section 2] The text first says the output layer has 'no activation function' and then says 'The activation of the output layer is also set to ReLU'; please reconcile these statements.
  2. [Section 4, Eq. (4)] Please specify the summation range in Eq. (4); as written it appears to sum over all positions, whereas the earlier merging-window description suggests only positions within the window T around the maximum should contribute.
  3. [Table 1 and Figures 8-9] Please state the number of test events used for Table 1 and report uncertainties on the MAE/MSE values; Figures 8 and 9 contain 97,493 entries.
  4. [Figure 8] The UMAC histogram mean (0.02121 ns) differs from the Gaussian fit mean (0.000608 ns); please clarify which quantity corresponds to the 'error distribution mean' in Table 1.
  5. [Section 2] Adding a short description of the simulation (pulse multiplicity distribution, amplitude distribution, noise level, and the generation of t0 with 10^-6 ns precision before rounding) would improve reproducibility.
  6. [Section 5] The statement that the model might remove the need for energy calibration is based only on the simulated test-set mean offset; please qualify it with the caveat that the real-data application described in Section 2 required additional calibration.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: UMAC claims rest on an independent held-out evaluation; the possible unequal application of Eq. (4) post-processing to UMAC only is a controlled-comparison concern, not a construction-level circularity.

full rationale

Walking the derivation chain: the MAC and UMAC architectures are defined by model structure and are trained on simulated waveforms with ground-truth labels, and the reported arrival-time and amplitude errors are computed on an independent test dataset (Section 4, Table 1). No parameter is fitted to the test set and then renamed as a prediction; the UMAC is a genuinely new architecture with 0.25 ns label granularity, and its performance advantage is an empirical result rather than a consequence of an equation identical to its input. The xAI-motivated weighted arrival-time formula (Eq. 4) is a post-processing rule, not a fitted parameter. The self-citations (Refs. [13,14,17,18]) refer to the authors' earlier models and applications; the baseline MAC is re-evaluated in this paper, and the central UMAC-vs-MAC comparison does not reduce to those citations. A separate methodological caveat is that the paper states 'The results are post-processed using the weighted arrival time determination' immediately after describing UMAC, while Figure 8's MAC Gaussian mean of 0.7521 ns corresponds to the unweighted rounding bias that Eq. (4) was introduced to remove; if Eq. (4) was not applied to MAC, the reported 4x timing advantage would be partly an artifact of unequal post-processing. That is a validity concern about controlled comparison, not circularity of the derivation, because UMAC's performance numbers are not mathematically forced by the inputs or by a fitted parameter. Overall, there is no significant circularity; score 2 reflects the reuse of the authors' own prior models as baseline with minor self-citation, but no load-bearing circular step.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The free parameters are design choices in the ML pipeline, and the axioms are domain-modeling assumptions about simulation fidelity and comparison fairness.

free parameters (4)
  • Upsampling factor (output label length multiplier) = 4 (4096 labels per 1024 ns)
    Chosen by hand as 4x finer time binning (0.25 ns). Only this value is tested, so the improvement attributable to the architecture is not separated from the label precision change.
  • Number of upsampling layers in UMAC decoder = 2
    Architecture choice; the decoder expands by 2x twice. The paper does not test alternative depths.
  • Merging window T = not stated
    The post-processing merging window T (Section 2) is defined but its value is never given, so arrival-time extraction is not fully reproducible.
  • Mask size for Occlusion Sensitivity = 18 (also 8 and 28 explored)
    The mask length is fixed to 18 to match the first convolutional kernel; central to the xAI analysis but not to the UMAC claim.
assumptions (5)
  • domain assumption Simulated waveforms follow the two-exponential pulse shape A(t)=A0(e^{-(t-t0)/τ1}-e^{-(t-t0)/τ2}) (Eq. 1) and faithfully represent real scintillator signals.
    All training and evaluation use this functional form; no real-data validation is shown for UMAC, so transfer to real detectors depends on this assumed fidelity.
  • domain assumption CAEN V17XX digitizer behavior (1024 samples at 1 ns, ~1 V amplitude range) is representative of the calorimeter readout systems targeted.
    The dataset is generated to match these boards, and the paper claims general applicability to electromagnetic calorimeters based on that match.
  • domain assumption The sparse label representation (zeros except amplitude at arrival position) is a valid training target for time and amplitude reconstruction.
    The model is optimized to reproduce this representation; the paper does not compare alternative label encodings.
  • domain assumption The single held-out simulated dataset with 97,493 events is statistically adequate and representative for comparing MAC and UMAC.
    No cross-validation, multiple seeds, or uncertainty on the metrics is reported.
  • domain assumption The ML model trained on simulated data generalizes to real detector data.
    For MAC this is asserted via ref. [17] but required calibration; for UMAC there is no real-data test, making the 'no calibration needed' speculation unsupported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Development and Explainability of Models for Machine-Learning-Based Reconstruction of Signals in Particle Detectors." pith.science (2026). https://pith.science/paper/5EBGA3GT

@misc{pith2026250417272,
  author       = {Pith},
  title        = {Pith review of: Development and Explainability of Models for Machine-Learning-Based Reconstruction of Signals in Particle Detectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5EBGA3GT}},
  note         = {Machine review of arXiv:2504.17272}
}
read the original abstract

Machine learning methods are being introduced at all stages of data reconstruction and analysis in various high-energy physics experiments. We present the development and application of convolutional neural networks with modified autoencoder architecture for the reconstruction of the pulse arrival time and amplitude in individual scintillating crystals in electromagnetic calorimeters and other detectors. The network performance is discussed as well as the application of xAI methods for further investigation of the algorithm and improvement of the output accuracy.

Figures

Figures reproduced from arXiv: 2504.17272 by the authors.

Figure 1
Figure 1. An example of a simulated event, used for training the MAC model. (Left) The simulated waveform (blue), used as input to the model and the model output (green), which aims to replicate the label (orange), used for this event. (Right) A detailed look into the region around the arrival of the first pulse from the same event with the simulated waveform (blue), the true desired output value (orange), and the actual mode… view at source ↗
Figure 2
Figure 2. shows the results from applying Occlusion Sensitivity to two events with a different number of pulses in them by monitoring the loss depending on the mask center position. An example of a masked event can be seen on the left panel of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. An event, masked in two different ways for Occlusion Sensitivity investigations. (Left) A mask of size 18 is moved along the waveform and the loss is calculated for each position of the mask center. The example shows the mask, placed in the signal fall region. The calculated loss for the data with a masked region is displayed in blue. The red portion of the waveform is set to 0. A fake prediction is made by the mode… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison of the output of the Occlusion Sensitivity method for different mask sizes on the same event. The same pattern is observed; however, larger mask sizes provide more detail in the total loss change. A more in-depth investigation of the most important regions o…
Figure 5
Figure 5. Figure 5: Output of the modified Occlusion Sensitivity for two events, each containing a single pulse. (Left) An event where the prediction is placed earlier than the rounded value position of the true arrival time. The loss remains constant at its lowest value after all the rel…
Figure 6
Figure 6. Figure 6: Distribution of the difference between the true and rounded values in the label arrival times (ttrue − tround) on the x axis and the difference between the predicted and rounded values in the label arrival time (tround − tpredicted) on the y axis. Events that actually …
Figure 7
Figure 7. Figure 7: Structure of the hidden layers of the MAC and the UMAC. On the left are the hidden layers in the MAC architecture. It consists of an encoder with three convolutional layers with dropout layers between them, followed by a decoder of three transposed convolution layers w…
Figure 8
Figure 8. Figure 8: Difference between the true and the predicted arrival time for the MAC (blue) and the UMAC (red). The MAC Gaussian fit shows a mean difference of 0.7 ns, due to the difference between the true signal arrival time and time bin position in the label. The UMAC reduces thi…
Figure 9
Figure 9. Figure 9: Difference between the true and the predicted arrival time versus the difference between true and predicted energy for the two models. (Left) The MAC shows an offset both in the predicted arrival time and the energy from the true values. (Right) The UMAC plot is center…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 17 canonical work pages

  1. [1]

    Novel 3-D clustering algorithm and two particle separation with tile HCAL

    Morgunov, V .; Raspereza, A. Novel 3-D clustering algorithm and two particle separation with tile HCAL. In Proceedings of the International Conference, LCWS 2004, Paris, France, 19–23 April 2004; pp. 431–436

  2. [2]

    [CALICE] Separation of two electromagnetic or electromagnetic-hadronic showers in CALICE SiW ECAL and ILD

    Shpak, K. [CALICE] Separation of two electromagnetic or electromagnetic-hadronic showers in CALICE SiW ECAL and ILD. In Proceedings of the International Workshop on Future Linear Colliders (LCWS2017), Strasbourg, France, 23–27 October 2017

  3. [3]

    Artificial Neural Networks on FPGAs for Real-Time Energy Reconstruction of the ATLAS LAr Calorimeters

    Aad, G.; Berthold, A.S.; Calvet, T.; Chiedde, N.; Fortin, E.M.; Fritzsche, N.; Hentges, R.; Olavi Laatu, L.A.; Monnier, E.; Straessner, A.; et al. Artificial Neural Networks on FPGAs for Real-Time Energy Reconstruction of the ATLAS LAr Calorimeters. Comput. Softw. Big Sci. 2021, 5, 19 [CrossRef]

  4. [4]

    Axiomatic attribution for deep networks

    Sundararajan, M.; Taly, A.; Yan, Q. Axiomatic attribution for deep networks. In ICML’17: Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017 ; PMLR: Birmingham, UK, 2017; pp. 3319–3328

  5. [5]

    SmoothGrad: Removing noise by adding noise

    Smilkov, D.; Thorat, N.; Kim, B.; Viégas, F.B.; Wattenberg, M. SmoothGrad: Removing noise by adding noise. arXiv 2017, arXiv:1706.03825

  6. [6]

    Occlusion Sensitivity Analysis with Augmentation Subspace Perturbation in Deep Feature Space

    Valois, P .; Niinuma, K.; Fukui, K. Occlusion Sensitivity Analysis with Augmentation Subspace Perturbation in Deep Feature Space. In Proceedings of the 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3–8 January 2024; pp. 4817–4826

  7. [7]

    Explainable Artificial Intelligence (XAI) on Timeseries Data: A Survey

    Rojat, T.; Puget, R.; Filliat, D.; Del Ser, J.; Gelin, R.; Díaz-Rodríguez, N. Explainable Artificial Intelligence (XAI) on Timeseries Data: A Survey. arXiv 2021, arXiv:2104.00950

  8. [8]

    TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems

    Abadi, M.; Agarwal, A.; Barham, P .; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. 2015. Available online: https://tensorflow.org (accessed on 22 January 2025)

Show all 19 references
  1. [9]

    Chollet, F. Keras. 2015. Available online: https://keras.io (accessed on 22 January 2025)

  2. [10]

    Matplotlib: A 2D Graphics Environment

    Hunter, J.D. Matplotlib: A 2D Graphics Environment. Comput. Sci. Eng. 2007, 9, 90–95. [CrossRef]

  3. [11]

    ROOT: An object oriented data analysis framework

    Brun, R.; Rademakers, F. ROOT: An object oriented data analysis framework. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip. 1997, 389, 81–86. [CrossRef] Particles 2025, 8, 48 11 of 11

  4. [12]

    Visualkeras

    Gavrikov, P . Visualkeras. 2020. Available online: https://github.com/paulgavrikov/visualkeras (accessed on 22 January 2025)

  5. [13]

    Using Artificial Intelligence in the Reconstruction of Signals from the PADME Electromag- netic Calorimeter

    Dimitrova, K.; PADME Collaboration. Using Artificial Intelligence in the Reconstruction of Signals from the PADME Electromag- netic Calorimeter. Instruments 2022, 6, 46. [CrossRef]

  6. [14]

    Pattern recognition and signal parameters extraction using machine learning methods

    Buchakchiev, V .; Dimitrova, K.; Georgiev, G.; Georgieva, G.; Kozhuharov, V . Pattern recognition and signal parameters extraction using machine learning methods. J. Phys. Conf. Ser. 2023, 2668, 012001. [CrossRef]

  7. [15]

    Convolutional Autoencoders for Signal Reconstruction and their Application to Damage Signature Extraction

    Azuara de Pablo, G.; Barrera López de Turiso, E.; Ruiz González, M.; Barba Navarrete, P .L.; Piñas Higueruela, A. Convolutional Autoencoders for Signal Reconstruction and their Application to Damage Signature Extraction. In Proceedings of the 10th European Workshop on Structur...

  8. [16]

    6; V1751 Technical Information Manual Rev

    CAEN 2012 V1742 Technical Information Manual Rev. 6; V1751 Technical Information Manual Rev. 12. Available online: http://www.caen.it (accessed on 22 January 2025)

  9. [17]

    Machine learning assisted reconstruction of positron-on-target annihilation events in the PADME experiment

    Dimitrova, K.; PADME Collaboration. Machine learning assisted reconstruction of positron-on-target annihilation events in the PADME experiment. J. Phys. Conf. Ser. 2024, 2794, 012001. [CrossRef]

  10. [18]

    Applicability evaluation of selected xAI methods for machine learning algorithms for signal parameters extraction

    Dimitrova, K.; Kozhuharov, V .; Petkov, P . Applicability evaluation of selected xAI methods for machine learning algorithms for signal parameters extraction. In Proceedings of the Second Workshop on Soliton Theory, Nonlinear Dynamics and Machine Learning, Varna, Bulgaria, 16–...

  11. [19]

    Visualizing and Understanding Convolutional Networks

    Zeiler, M.D.; Fergus, R. Visualizing and Understanding Convolutional Networks. In Computer Vision—ECCV 2014, Proceedings of the 13th European Conference, Zurich, Switzerland, 6–12 September 2014 ; Springer: Berlin/Heidelberg, Germany, 2014; pp. 818–833. Disclaimer/Publisher’s ...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.