Pith. sign in

REVIEW 3 major objections 6 minor 24 references

Double Descent and Overparameterization in Particle Physics Data

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that double descent—where test error first worsens then improves as model capacity or training time grows—appears in particle physics data, shown for jet pT regression and event classification, and that overparameterized mo

desk verdict A credible first look at double descent in HEP data, but the practical benefit claim rests on test-label oracle checkpoint selection and single runs. read the letter →

arxiv 2509.01397 v1 pith:JUWO6FLX submitted 2025-09-01 hep-ex cs.LGhep-phphysics.data-an

Matthias Vigl , Lukas Heinrich This is my paper
classification hep-excs.LGhep-phphysics.data-an
keywords doubledescentoverparameterizationparticlephysicsjetpTregressioneventclassificationepoch-wisemodel-wiseearlystopping
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that double descent, a non-monotonic generalization curve previously seen in images, quantum systems, and protein folding, also occurs in particle physics data. It reports the first explicit instances of both model-wise double descent (across model width) and epoch-wise double descent (across training time) in a jet pT regression task, and model-wise double descent in a SUSY-versus-Standard-Model event classification task. It further claims that when the best test-loss checkpoint is selected, a network with roughly one billion parameters can beat classical models even on a modest 150k-event dataset. The broader point is that generalization behavior is task- and dataset-dependent, so scaling decisions in particle physics should be informed by where overparameterization actually helps.

What carries the argument

The central object is the double-descent curve: test error plotted against model width (model-wise) or training epoch (epoch-wise), anchored by the interpolation threshold, the capacity at which training loss first vanishes. In the jet regression task, the scaling parameter is the transformer width d_model; in classification, it is the hidden-layer width of a three-layer or sixteen-layer MLP. The experimental protocol holds optimizer and batch size fixed across all model sizes—Adam with learning rate 1e-4, batch size 128, and a 5% warmup for transformers—so that differences in test loss are attributed to model size and training time rather than to optimizer choices.

What would settle it

Retrain the same model families with several random seeds and a small grid of learning rates, then measure test loss at each width and epoch; the central claim weakens if the double-descent peaks and the overparameterization benefit do not reproduce. Alternatively, rerun the early-stopping comparison using a held-out validation set instead of the test set; if the 1B-model advantage disappears, the benefit claim depends on leaking test information into checkpoint selection.

Watch

Extended reading notes

Core claim

The central discovery is empirical: neural networks trained on particle physics data reproduce the non-monotonic generalization curves associated with double descent. In jet transverse-momentum regression with a two-layer transformer, test loss as a function of model width peaks near the interpolation threshold—where training loss first reaches zero—and then decreases again, which is model-wise double descent. Along training time, test loss also decreases, increases, and then decreases again, which is epoch-wise double descent. In a 15-way event classification task, model-wise double descent appears when models train far past the interpolation threshold; if instead the checkpoint with the be

Load-bearing premise

The generalization curves rest on single runs with fixed Adam hyperparameters, so differences across widths and epochs could in principle be artifacts of training dynamics or randomness rather than genuine double descent; the overparameterization benefit additionally depends on selecting checkpoints by test loss.

Editorial extensions

If this is right

  • If correct, test error in particle physics tasks is not a monotone function of model size or training time, so model selection must look beyond the interpolation peak and beyond the first test-loss minimum.
  • Overparameterized networks can, under best-test-checkpoint selection, outperform smaller classical models even with modest training data, supporting continued scaling of high-capacity models in the field.
  • Double descent is not universal across particle physics tasks: jet pT regression shows both model-wise and epoch-wise double descent, while event classification shows model-wise but not always epoch-wise double descent.
  • Class imbalance is a plausible driver of epoch-wise double descent in classification, since balanced and imbalanced datasets of the same task behave differently.
  • The benefit of overparameterization in the small-data classification case appears only when the best test-loss checkpoint is chosen, not when comparing final checkpoints.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: reweight the imbalanced 7M-event dataset to equal class frequencies while keeping its size, to confirm that class imbalance, rather than dataset size or process composition, triggers epoch-wise double descent.
  • The best-test-checkpoint comparison leaks test information into model selection; a fairer practical rule would use a held-out validation set, and the claim that 1B-parameter models beat classical models should be rechecked under that rule.
  • If the observed curves are stable across random seeds and optimizer settings, particle physics practitioners could adopt a simple recipe: train wide models, evaluate checkpoints on a validation set, and expect the interpolation peak to be avoidable.
  • The results imply that foundation-model scaling studies in high-energy physics should include per-task scans over model width and training time, because aggregate scaling laws may hide non-monotonic generalization behavior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper reports empirical observations of double-descent behavior in particle physics datasets. Using the ATLAS JetSet jet-pT regression task and the ATLAS SUSY Wh1Lbb event-classification dataset, the authors train transformers and MLPs of varying widths and show model-wise and epoch-wise non-monotonic test-loss curves. They claim the first demonstration of double descent in particle physics data, and further claim that in some settings overparameterized models outperform classical models when the best test-loss epoch checkpoint is selected. The paper also compares balanced and unbalanced classification datasets to study the dependence of epoch-wise double descent on class imbalance.

Significance. If the empirical claims are substantiated, the paper would provide useful domain-specific evidence on generalization in HEP machine learning, with implications for the current trend toward large foundation models in particle physics. The paper has strengths: it uses public ATLAS datasets, a transparent training setup, and clear visualizations, and the authors state that code is released. However, the headline claims rest on three load-bearing methodological choices that are not yet adequately supported: the use of oracle test-based early stopping, the absence of uncertainty quantification, and the confounding of class balance with dataset size. These issues can be addressed with additional experiments, so the contribution is potentially valid but requires major revision.

major comments (3)
  1. [Sec. 4.2, Fig. 3] The early-stopping curves are obtained by 'taking the epoch checkpoint with the best test-set performance'. This is an oracle selection rule: the test labels are used to choose the training epoch, so the reported early-stopping value is the running minimum of the test-loss trajectory. A larger model can reach a lower oracle minimum even when its final test error is worse; the apparent benefit of the 1B-parameter model may reflect trajectory flexibility rather than better generalization. The 'performance gain' claim in the abstract is therefore not supported as stated. Please redo the model-selection comparison using a validation split for checkpoint selection and report held-out test error, alongside the final-error curves.
  2. [Sec. 2, Figs. 1-3] All reported curves appear to be single training runs with no seeds, error bars, or confidence intervals. The double-descent signature is a non-monotonicity in test loss; without an estimate of run-to-run variance, the visual features in Figs. 1-3 could be attributable to optimization noise, especially near the interpolation threshold where training dynamics are sensitive. Please run several seeds per configuration and report the spread, or otherwise justify that the phenomena are reproducible across random initializations and data subsamples. This is central to the claim of observing double descent for the first time in particle physics data.
  3. [Sec. 4.1, Fig. 2] The comparison used to support the 'prior dependence' claim contrasts a balanced 800k-event dataset with an unbalanced 7M-event dataset. These two settings differ both in class balance and in total dataset size, so the observed difference in epoch-wise double descent cannot be attributed to class imbalance alone. The hypothesis that class imbalance affects feature-learning speeds is plausible but untested as presented. A controlled comparison should vary class proportions at fixed dataset size, or vary dataset size at fixed class proportions, before drawing this conclusion.
minor comments (6)
  1. [Sec. 2] Typo: 'straining steps' should be 'training steps'.
  2. [Sec. 3] Typo: 'is a interesting future research direction' should be 'is an interesting future research direction'.
  3. [General] The paper states 'we release code' but provides no repository URL or DOI. Please include the actual location of the code.
  4. [Fig. 3 caption] The caption says 'overparameterised models outperform underparameterised ones', but this is only true under the oracle early-stopping rule. Please qualify this statement or revise it to match the final-error results.
  5. [Fig. 1d] Please state explicitly what the 'Early Stopping' curve represents (best test loss over training?) and how it is computed, since this terminology is nonstandard and affects interpretation.
  6. [Sec. 2] The preprocessing of JetSet (track selection, normalization, tokenization) and the exact train/test split are not described. Please provide enough detail to make the experiments reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical observations on public benchmarks, no derivation from fitted inputs.

full rationale

This paper is an empirical observation study. The central claims—model-wise and epoch-wise double descent, and the overparameterization benefit under early stopping—are read directly from train/test loss curves of models trained on public ATLAS datasets. No parameter is fitted to a subset of data and then used to predict a closely related quantity. Self-citations are contextual and not load-bearing. The only methodological caveat is the oracle test-set checkpoint selection in §4.2, which is a validation concern, not circularity: the early-stopping curve is an empirical minimum over epochs, not a quantity equal to the model definition or to the input by construction. Therefore score 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper's claims are empirical observations, so the ledger records hand-chosen experimental settings rather than fitted derivations. No parameters are fitted to produce a prediction, which keeps the circularity burden low. The load-bearing choices are the fixed optimizer settings across the scaling sweep, the single-seed curves, and the test-based early stopping rule; the paper introduces no new physical entities.

free parameters (5)
  • Adam learning rate = 1e-4
    Fixed by hand across all model sizes (Sec 2); no per-width tuning, so optimization quality is not matched across the scaling sweep.
  • Batch size = 128
    Fixed by hand (Sec 2); chosen to isolate model-size effects but itself affects where the interpolation threshold sits.
  • Warmup schedule = 5% of training steps for transformers, none for MLPs
    Hand-chosen (Sec 2); differs between model families, so the jet and classification experiments are not directly comparable in optimization dynamics.
  • Training horizon = up to 4k epochs (jet task); unspecified for classification
    Hand-chosen (Sec 3); epoch-wise double descent is defined relative to this horizon, and longer training could change the curves.
  • Early stopping rule = checkpoint with best test loss
    Hand-chosen selection rule (Sec 4.2); load-bearing for the claim that overparameterized models outperform, and it uses test labels for selection.
assumptions (4)
  • domain assumption Single-run test-loss curves without error bars are representative of each model family's generalization behavior.
    Figs 1-3 show one curve per configuration; no seeds, variance, or significance tests are reported.
  • domain assumption The public ATLAS datasets (JetSet, SUSY Wh1Lbb) are used as provided, with implicit train/test splits that are unbiased for the stated tasks.
    Sec 2 cites the open-data releases; split construction and preprocessing details are not given.
  • domain assumption Adam training at fixed hyperparameters reaches the interpolation threshold for large widths within the stated epochs.
    Fig 1a shows train loss vanishing; the paper relies on this to place the interpolation threshold.
  • ad hoc to paper Epoch-wise double descent observed in the unbalanced dataset is attributable to class imbalance.
    Sec 4.1 and Fig 2 compare 800k balanced vs 7M unbalanced events, conflating class balance with dataset size; the attribution is the paper's hypothesis, not a controlled result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Double Descent and Overparameterization in Particle Physics Data." pith.science (2026). https://pith.science/paper/JUWO6FLX

@misc{pith2026250901397,
  author       = {Pith},
  title        = {Pith review of: Double Descent and Overparameterization in Particle Physics Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JUWO6FLX}},
  note         = {Machine review of arXiv:2509.01397}
}
read the original abstract

Recently, the benefit of heavily overparameterized models has been observed in machine learning tasks: models with enough capacity to easily cross the \emph{interpolation threshold} improve in generalization error compared to the classical bias-variance tradeoff regime. We demonstrate this behavior for the first time in particle physics data and explore when and where `double descent' appears and under which circumstances overparameterization results in a performance gain.

Figures

Figures reproduced from arXiv: 2509.01397 by the authors.

Figure 1
Figure 1. (a,b): Train and test jet pT regression loss as a function of model size and epoch, model￾and epoch-wise double descent can be observed looking at horizontal and vertical slices. (c): Vertical slices of plot (b) showing epoch-wise double descent. (d): Model-wise double descent. 4 Event classification: Prior Dependence and Early-Stopping Behavior The rich context dependence of generalization performance of neural net… view at source ↗
Figure 2
Figure 2. Large models trained on the 800k balanced dataset (left) do not exhibit epoch-wise double [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Model-wise double descent for a 3-layer MLP trained on 3k events (left) and a 16-layer [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 10 canonical work pages

  1. [1]

    Dillon, Malcolm Fairbairn, Darius A

    Gregor Kasieczka, Tilman Plehn, Anja Butter, Kyle Cranmer, Dipsikha Debnath, Barry M. Dillon, Malcolm Fairbairn, Darius A. Faroughy, Wojtek Fedorko, Christophe Gay, Loukas Gouskos, Jernej Fesel Kamenik, Patrick Komiske, Simon Leiss, Alison Lister, Sebastian Macaluso, Eric Metodiev, Liam Moore, Benjamin Nachman, Karl Nordström, Jannicke Pearkes, Huilin Qu,...

    work page 2019

  2. [2]

    Komiske, Eric M

    Patrick T. Komiske, Eric M. Metodiev, and Jesse Thaler. Energy Flow Networks: Deep Sets for Particle Jets. JHEP, 01:121, 2019. doi: 10.1007/JHEP01(2019)121

    doi:10.1007/jhep01(2019)121 2019

  3. [3]

    MLPF: efficient machine-learned particle-flow reconstruction using graph neural networks

    Joosep Pata, Javier Duarte, Jean-Roch Vlimant, Maurizio Pierini, and Maria Spiropulu. MLPF: efficient machine-learned particle-flow reconstruction using graph neural networks. The Eu- ropean Physical Journal C, 81(5), may 2021. doi: 10.1140/epjc/s10052-021-09158-w. URL https://doi.org/10.1140%2Fepjc%2Fs10052-021-09158-w

    work page doi:10.1140/epjc/s10052-021-09158-w 2021

  4. [4]

    Reconstructing particles in jets using set transformer and hypergraph prediction networks

    Francesco Armando Di Bello, Etienne Dreyer, Sanmay Ganguly, Eilam Gross, Lukas Heinrich, Anna Ivina, Marumi Kado, Nilotpal Kakati, Lorenzo Santi, Jonathan Shlomi, and Matteo Tusoni. Reconstructing particles in jets using set transformer and hypergraph prediction networks. The European Physical Journal C, 83(7), jul 2023. doi: 10.1140/epjc/s10052-023-11677...

    doi:10.1140/epjc/s10052-023-11677-7 2023

  5. [5]

    Calogan: Simulating 3d high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks

    Michela Paganini, Luke de Oliveira, and Benjamin Nachman. Calogan: Simulating 3d high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks. Physical Review D, 97(1), January 2018. ISSN 2470-0029. doi: 10.1103/physrevd. 97.014021. URL http://dx.doi.org/10.1103/PhysRevD.97.014021

    doi:10.1103/physrevd 2018

  6. [6]

    Deep generative models for detector signature simulation: A taxonomic review

    Baran Hashemi and Claudius Krause. Deep generative models for detector signature simulation: A taxonomic review. Reviews in Physics, 12:100092, December 2024. ISSN 2405-4283. doi: 10. 1016/j.revip.2024.100092. URL http://dx.doi.org/10.1016/j.revip.2024.100092

    arXiv 2024

  7. [7]

    Is Tokenization Needed for Masked Particle Modelling? 9 2024

    Matthew Leigh, Samuel Klein, François Charton, Tobias Golling, Lukas Heinrich, Michael Kagan, Inês Ochoa, and Margarita Osadchy. Is Tokenization Needed for Masked Particle Modelling? 9 2024

    work page 2024

  8. [8]

    Masked particle modeling on sets: towards self-supervised high energy physics foundation models

    Tobias Golling, Lukas Heinrich, Michael Kagan, Samuel Klein, Matthew Leigh, Margarita Osadchy, and John Andrew Raine. Masked particle modeling on sets: towards self-supervised high energy physics foundation models. Mach. Learn. Sci. Tech. , 5(3):035074, 2024. doi: 10.1088/2632-2153/ad64a8

    doi:10.1088/2632-2153/ad64a8 2024

Show all 24 references
  1. [9]

    Finetuning foundation models for joint analysis optimization in High Energy Physics

    Matthias Vigl, Nicole Hartman, and Lukas Heinrich. Finetuning foundation models for joint analysis optimization in High Energy Physics. Mach. Learn. Sci. Tech., 5(2):025075, 2024. doi: 10.1088/2632-2153/ad55a3

    2024 doi

  2. [10]

    OmniJet-α_C: learning point cloud calorimeter simulations using generative transformers

    Joschka Birk, Frank Gaede, Anna Hallin, Gregor Kasieczka, Martina Mozzanica, and Henning Rose. OmniJet-α_C: learning point cloud calorimeter simulations using generative transformers. JINST, 20(07):P07007, 2025. doi: 10.1088/1748-0221/20/07/P07007

    2025 doi

  3. [11]

    OmniJet-α: the first cross-task foundation model for particle physics.Mach

    Joschka Birk, Anna Hallin, and Gregor Kasieczka. OmniJet-α: the first cross-task foundation model for particle physics.Mach. Learn. Sci. Tech., 5(3):035031, 2024. doi: 10.1088/2632-2153/ ad66ad

    2024 doi

  4. [12]

    Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data

    James Giroux and Cristiano Fanelli. Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data. 5 2025. 5

    2025

  5. [13]

    Deep double descent: Where bigger models and more data hurt, 2019

    Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt, 2019. URL https://arxiv. org/abs/1912.02292

    2019 arXiv

  6. [14]

    The implicit bias of gradient descent on separable data

    Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent on separable data. Journal of Machine Learning Research, 19 (70):1–57, 2018

    2018

  7. [15]

    When and how epochwise double descent happens, 2021

    Cory Stephenson and Tyler Lee. When and how epochwise double descent happens, 2021. URL https://arxiv.org/abs/2108.12006

    2021 arXiv

  8. [16]

    Double descent in quantum machine learning

    Marie Kempkes, Aroosa Ijaz, Elies Gil-Fuster, Carlos Bravo-Prieto, Jakob Spiegelberg, Evert van Nieuwenburg, and Vedran Dunjko. Double descent in quantum machine learning. 1 2025

    2025

  9. [17]

    Open- fold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization

    Gustaf Ahdritz, Nazim Bouatta, Christina Floristean, Sachin Kadyan, Qinghui Xia, William Gerecke, Timothy J O’Donnell, Daniel Berenberg, Ian Fisk, Niccolò Zanichelli, et al. Open- fold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for gen...

    2024

  10. [18]

    The ATLAS Experiment at the CERN Large Hadron Collider

    ATLAS Collaboration. The ATLAS Experiment at the CERN Large Hadron Collider. JINST, 3: S08003, 2008. doi: 10.1088/1748-0221/3/08/S08003

    2008 doi

  11. [19]

    ATLAS ttˆ Simulation for ML-based Jet Flavour Tagging (JetSet)

    ATLAS Collaboration. ATLAS ttˆ Simulation for ML-based Jet Flavour Tagging (JetSet). CERN Open Data Portal, 2025

    2025

  12. [20]

    Salam, and Gregory Soyez

    Matteo Cacciari, Gavin P. Salam, and Gregory Soyez. The anti- kt jet clustering algorithm. JHEP, 04:063, 2008. doi: 10.1088/1126-6708/2008/04/063

    2008 doi

  13. [21]

    Transforming jet flavour tagging at atlas, 2025

    ATLAS Collaboration. Transforming jet flavour tagging at atlas, 2025. URL https://arxiv. org/abs/2505.19689

    2025

  14. [22]

    ATLAS SUSY Searches in Wh1Lbb Channel Open Data Set

    ATLAS Collaboration. ATLAS SUSY Searches in Wh1Lbb Channel Open Data Set. CERN Open Data Portal, 2024

    2024

  15. [23]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv e-prints, art. arXiv:1412.6980, December 2014. doi: 10.48550/arXiv.1412.6980

    2014 doi arXiv

  16. [24]

    STEPHEN P. MARTIN. A SUPERSYMMETRY PRIMER , page 1–98. WORLD SCIEN- TIFIC, July 1998. doi: 10.1142/9789812839657_0001. URL http://dx.doi.org/10.1142/ 9789812839657_0001. 6

    1998 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.