Pith. sign in

REVIEW 2 major objections 1 minor 1 cited by

Pretraining data composition can shift neural scaling laws for jet classification to favor more data over larger models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 11:25 UTC pith:X5WVKWM6

load-bearing objection Pretraining data composition can shift scaling exponents toward data-heavy regimes in jet classification, but the abstract leaves open whether architecture or fine-tuning choices drive the effect instead. the 2 major comments →

arxiv 2606.19781 v2 pith:X5WVKWM6 submitted 2026-06-18 hep-ex cs.AI

Towards Engineering Scaling Laws with Pretraining Data Composition

Jan-Lucas Uslu , Kevin Greif , Daniel Whiteson , Benjamin Nachman This is my paper
classification hep-ex cs.AI
keywords neural scaling lawspretraining data compositionjet classificationhadronic jetsparticle physicssynthetic datapower-law scaling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Neural scaling laws describe performance improving as power laws in model size and dataset size. Particle physics has cheap synthetic data from high-fidelity simulators, so the pretraining dataset itself can be designed. The work shows that choosing pretraining data that is more diverse and better aligned with the downstream jet classification task changes the observed scaling exponents. This favors regimes where adding data improves performance more than adding parameters. A sympathetic reader cares because it lets compute be spent on the cheaper resource when simulators are available.

Core claim

For the task of classifying hadronic jets produced in high-energy collisions, the scaling behavior can be engineered towards requiring more data rather than larger models by inclusion of pretraining data which is more diverse and better aligned with the downstream classification task.

What carries the argument

Pretraining data composition, defined by its diversity and alignment with the jet classification task, which alters the power-law scaling exponents between performance, model size, and data volume.

Load-bearing premise

Differences in observed scaling exponents arise primarily from the pretraining data composition rather than from unstated choices in model architecture, optimizer, or downstream fine-tuning procedure.

What would settle it

An experiment that fixes architecture, optimizer, and fine-tuning procedure but varies only the diversity and alignment of the pretraining data and finds identical scaling exponents would falsify the central claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Scaling exponents become tunable by deliberate choice of pretraining data rather than fixed properties of the task.
  • When simulators make data cheap, optimal compute allocation shifts toward larger datasets and smaller models.
  • Performance on jet classification improves more rapidly with added examples once the pretraining mix is better aligned.
  • The same simulator pipeline can be reused to generate task-specific pretraining sets that change scaling behavior.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same data-composition approach could be tested in other domains that have cheap simulators, such as certain chemistry or materials tasks.
  • Quantitative measures of data diversity and task alignment could be developed to predict the direction and size of scaling shifts in advance.
  • Hybrid pretraining mixes that blend real collision data with simulated events might produce intermediate scaling regimes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The manuscript investigates neural scaling laws for hadronic jet classification in high-energy physics. It claims that pretraining data composition can be engineered—specifically by using more diverse and task-aligned synthetic data—to shift scaling behavior toward regimes that favor additional data over larger models, leveraging the availability of cheap high-fidelity simulators in particle physics.

Significance. If the central empirical claim holds after proper controls, the result would be significant for ML applications in HEP. It identifies a domain-specific lever (data composition) to influence scaling exponents in a setting where data is abundant relative to parameters, potentially enabling more efficient model development than in language or vision domains. The work also highlights an opportunity to move beyond passive observation of scaling laws toward active engineering.

major comments (2)
  1. [Methods / Experimental Setup] The central claim requires that observed differences in scaling exponents be attributable to pretraining data composition rather than confounding factors. The manuscript provides no description of whether model architecture, optimizer settings, learning-rate schedules, or downstream fine-tuning procedures were held fixed (or explicitly ablated) across data-composition variants. Without such controls the attribution cannot be isolated.
  2. [Results] No quantitative results, fitted scaling exponents, error bars, or tables comparing scaling regimes across data compositions are visible. The abstract states the claim at a high level only, preventing evaluation of effect size or statistical support for the data-vs-parameter shift.
minor comments (1)
  1. [Abstract] The abstract would be strengthened by a single sentence summarizing the specific jet classification task, datasets, and model family used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful review and constructive feedback. The two major comments identify areas where the manuscript can be strengthened for clarity and rigor. We address each point below and will incorporate revisions as noted.

read point-by-point responses

  1. Referee: [Methods / Experimental Setup] The central claim requires that observed differences in scaling exponents be attributable to pretraining data composition rather than confounding factors. The manuscript provides no description of whether model architecture, optimizer settings, learning-rate schedules, or downstream fine-tuning procedures were held fixed (or explicitly ablated) across data-composition variants. Without such controls the attribution cannot be isolated.

    Authors: We agree that explicit documentation of controls is necessary to support attribution to data composition. All experiments used the same model architecture, optimizer (AdamW), learning-rate schedule (cosine annealing with warmup), and fine-tuning protocol; these were not varied or ablated because the experimental design held them fixed by construction. The revised manuscript will add a dedicated Methods subsection that states these controls explicitly and notes that no hyperparameter ablations were performed on them. revision: yes

  2. Referee: [Results] No quantitative results, fitted scaling exponents, error bars, or tables comparing scaling regimes across data compositions are visible. The abstract states the claim at a high level only, preventing evaluation of effect size or statistical support for the data-vs-parameter shift.

    Authors: The manuscript presents scaling curves in the Results section that illustrate the shift toward data-heavy regimes. We acknowledge that fitted exponents, associated uncertainties, and a comparative table are not currently included and would improve evaluability. The revised version will add these quantitative elements, including the power-law fits, error estimates from the fitting procedure, and a summary table of scaling exponents across data compositions. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical scaling comparisons are self-contained

full rationale

The manuscript describes an empirical study comparing scaling exponents for jet classification under different pretraining data compositions. No equations, fitted parameters, or predictions are presented that reduce to inputs by construction. No self-citation chains, uniqueness theorems, or ansatzes are invoked as load-bearing steps. The central claim rests on direct performance measurements across data variants, which constitutes independent empirical content rather than a definitional or fitted tautology.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no equations, no fitted parameters, and no explicit axioms or invented entities; all ledger entries are therefore empty.

pith-pipeline@v0.9.1-grok · 5665 in / 1020 out tokens · 24432 ms · 2026-06-30T11:25:08.861523+00:00 · methodology

0 comments
read the original abstract

Neural scaling laws describe how model performance improves as a power law in compute, model size, and dataset size. While well-established for large language models, these relationships are emerging for large models in particle physics. As with language, empirical studies show that the performance scales as a power law. However, unlike natural language or image domains, fundamental physics has high-fidelity simulators that produce synthetic data cheaply. This favors scaling regimes where additional data is cheaper than additional parameters, and allows the pretraining dataset itself to be engineered to influence the scaling. For the task of classifying hadronic jets produced in collisions of high-energy particle beams, we show that the scaling behavior can be engineered towards requiring more data rather than larger models by inclusion of pretraining data which is more diverse and better aligned with the downstream classification task.

Figures

Figures reproduced from arXiv: 2606.19781 by Benjamin Nachman, Daniel Whiteson, Jan-Lucas Uslu, Kevin Greif.

Figure 1
Figure 1. Figure 1: Scaling analysis using test loss on JetClass after scratch training (a) and QCD + res2p + [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Scaling diagnostics for the intermediate pretraining configurations. The panels follow [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Fine-tuning loss curves as a function of training steps for each model size, comparing [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Predict before you train: Scaling Laws for particle physics foundation models

    hep-ex 2026-07 conditional novelty 7.0

    A Chinchilla-style law fit on ParticleViT runs below 10^19 FLOPs predicts held-out pretraining loss within ~1% at >100× compute and tracks downstream jet-tagging rejection.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages · cited by 1 Pith paper · 7 internal anchors

  1. [1]

    Scaling Laws for Neural Language Models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  2. [2]

    Training compute-optimal large language models

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. InAdvances in Neural Information Pro- cessing Systems (NeurIPS), 2022

    work page 2022

  3. [3]

    Overtones and nonlinearities in bi- nary black hole ringdowns.Phys

    Vinicius Mikuni and Benjamin Nachman. OmniLearn: A method to simultaneously facilitate all jet physics tasks.Physical Review D, 111:054015, 2025. doi: 10.1103/PhysRevD.111. 054015

    work page doi:10.1103/physrevd.111 2025

  4. [4]

    OmniLearned: A foundation model framework for all tasks involving jet physics.Physical Review D, 113: 032020, 2026

    Wahid Bhimji, Chris Harris, Vinicius Mikuni, and Benjamin Nachman. OmniLearned: A foundation model framework for all tasks involving jet physics.Physical Review D, 113: 032020, 2026. doi: 10.1103/PhysRevD.113.032020

    work page doi:10.1103/physrevd.113.032020 2026

  5. [5]

    OmniJet-α: The first cross-task foundation model for particle physics.Machine Learning: Science and Technology, 5:035031, 2024

    Joschka Birk, Anna Hallin, and Gregor Kasieczka. OmniJet-α: The first cross-task foundation model for particle physics.Machine Learning: Science and Technology, 5:035031, 2024. doi: 10.1088/2632-2153/ad66ad

    work page doi:10.1088/2632-2153/ad66ad 2024

  6. [6]

    Pretrained Event Classification Model for High Energy Physics Analysis

    Joshua Ho, Benjamin Ryan Roberts, Shuo Han, and Haichen Wang. Pretrained Event Classifi- cation Model for High Energy Physics Analysis.arXiv preprint arXiv:2412.10665, 12 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    Accelerating resonance searches via signature-oriented pre-training.arXiv preprint arXiv:2405.12972, 2024

    Congqiao Li, Antonios Agapitos, Jovin Drews, Javier Duarte, Dawei Fu, Leyun Gao, Raghav Kansal, Gregor Kasieczka, Louis Moureaux, Huilin Qu, Cristina Mantilla Suarez, and Qiang Li. Accelerating resonance searches via signature-oriented pre-training.arXiv preprint arXiv:2405.12972, 2024

    work page arXiv 2024

  8. [8]

    Masked particle modeling on sets: towards self-supervised high energy physics foundation models.Mach

    Tobias Golling, Lukas Heinrich, Michael Kagan, Samuel Klein, Matthew Leigh, Margarita Osadchy, and John Andrew Raine. Masked particle modeling on sets: towards self-supervised high energy physics foundation models.Mach. Learn. Sci. Tech., 5(3):035074, 2024. doi: 10.1088/2632-2153/ad64a8

    work page doi:10.1088/2632-2153/ad64a8 2024

  9. [9]

    Resimulation-based self-supervised learning for pretraining physics foundation models.Phys

    Philip Harris, Jeffrey Krupa, Michael Kagan, Benedikt Maier, and Nathaniel Woodward. Resimulation-based self-supervised learning for pretraining physics foundation models.Phys. Rev. D, 111(3):032010, 2025. doi: 10.1103/PhysRevD.111.032010

    work page doi:10.1103/physrevd.111.032010 2025

  10. [10]

    Learning Symmetry-Independent Jet Representations via Jet-Based Joint Embedding Predic- tive Architecture

    Subash Katel, Haoyang Li, Zihan Zhao, Farouk Mokhtar, Javier Duarte, and Raghav Kansal. Learning Symmetry-Independent Jet Representations via Jet-Based Joint Embedding Predic- tive Architecture. InPostponed: Machine Learning and the Physical Sciences: Workshop at NeurIPS 2024, 12 2024

    work page 2024

  11. [11]

    HEP-JEPA: A foundation model for collider physics using joint embedding predictive architecture

    Jai Bardhan, Radhikesh Agrawal, Abhiram Tilak, Cyrin Neeraj, and Subhadip Mitra. HEP- JEPA: A foundation model for collider physics using joint embedding predictive architecture. arXiv preprint arXiv:2502.03933, 2 2025

    work page arXiv 2025

  12. [12]

    Wildridge, Jack P

    Andrew J. Wildridge, Jack P. Rodgers, Ethan M. Colbert, Yao yao, Andreas W. Jung, and Miaoyuan Liu. Bumblebee: Foundation Model for Particle Physics Discovery. InPostponed: Machine Learning and the Physical Sciences: Workshop at NeurIPS 2024, 12 2024

    work page 2024

  13. [13]

    Foundation models for high-energy physics.arXiv preprint arXiv:2509.21434, 2025

    Anna Hallin. Foundation models for high-energy physics.arXiv preprint arXiv:2509.21434, 2025

    work page arXiv 2025

  14. [14]

    FM4NPP: A scaling foundation model for nuclear and particle physics

    David Park et al. FM4NPP: A scaling foundation model for nuclear and particle physics. In International Conference on Learning Representations (ICLR), 2026

    work page 2026

  15. [15]

    Solving key challenges in collider physics with foun- dation models.Phys

    Vinicius Mikuni and Benjamin Nachman. Solving key challenges in collider physics with foun- dation models.Phys. Rev. D, 111(5):L051504, 2025. doi: 10.1103/PhysRevD.111.L051504

    work page doi:10.1103/physrevd.111.l051504 2025

  16. [16]

    https://arxiv.org/abs/2510.17960

    Liam Parker et al. AION-1: Omnimodal Foundation Model for Astronomical Sciences.arXiv preprint arXiv:2510.17960, 10 2025. 10

    work page arXiv 2025

  17. [17]

    Wells, Salman Habib, and John Wise

    Bin Xia, Nesar Ramachandra, Azton I. Wells, Salman Habib, and John Wise. Multi-modal foundation model for cosmological simulation data, 2025. URLhttps://arxiv.org/abs/ 2510.07684

    work page arXiv 2025

  18. [18]

    EveNet: A Foundation Model for Particle Collision Data Analysis

    Ting-Hsiang Hsu et al. EveNet: A Foundation Model for Particle Collision Data Analysis. arXiv preprint arXiv:2601.17126, 1 2026

    work page arXiv 2026

  19. [19]

    Particle trajectory representation learning with masked point modeling.Mach

    Samuel Young, Yeon-jae Jwa, and Kazuhiro Terao. Particle trajectory representation learning with masked point modeling.Mach. Learn. Sci. Tech., 7(2):025023, 2026. doi: 10.1088/ 2632-2153/ae47b8

    work page 2026

  20. [20]

    Panda: Self-distillation of Reusable Sensor-level Repre- sentations for High Energy Physics.arXiv preprint arXiv:2512.01324, 12 2025

    Samuel Young and Kazuhiro Terao. Panda: Self-distillation of Reusable Sensor-level Repre- sentations for High Energy Physics.arXiv preprint arXiv:2512.01324, 12 2025

    work page arXiv 2025

  21. [21]

    Scaling laws in jet classification

    Joshua Batson and Yonatan Kahn. Scaling laws in jet classification.SciPost Physics Core, 8: 034, 2025. doi: 10.21468/SciPostPhysCore.8.1.034

    work page doi:10.21468/scipostphyscore.8.1.034 2025

  22. [22]

    arXiv preprint arXiv:2602.15781 , year=

    Matthias Vigl, Nicole Hartman, Michael Kagan, and Lukas Heinrich. Neural scaling laws for boosted jet tagging.arXiv preprint arXiv:2602.15781, 2026

    work page arXiv 2026

  23. [23]

    Carpe Datum: Scaling behavior of transformers for heavy hadron flavor identification

    ATLAS Collaboration. Carpe Datum: Scaling behavior of transformers for heavy hadron flavor identification. Technical report, CERN, Geneva, 2026. URLhttps:// cds.cern.ch/record/2953659. All figures including auxiliary figures are available at https://atlas.web.cern.ch/Atlas/GROUPS/PHYSICS/PUBNOTES/ATL-SOFT-PUB-2026-002

    work page arXiv 2026

  24. [24]

    Scaling laws for amplitude surrogates.arXiv preprint arXiv:2601.13308, 2026

    Henning Bahl, Victor Bres ´o-Pla, Anja Butter, and Joaquin Iturriza Ramirez. Scaling laws for amplitude surrogates.arXiv preprint arXiv:2601.13308, 2026

    work page arXiv 2026

  25. [25]

    Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari S. Morcos. Beyond neural scaling laws: Beating power law scaling via data pruning. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    work page 2022

  26. [26]

    Particle transformer for jet tagging

    Huilin Qu, Congqiao Li, and Sitian Qian. Particle transformer for jet tagging. InInternational Conference on Machine Learning (ICML), 2022

    work page 2022

  27. [27]

    Deep Learning Scaling is Predictable, Empirically

    Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kia- ninejad, Md. Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically.arXiv preprint arXiv:1712.00409, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  28. [28]

    arXiv preprint arXiv:2004.10802 , year =

    Utkarsh Sharma and Jared Kaplan. A neural scaling law from the dimension of the data mani- fold.arXiv preprint arXiv:2004.10802, 2020

    work page arXiv 2004

  29. [29]

    Explaining neural scaling laws , volume=

    Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma. Explaining neural scaling laws.Proceedings of the National Academy of Sciences, 121(27), 2024. doi: 10.1073/pnas.2311878121

    work page doi:10.1073/pnas.2311878121 2024

  30. [30]

    Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, and Colin Raffel

    Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. Scaling data-constrained language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    work page 2023

  31. [31]

    Neural Scaling Laws for Jet Generation

    Oz Amram, Darius A. Faroughy, Tjarko Gerdes, Anna Hallin, Gregor Kasieczka, Michael Kr¨amer, Humberto Reyes-Gonzalez, and David Shih. Neural Scaling Laws for Jet Generation. arXiv preprint arXiv:2605.28940, 5 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  32. [32]

    Scaling Laws for Transfer

    Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer.arXiv preprint arXiv:2102.01293, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  33. [33]

    Scaling laws for downstream task performance in machine translation

    Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, and Sanmi Koyejo. Scaling laws for downstream task performance in machine translation. In International Conference on Learning Representations (ICLR), 2025

    work page 2025

  34. [34]

    Reproducible scaling laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 11

    work page 2023

  35. [35]

    Tsukamoto, Circular light orbits of a general, static, andsphericalsymmetricalwormholewithZ 2 symmetry, Eur

    Andrew Chappell and Leigh H. Whitehead. Application of transfer learning to neu- trino interaction classification.Eur . Phys. J. C, 82(12):1099, 2022. doi: 10.1140/epjc/ s10052-022-11066-6

    work page doi:10.1140/epjc/ 2022

  36. [36]

    GLU Variants Improve Transformer

    Noam Shazeer. GLU variants improve transformer.arXiv preprint arXiv:2002.05202, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2002

  37. [37]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URLhttps: //arxiv.org/abs/1711.05101

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  38. [38]

    Transforming jet flavour tagging at ATLAS.Nature Commun., 17(1): 541, 2026

    ATLAS Collaboration. Transforming jet flavour tagging at ATLAS.Nature Commun., 17(1): 541, 2026. doi: 10.1038/s41467-025-65059-6

    work page doi:10.1038/s41467-025-65059-6 2026

  39. [39]

    BSMScaling: com- panion code for ”Towards Engineering Scaling Laws with Pretraining Data Composition”

    Jan-Lucas Uslu, Kevin Greif, Daniel Whiteson, and Benjamin Nachman. BSMScaling: com- panion code for ”Towards Engineering Scaling Laws with Pretraining Data Composition”. https://github.com/Jaluus/BSMScaling, 2026. 12

    work page 2026