REVIEW 12 cited by
Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose masked particle modeling (MPM) as a self-supervised method for learning generic, transferable, and reusable representations on unordered sets of inputs for use in high energy physics (HEP) scientific data. This work provides a novel scheme to perform masked modeling based pre-training to learn permutation invariant functions on sets. More generally, this work provides a step towards building large foundation models for HEP that can be generically pre-trained with self-supervised learning and later fine-tuned for a variety of down-stream tasks. In MPM, particles in a set are masked and the training objective is to recover their identity, as defined by a discretized token representation of a pre-trained vector quantized variational autoencoder. We study the efficacy of the method in samples of high energy jets at collider physics experiments, including studies on the impact of discretization, permutation invariance, and ordering. We also study the fine-tuning capability of the model, showing that it can be adapted to tasks such as supervised and weakly supervised jet classification, and that the model can transfer efficiently with small fine-tuning data sets to new classes and new data domains.
Forward citations
Cited by 12 Pith papers
-
Predict before you train: Scaling Laws for particle physics foundation models
A Chinchilla-style law fit on ParticleViT runs below 10^19 FLOPs predicts held-out pretraining loss within ~1% at >100× compute and tracks downstream jet-tagging rejection.
-
Learning Standard Model structure from LHC data with Riemannian flow matching
ShellFlow, a Riemannian flow-matching transformer fed only on-shell and invariant-mass priors and ~8×10^8 recorded ATLAS events, reproduces the SM's dilepton resonances, Weinberg angle, and top/W mass peaks in a singl...
-
Masked-Token Prediction for Anomaly Detection at the Large Hadron Collider
The work demonstrates masked-token prediction with transformers for model-independent anomaly detection in LHC data, achieving strong results on top-rich BSM signatures like four-top production using VQ-VAE tokenization.
-
Learning transferable event representations for charmed baryon physics at BESIII
A Particle Transformer pre-trained on simulated Lambda_c events transfers across 12 decay channels, improving classification and momentum-direction regression over training from scratch in low-statistics regimes.
-
D$e^+e^-$ffusion: Capturing the Beam-Beam Physics of $e^+e^-$ Collisions with Diffusion Models
A diffusion model trained on GuineaPig++ reproduces FCC-ee beam-induced pair-production distributions at particle and detector level, about 10^4 times faster.
-
Explicit or Implicit? Encoding Physics at the Precision Frontier
On three precision classification tasks — reweighting-based unfolding, likelihood-ratio estimation, and weakly supervised anomaly detection — a Lorentz-equivariant transformer and a pretrained foundation model perform...
-
A universal vision transformer for fast calorimeter simulations
A vision-transformer flow-matching model generates calorimeter showers across regular and irregular detector geometries at millisecond speeds, and pretraining plus fine-tuning cuts training cost by about half.
-
Enhancing next token prediction based pre-training for jet foundation models
Using continuous particle features as input and combining next-token with masked-token pre-training markedly improves classification accuracy of the OmniJet jet foundation model without visibly hurting its generative quality.
-
Pretrained Event Classification Model for High Energy Physics Analysis
A GNN pretrained on 120M simulated HEP events generalizes to unseen processes and ATLAS data; fine-tuning boosts accuracy especially with small datasets, with CKA showing preserved encoders but altered intermediate layers.
-
A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation
A lightweight autoencoder pre-trained on LHC track data transfers to collider and out-of-domain scientific tasks, matching a transformer within ~2% at about 46x lower per-epoch training cost.
-
Are We Ready for AI-Driven Discovery? AI Verification Before the Next Fundamental Physics Breakthrough
Verification of ML in fundamental physics is essential precisely when models enter statistical modeling, inference, or hypothesis testing, and is bounded by unavoidable inductive bias, sample complexity, and experimen...
-
HEPTAPOD: Orchestrating High Energy Physics Workflows Towards Autonomous Agency
HEPTAPOD uses LLM agents to drive FeynRules, MadGraph, Pythia, and analysis tools through schema-validated tool calls and run-card templates, demonstrated on a leptoquark signal scan.
Discussion (0). Sign in to comment.