REVIEW 3 major objections 5 minor 4 cited by
Real-time Anomaly Detection at the L1 Trigger of CMS Experiment
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read An autoencoder embedded in the Level-1 trigger FPGAs assigns an anomaly score to every 40 MHz LHC collision within the 50 ns latency budget.
desk verdict A genuine first deployment of an ML anomaly trigger in the CMS L1 test crate, with the physics-sensitivity claims still unsubstantiated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the variational autoencoder: an encoder compresses L1 trigger objects (ten jets, four electron/photon objects, four muons, and missing transverse energy) into an eight-dimensional latent space constrained toward a standard normal distribution, and a decoder tries to reconstruct the input from that compressed representation. Only the encoder runs in real time, and the anomaly score is approximated by the sum of squared latent means, $\sum_{i=1}^{8}\mu_i^2$, which avoids running the decoder during inference. To fit inside the latency budget, the network is trained with quantization-aware techniques, translated to high-level synthesis firmware, and bit-exactly matched to the software emulation, so the FPGA score agrees with the trained model.
What would settle it
Feed simulated or embedded signal events, such as an exotic Higgs boson decaying to four b quarks, through the same L1 reconstruction and the AXOL1TL firmware emulation, and measure the score distribution against ZeroBias background; if the signal does not populate the high-score tails at any of the five thresholds, the claim that reconstruction error discriminates new physics fails. A control check would compare high-scoring and low-scoring events at the same object multiplicity, since the score difference disappearing once multiplicity is matched would indicate the score mostly measures event complexity rather than new physics.
Extended reading notes
Core claim
On its own terms, the paper establishes that a variational autoencoder, trained on triggerless ZeroBias data from 2023 and compressed to a latent space of size eight, can be implemented in the Global Trigger test crate FPGAs and produce an anomaly score for every 40 MHz collision within the 50 ns latency window. During June 2024 data-taking, five AXOL1TL thresholds ran stably, and the events selected were largely orthogonal to those selected by the standard L1 trigger menu. High-multiplicity events tend to receive higher anomaly scores, and the triggered scouting data show invariant mass distributions of jets, muons, and photons without visible trigger-induced sculpting. The paper's deliverable is therefore a running, model-independent trigger stream of anomalous events rather than a measurement of any particular new-physics signature.
Load-bearing premise
Everything depends on the assumption that events containing new physics will be harder for the autoencoder to reconstruct than ordinary background events, so a high reconstruction-error score reliably marks events worth keeping; if new physics looks just like the background in the features the network sees, the trigger will run perfectly but add nothing.
Editorial extensions
If this is right
- Events selected by AXOL1TL are stored for offline use, giving analyses a data stream that was not preselected by any particular BSM model.
- The nominal-threshold stream feeds HLT scouting, producing a compact, continuously recorded sample of anomalous events, while the very-tight-threshold stream produces fully reconstructed events for discovery-oriented searches.
- The orthogonality of AXOL1TL events to the standard L1 menu adds new event classes, especially high-multiplicity final states, to the CMS collected dataset.
- At a 1 kHz rate, the paper estimates AXOL1TL would gain about 46% in efficiency over the rest of the L1 trigger for an exotic Higgs decay to four b quarks, indicating that the anomaly trigger finds events rule-based triggers miss.
- Mass distributions in the AXOL1TL scouting data show no obvious selection sculpting, so the collected events may be usable for resonance searches without large trigger-correction uncertainties.
Reading between the lines
- The paper does not report a signal-efficiency measurement for any specific BSM model; injecting simulated signals into the L1 object stream and measuring efficiency at each of the five thresholds would be a direct validation, with the 46% Higgs-to-four-b estimate serving as a concrete benchmark.
- Because high multiplicity drives the anomaly score, part of the score likely tracks event complexity or pileup; comparing scores at fixed multiplicity or pileup would separate genuine new-physics selection from busy-event selection.
- The same deployment pipeline could be applied to other architectures or input feature sets, and a trigger combining AXOL1TL's object-level score with calorimeter-image scores would cover complementary anomaly classes while being testable in the same test crate.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper reports the preparation, deployment, and live testing of two autoencoder-based anomaly detection algorithms, AXOL1TL and CICADA, at the CMS Level-1 trigger. AXOL1TL is a variational autoencoder that takes as input L1 objects (jets, electrons/photons, muons, and MET) in hardware integer precision, computes a latent vector of size 8, and uses the sum of squared latent means as the anomaly score. The network is translated to FPGA firmware via hls4ml and installed in the CMS Global Trigger test crate, which receives the same inputs as the main trigger without affecting data taking. The paper describes the methodology, the five thresholds tested, and presents monitoring data from June 2024 (Fig. 3) showing stable trigger rates, score distributions as a function of L1 object multiplicity (Fig. 4), and invariant mass distributions from scouting data (Fig. 5). The authors claim the AXOL1TL trigger operated stably and selects events orthogonal to the standard L1 menu, with a quoted 46% efficiency gain for H→4b relative to a rule-based trigger at 1 kHz.
Significance. If the deployment claims are correct, this is a noteworthy technical milestone: it demonstrates that unsupervised anomaly detection can be executed in the FPGA-based L1 trigger within the 50 ns latency budget and operated on live proton collision data. The paper also provides evidence of bit-exactness between the HLS emulation and the qkeras model (Fig. 2), and it references a public CMS dataset record [3], which supports reproducibility. However, the broader scientific significance depends on the anomaly score genuinely selecting events that are both new and useful for BSM searches. That property is asserted but not quantitatively demonstrated in this manuscript: no signal efficiency or background rejection for any BSM model is shown, and the only indirect evidence (Fig. 4) shows a preference for high-multiplicity events, which may be a trivial feature of the training distribution. Thus the paper is valuable as an engineering and operational report, but its physics claims are currently unsupported.
major comments (3)
- [Section 2, paragraph 2] The sentence 'In the case of AXOL1TL, it is to have 46% efficiency gain when compared to rest of L1 trigger, when operating at a rate of 1kHz for capturing exotic decay of higgs to four b quarks' states a precise, quantitative physics result without any supporting study, reference, or definition of the efficiency measurement. There is no description of the signal sample, the baseline trigger, or the statistical procedure. Because this number is the only quantitative claim about physics performance in the paper, it is load-bearing for the motivation, but the reader cannot verify or reproduce it. The authors should either provide a reference to a public CMS note or analysis, or remove the claim and replace it with a qualitative statement.
- [Section 3, Fig. 4 and surrounding text] The claim that 'The dataset triggered by AXOL1TL tends to be orthogonal to events triggered by the regular L1 Trigger menu, as seen in Fig. 4' is not supported by the evidence shown. The right panel of Fig. 4 plots AXOL1TL score as a function of L1 object multiplicity, which demonstrates only a multiplicity preference; it does not quantify the overlap or complementarity between AXOL1TL-triggered events and standard L1 menu events. Orthogonality is a stronger statement that requires, for example, the fraction of AXOL1TL events that pass or fail standard seeds, or a comparison of trigger efficiencies on a common event sample. As written, the 'orthogonality' claim is unsubstantiated, and the subsequent statement that this 'highlights the novelty of the events' overinterprets the figure.
- [Section 2, paragraph 2 and Section 2.2] The manuscript relies on the assumption that the autoencoder reconstruction error (or its latent-proxy, the sum of squared latent means) is a reliable discriminant for BSM or rare SM physics, but no validation of this assumption is presented. The paper shows no signal efficiency for any BSM model, no background rejection curve, and no closure test demonstrating that anomalous events actually produce high anomaly scores in the chosen L1 input features. In particular, the observed preference for high-multiplicity events (Fig. 4) raises the possibility that the score is a proxy for object multiplicity rather than a physically meaningful anomaly measure. Since the scientific value of the deployed trigger depends on the score's discriminatory power, this is a load-bearing gap. The authors should at least include a simulation-based benchmark (e.g., H→4b or another CMS-endorsed signature) or explicitly state that such validation is deferred to a separate publication.
minor comments (5)
- [Section 2, paragraph 2] The phrase 'it is to have 46% efficiency gain' should be corrected to 'it is estimated to have a 46% efficiency gain', and 'exotic decay of higgs to four b quarks' should be 'exotic decay of the Higgs boson to four b quarks'.
- [Section 3, Fig. 5 caption] The text states that invariant mass distributions of pairs of 'jets, electrons, and photons' are studied, but the figure caption lists 'jets (left), muons (center), and photons (right)'. This inconsistency should be resolved; if muons were studied, the text should say so, and electrons should be mentioned only if included.
- [Section 3, Fig. 3 caption] The rate monitoring plot would benefit from axis labels and units. The caption says 'Global trigger rate monitoring time series' but does not specify the y-axis unit (presumably Hz or kHz), which is important for interpreting stability.
- [Section 2.2] The choice to use the sum of squared latent means, rather than a full reconstruction-based anomaly score, is stated but not motivated. A brief justification of why this proxy is adequate (e.g., empirical equivalence or latency constraints) would improve the manuscript.
- [General] There are several typographical issues, including 'multijet' (should be 'multijet') and minor grammatical errors such as 'The Neural Network makes a prediction for each event within these constraints' where 'each event' is not strictly accurate for all events; these should be corrected in a final proofreading.
Circularity Check
No circularity: the anomaly score is defined from a VAE trained on ZeroBias data and is not fitted to the target signals; the paper's remaining weaknesses are evidentiary gaps, not circular reasoning.
full rationale
The paper makes no load-bearing derivation that reduces to its own inputs. The AXOL1TL anomaly score is the sum of squared latent means of a variational autoencoder, with the encoder trained on a 2023 ZeroBias stream at pile-up 62; the BSM/rare-SM discrimination assumption is a stated premise rather than a result fitted to new-physics targets. The quoted 46% efficiency gain for exotic Higgs decays is not derived in this text and lacks a supporting reference, but an unsupported or unvalidated number is an evidentiary issue, not a circular construction. The self-citations in the reference list, notably the CMS data-collection record and the authors' earlier knowledge-distillation work, are tooling or data-release citations and are not used as a uniqueness argument or as fitted inputs that force the central claim. The stable trigger operation and the orthogonality observation are empirical deployment results, and the preference for high-multiplicity events is reported as an observed property rather than presented as independent proof of new-physics sensitivity. Therefore the derivation chain is self-contained with respect to the stated inputs, and no circular step can be exhibited from the paper's equations or citations.
Assumptions & free parameters
free parameters (2)
- AXOL1TL anomaly score trigger thresholds (very tight, tight, nominal, loose, very loose) =
not specified numerically
- Latent vector size (8) for the VAE encoder =
8
assumptions (4)
- domain assumption The reconstruction error of an autoencoder trained on ZeroBias data is a reliable anomaly score that flags BSM and rare SM events with higher values than the dominant background.
- domain assumption The 2023 ZeroBias dataset at pile-up 62 is representative of the L1 trigger input distribution during the 2024 data-taking period in which the trigger ran.
- domain assumption The FPGA implementations are bit-exact with the trained networks, so the firmware anomaly score matches the software emulation.
- domain assumption The Global Trigger test crate is a faithful copy of the production GT, receiving identical inputs, so the behavior observed on the test crate is representative of the production system.
Cite this review
Pith. "Pith review of Real-time Anomaly Detection at the L1 Trigger of CMS Experiment." pith.science (2026). https://pith.science/paper/MTUPBPDU
@misc{pith2026241119506,
author = {Pith},
title = {Pith review of: Real-time Anomaly Detection at the L1 Trigger of CMS Experiment},
year = {2026},
howpublished = {\url{https://pith.science/paper/MTUPBPDU}},
note = {Machine review of arXiv:2411.19506}
}
read the original abstract
We present the preparation, deployment, and testing of an autoencoder trained for unbiased detection of new physics signatures in the CMS experiment Global Trigger (GT) test crate FPGAs during LHC Run 3. The GT makes the final decision whether to readout or discard the data from each LHC collision, which occur at a rate of 40 MHz, within a 50 ns latency. The Neural Network makes a prediction for each event within these constraints, which can be used to select anomalous events for further analysis. The GT test crate is a copy of the main GT system, receiving the same input data, but whose output is not used to trigger the readout of CMS, providing a platform for thorough testing of new trigger algorithms on live data, but without interrupting data taking. We describe the methodology to achieve ultra low latency anomaly detection, and present the integration of the DNN into the GT test crate, as well as the monitoring, testing, and validation of the algorithm during proton collisions.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 4 Pith papers
-
RTL Fault Injection of a Deployed Graph Neural Network Trigger for Belle II
Inter-stage liveness monitoring of GNN-ETM reveals MTTF estimates up to 78.7% lower than output-only observation and ranks Preprocessing and GNN stages as top hardening targets.
-
SparsePixels: Efficient Convolution for Sparse Data on FPGAs
A fixed-budget sparse-convolution FPGA framework runs CNNs on <=20 of ~4000 pixels, achieving 0.665 us inference for MicroBooNE with a 73x speedup and ~2% AUC loss.
-
Wasserstein normalized autoencoder for anomaly detection
A Wasserstein-distance-trained normalized autoencoder detects semivisible jets in simulated LHC events with AUCs around 0.69–0.77, outperforming standard and normalized autoencoders on a ttbar background.
-
Review of Machine Learning for Real-Time Analysis at the Large Hadron Collider experiments ALICE, ATLAS, CMS and LHCb
A SMARTHEP-network review of deployed and developing machine-learning methods for real-time triggering at ALICE, ATLAS, CMS and LHCb, with examples of industrial crossover.
Reference graph
Works this paper leans on
-
[3]
2024 Data Collected with AXOL1TL Anomaly Detection at the CMS Level-1 Trigger
CMS collaboration. 2024 Data Collected with AXOL1TL Anomaly Detection at the CMS Level-1 Trigger. 2024. URLhttps://cds.cern.ch/record/2904695
arXiv 2024
-
[1]
T. Aarrestad et al. Fast convolutional neural networks on FPGAs with hls4ml.Mach. Learn. Sci. Tech., 2(4):045015, 2021. doi: 10.1088/2632-2153/ac0ea1
-
[2]
S. Chatrchyan et al. The CMS Experiment at the CERN LHC.JINST, 3:S08004, 2008. doi: 10.1088/1748-0221/3/08/S08004
-
[4]
C. N. Coelho et al. Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors.Nature Machine Intelligence, 3(8): 675–686, June 2021. ISSN 2522-5839. doi: 10.1038/s42256-021-00356-5. URL http: //dx.doi.org/10.1038/s42256-021-00356-5
-
[5]
J. Duarte et al. Fast inference of deep neural networks in FPGAs for particle physics.JINST, 13(07):P07027, 2018. doi: 10.1088/1748-0221/13/07/P07027
-
[6]
fastmachinelearning/hls4ml, 2023
FastML Team. fastmachinelearning/hls4ml, 2023. URL https://github.com/ fastmachinelearning/hls4ml
work page 2023
-
[7]
E. Govorkova et al. Autoencoders on field-programmable gate arrays for real-time, unsu- pervised new physics detection at 40 mhz at the large hadron collider.Nature Machine Intelligence, 4(2):154–161, 2022
work page 2022
-
[8]
A. Hayrapetyan et al. Enriching the Physics Program of the CMS Experiment via Data Scouting and Data Parking. 3 2024
work page 2024
Show all 14 references
-
[9]
Hinton, O
G. Hinton, O. Vinyals, and J. Dean. Distilling the Knowledge in a Neural Network. 3 2015
2015
-
[10]
Khachatryan et al
V. Khachatryan et al. The CMS trigger system.JINST, 12:P01020, 2017. doi: 10.1088/ 1748-0221/12/01/P01020
2017
-
[11]
D. P. Kingma and M. Welling. Auto-encoding variational bayes, 2022. URLhttps:// arxiv.org/abs/1312.6114
2022 arXiv
-
[12]
LeCun et al
Y. LeCun et al. Backpropagation applied to handwritten zip code recognition.Neural Com- putation, 1(4):541–551, 1989. doi: 10.1162/neco.1989.1.4.541
1989 doi
-
[13]
EfficientandRobustJet Tagging at the LHC with Knowledge Distillation
R.Liu,A.Gandrakota,J.Ngadiuba,M.Spiropulu,andJ.-R.Vlimant. EfficientandRobustJet Tagging at the LHC with Knowledge Distillation. In37th Conference on Neural Information Processing Systems, 11 2023
2023
-
[14]
A. M. Sirunyan et al. Performance of the CMS Level-1 trigger in proton-proton collisions at√𝑠 = 13TeV. JINST, 15:P10017, 2020. doi: 10.1088/1748-0221/15/10/P10017. 6
2020 doi
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.