REVIEW 3 major objections 4 minor 28 references
A new open-source pipeline, SET-ANUBIS, aims to make ANUBIS long-lived-particle sensitivity studies fully auditable and model-agnostic.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 01:17 UTC pith:MQHWHKVK
load-bearing objection Solid software paper: the reproducibility architecture is real and the engineering is clean, but the physics validation is internal by design, so the acceptance numbers need an external benchmark before they should be trusted for published limits. the 3 major comments →
SET-ANUBIS: a modular pipeline for ANUBIS long-lived particle sensitivity studies
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that SET-ANUBIS v1.0.0 establishes a deterministic, auditable software baseline for ANUBIS sensitivity studies: starting from a UFO model or user-supplied rates, it evaluates decay widths, branching ratios and lifetimes; prepares and ingests events from Pythia8 or MadGraph; reconstructs the ATLAS cavern and ANUBIS RPC geometry; and applies a named cutflow (LLP decay, in-cavern, not-in-ATLAS, geometry, tracker, missing transverse momentum, isolation) to determine which events would be observed. The framework validates its geometry predicates by Monte Carlo volume reconstruction and track-intersection tests, and its R5 reproducibility scenario exercises the full se
What carries the argument
The load-bearing object is the hexagonal (ports-and-adapters) architecture that partitions the workflow into bounded domains—model, branching-ratio, MadGraph/Pythia interfaces, geometry, selection, database—with narrow APIs and replaceable adapters. The selection engine applies configurable geometry predicates ('is_in' functions for the ATLAS cavern, shafts, and ANUBIS tracking layers) and kinematic/isolation cuts, while the content-addressed store (SHA-256) and SQLite catalogue preserve provenance so a campaign can be re-materialised from stored cards, seeds, and metadata. This combination is what lets the framework claim both model-agnosticism and reproducibility.
Load-bearing premise
The load-bearing premise is that the framework's internal consistency—geometry volume reproduction and deterministic reference outputs on a tiny hand-made sample—is sufficient to trust the physics reliability of the resulting sensitivity limits, since the cutflow thresholds and background model are not validated against real collision data or a realistic full Monte Carlo background sample.
What would settle it
Generate a realistic background Monte Carlo sample (e.g., Pythia8 QCD dijet events with full showering) and pass the reconstructed events through the SET-ANUBIS selection chain, counting how many pass the E_T^miss > 30 GeV, geometric acceptance, and isolation requirements. If the resulting background rate is non-negligible (comparable to or above the N_LLP ≥ 4 threshold assumed for a background-free search) or deviates significantly from the expected near-zero rate, the framework's selection model is wrong.
If this is right
- If the framework's correctness holds, future ANUBIS sensitivity limits can be produced by independent groups and checked against the R1–R5 reference outputs.
- Different BSM models can be compared using the same geometry, cutflow and acceptance definitions, removing a common source of inconsistency between studies.
- The lifetime re-weighting and compact selection-ready bundles make large parameter scans substantially cheaper in both CPU time and storage.
- The R1–R5 reproducibility contract offers a template for how other detector-sensitivity software could package deterministic validation evidence.
- Because the selection chain is configurable, the same tooling can be repointed to non-ANUBIS far-detector geometries, widening the framework's use.
Where Pith is reading between the lines
- The paper's validation is mostly about software consistency—e.g., seven hand-crafted events designed to fail at successive stages—not about whether the chosen cutflow (E_T^miss > 30 GeV, isolation cone 0.5, 50% signal efficiency) correctly models real ANUBIS backgrounds; that physics validation remains open work.
- If the ANUBIS collaboration adopts SET-ANUBIS as its default sensitivity tool, as the paper indicates, the framework's approximations for hadronic decays and simplified geometry will directly shape all published limits, raising the stakes for a dedicated background-validation campaign.
- A concrete testable extension would be to run the full chain on realistic background samples (e.g., QCD dijets producing K_L/n punch-through) and compare the efficiency and background rejection against the current parameter choices.
- The architecture is generic enough that the same pipeline could be adapted to other proposed displaced-decay detectors, provided their geometry is encoded in the same predicate language.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes SET-ANUBIS, an open-source Python framework for end-to-end long-lived-particle sensitivity studies for the proposed ANUBIS detector. It connects UFO model handling, decay-width and branching-ratio calculation, Pythia/MadGraph event-generation preparation, HepMC ingestion, ATLAS-cavern and ANUBIS-RPC geometry, a named selection cutflow, and SQLite/content-addressed provenance storage. Release v1.0.0 introduces five reproducibility scenarios R1–R5, machine-readable JSON and HTML selection traces, optional Dash inspection tools, and 247 automated tests with 67.10% line coverage. The paper's central claim is that the framework provides an auditable, reproducible path from BSM model parameters to ANUBIS acceptance and sensitivity projections with minimal bespoke user code.
Significance. If the geometry and selection implementation are physically correct, SET-ANUBIS would be a valuable community tool: it is open source, has a clear ports-and-adapters architecture, ships deterministic card generation, versioned reproducibility scenarios, content-addressed storage, and machine-readable selection traces. The reproducibility contract R1–R5 is a genuine strength and makes the software's internal consistency machine-checked. However, the paper currently validates only internal consistency: the geometry test compares predicates with volumes derived from the same geometry definition, and R5 is explicitly a seven-event regression fixture rather than a physical signal sample. No external benchmark against a published ANUBIS acceptance, a full realistic Monte Carlo sample, or an independent selection implementation is provided. The central physics output, the acceptance A in Eq. (1), is therefore not yet anchored to an external authority. This is a load-bearing gap for a paper whose stated purpose is to support auditable sensitivity studies.
major comments (3)
- [§5.4] The Monte-Carlo volume-reconstruction test compares the geometry predicates against 'reference volumes computed directly from the geometry definition.' This is an internal-consistency check of coordinate transforms and region classification, not a validation of the geometry parameters or of the resulting acceptance. A systematic error in the cavern dimensions, ceiling curvature, IP offset, or RPC placement would pass both sides of the test. To support the claim that the geometry acceptance is 'correctly implemented and validated,' please add an external anchor: reproduce the ANUBIS acceptance or sensitivity contour of Ref. [2] for a benchmark model, or compare the implemented geometry against an independent CAD/analytic geometry model.
- [§6.2 and §4.5–4.6] Scenario R5 uses a seven-event HepMC2 sample that the manuscript itself states 'is not intended to represent a physical signal distribution'; the expected cutflow counts are versioned references produced by the same pipeline. This validates cutflow bookkeeping, determinism, and trace generation, but it does not establish that the implemented selection (E_T^miss > 30 GeV, ΔR = 0.5 isolation, d_IP < 10 mm, tracker-intersection logic) yields physically correct acceptances. No comparison is made against a full realistic Monte Carlo sample, an independent implementation of the ANUBIS selection, or the published sensitivity study [22] that used the framework. Because A in Eq. (1) is the central physics output, please add a closure test: generate a realistic LLP sample (or use a public sample) and compare the resulting acceptance/cutflow/N_LLP against Ref. [2] or an independent calculation. Not
- [Abstract, §3.1, §8.1] The paper describes the framework as supporting 'an arbitrary BSM model' with 'minimal additional bespoke code,' but Section 8.1 states that hadronic decay channels are not implemented and rely on user-supplied calculators or tabulated rates, and that MARTY coverage of hadronic matrix elements is not guaranteed. For models whose dominant decays are hadronic, the automated width/BR and lifetime chain is therefore incomplete. Since the framework's stated goal is direct comparison of alternative LLP models, please qualify the scope in the abstract and introduction and state explicitly which model classes are supported without additional user input.
minor comments (4)
- [§3.1, §4.6, §6.2] The manuscript cites HepMC3 [15] and lists HepMC3 as an external dependency, but R5 and Section 6.2 use a 'HepMC2' sample. Please clarify which HepMC versions are supported for ingestion and which are covered by the validation scenarios.
- [§7.2, Eq. (14)] The indicator I_HepMC in Eq. (14) is not defined in the text. Please define it explicitly, e.g. as 1 when raw HepMC retention is requested and 0 otherwise.
- [§5.2] In the selection example, the code prints cutflow["nLLP_Final"], while the text's stage list uses 'Final' and earlier mentions 'nLLP_original'. Please ensure the key names in the printed example match the documented JSON output exactly.
- [References [2], [22]] The paper relies on Ref. [2] for the background model and selection thresholds and on Ref. [22] as an application of the framework, but both are preprint citations dated 2025/2026. Please confirm that they are publicly available and include the exact arXiv identifiers in the bibliography.
Circularity Check
No significant circularity: the paper is a software/reproducibility report, and its validation checks are internal-consistency tests rather than physics predictions derived from their own inputs.
full rationale
SET-ANUBIS is a software framework paper, not a physics-derivation paper. Its central deliverables are the pipeline architecture, the R1–R5 reproducibility scenarios, and the versioned reference outputs. The R5 cutflow fixture is explicitly described as 'not intended to represent a physical signal distribution' and as 'a transparent regression fixture for the interpretation and ordering of the production cuts'; the expected cutflow counts are constructed by hand, not fitted from data and then renamed as a prediction. The geometry validation in Sec. 5.4 compares Monte Carlo volume estimates with 'reference volumes computed directly from the geometry definition'; this is a check that the predicates are implemented consistently with the parameterized geometry, i.e. a software-consistency test, not an independent physics prediction that reduces to its own input. The self-citation of the ANUBIS HNL sensitivity study [22] is presented as a use of the framework, not as evidence for the correctness of its physics outputs. The selection thresholds (E_T^miss > 30 GeV, isolation cone, 50% efficiency) are taken from the separate ANUBIS sensitivity study [2] and are not derived in this paper. The paper also flags its own limitation in Sec. 8.1 that hadronic decays and form factors remain a limiting factor. None of the load-bearing claims—deterministic card generation, reproducible cutflow execution, geometry-predicate consistency, provenance tracking—are obtained by defining a variable in terms of the result, by fitting a parameter and calling it a prediction, or by invoking a self-citation as the sole justification. A weaker external physics benchmark would strengthen the physics validation, but that is a correctness/validation gap, not circularity under the specified criteria.
Axiom & Free-Parameter Ledger
free parameters (3)
- Selection thresholds (E_T^miss > 30 GeV, Delta R = 0.5, pT > 10 GeV, d_IP < 10 mm, 50% signal efficiency, N_LLP >= 4 or =
E_T^miss > 30 GeV; DeltaR = 0.5; pT > 10 GeV; d_IP < 10 mm; epsilon_sig = 50%
- Geometry configuration (2 tracking layers, 1 m separation, first layer 20 cm below ceiling, RPC efficiency 100%) =
2 layers, 1 m separation, 20 cm clearance, 100% efficiency
- Luminosity L = 3 ab^-1 =
3 ab^-1
axioms (5)
- domain assumption HepMC event records faithfully represent generator output and the dataframe conversion preserves the information needed for the selection.
- domain assumption The simplified cavern/RPC geometry (cylindrical shells, vaulted ceiling, shafts) is an adequate model of the real ATLAS cavern and ANUBIS stations for acceptance purposes.
- domain assumption The selection thresholds and the signal-efficiency factor epsilon_sig = 50% correctly represent ANUBIS background rejection and detector efficiency.
- domain assumption External generators (Pythia8, MadGraph 3.5.8, MARTY) and UFO models correctly compute the physics input (cross sections, branching ratios, lifetimes).
- standard math The QCDRunner's three-loop alpha_s and mass running formulae are correct implementations of the standard MS-bar beta functions.
invented entities (1)
-
No new physical entities are invented
no independent evidence
read the original abstract
SET-ANUBIS (Simulation, accEptance and sensiTivity studies framework for ANUBIS) is an open-source Python framework for end-to-end studies of long-lived particle signatures in the proposed ANUBIS detector. Starting from a Universal FeynRules Output model or user-supplied rates, it exposes model parameters and particle content, evaluates decay widths, branching ratios and lifetimes, prepares or runs event generation through Pythia8 or MadGraph, ingests HepMC event records, models the ATLAS cavern and ANUBIS tracking stations, and applies a configurable sequence of geometric, kinematic and isolation requirements. The implementation follows a ports-and-adapters architecture so that physics-domain logic remains separate from external generators, persistent storage and visualisation. A scan-aware SQLite catalogue and content-addressed store preserve cards, metadata and compact selection-ready event bundles while avoiding duplicate artefacts. Release v1.0.0 introduces a deterministic reproducibility suite with five independent scenarios (R1--R5) covering model access, branching-ratio interpolation, Pythia command generation, MadGraph card generation and the complete selection chain on a compact HepMC2 sample. The final scenario produces machine-readable JSON and a standalone HTML selection trace that records cumulative cutflows and event-level pass/fail transitions. Optional scientific dashboards provide event, geometry, campaign and provenance inspection without changing the physics results. The framework, documentation and reference inputs are distributed under the GNU GPL version 3 or later and are intended to support auditable ANUBIS sensitivity studies and direct comparison of alternative long-lived particle models.
Figures
Reference graph
Works this paper leans on
-
[1]
Bauer, O
M. Bauer, O. Brandt, L. Lee, C. Ohm, ANUBIS: Proposal to search for long-lived neutral particles in CERN service shafts (9 2019).arXiv:1909.1 3022
2019
-
[2]
O. Brandt, et al., The ANUBIS detector and its sensitivity to neutral long-lived particles (10 2025).arXiv:2510.26932
arXiv 2025
-
[3]
T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Ras- mussen, P. Z. Skands, An introduction to PYTHIA 8.2, Comput. Phys. Commun. 191 (2015) 159– 177.arXiv:1410.3012,doi:10.1016/j.cp c.2015.01.024
Pith/arXiv arXiv 2015
-
[4]
J. Alwall, R. Frederix, S. Frixione, V . Hirschi, F. Maltoni, O. Mattelaer, H. S. Shao, T. Stelzer, P. Torrielli, M. Zaro, The automated computation of tree-level and next-to-leading order differen- tial cross sections, and their matching to parton shower simulations, JHEP 07 (2014) 079.arXiv: 1405.0301,doi:10.1007/JHEP07(2014)079
Pith/arXiv arXiv 2014
-
[5]
G. Uhlrich, F. Mahmoudi, A. Arbey, MARTY , a new C++framework for automated symbolic cal- culations in Beyond the Standard Model physics, PoS ICHEP2020 (2021) 928.arXiv:2011.065 58,doi:10.22323/1.390.0928
-
[6]
J. Alimena, et al., Searching for long-lived par- ticles beyond the Standard Model at the Large Hadron Collider, J. Phys. G 47 (9) (2020) 090501. arXiv:1903.04497,doi:10.1088/1361-647 1/ab4574
arXiv 2020
-
[7]
M. Aaboud, et al., Search for long-lived particles produced inppcollisions at √s=13 TeV that decay into displaced hadronic jets in the ATLAS muon spectrometer, Phys. Rev. D 99 (5) (2019) 052005.arXiv:1811.07370,doi:10.1103/Ph ysRevD.99.052005
Pith/arXiv arXiv 2019
-
[8]
V . Khachatryan, et al., Search for long-lived par- ticles that decay into final states containing two electrons or two muons in proton-proton colli- sions at √s=8 TeV, Phys. Rev. D 91 (5) (2015) 052012.arXiv:1411.6977,doi:10.1103/Ph ysRevD.91.052012
Pith/arXiv arXiv 2015
-
[9]
A. Tumasyan, et al., Search for long-lived par- ticles decaying into muon pairs in proton-proton collisions at √s=13 TeV collected with a dedi- cated high-rate data stream, JHEP 04 (2022) 062. arXiv:2112.13769,doi:10.1007/JHEP04(20 22)062
Pith/arXiv arXiv 2022
-
[10]
V . V . Gligorov, S. Knapen, M. Papucci, D. J. Robinson, Searching for Long-lived Particles: A Compact Detector for Exotics at LHCb, Phys. Rev. D 97 (1) (2018) 015023.arXiv:1708.09395, doi:10.1103/PhysRevD.97.015023
Pith/arXiv arXiv 2018
-
[11]
Curtin, et al., Long-Lived Particles at the En- ergy Frontier: The MATHUSLA Physics Case, Rept
D. Curtin, et al., Long-Lived Particles at the En- ergy Frontier: The MATHUSLA Physics Case, Rept. Prog. Phys. 82 (11) (2019) 116201.arXiv: 1806.07396,doi:10.1088/1361-6633/ab28 d6
Pith/arXiv arXiv 2019
-
[12]
Ahdida, et al., The SHiP experiment at the pro- posed CERN SPS Beam Dump Facility, Eur
C. Ahdida, et al., The SHiP experiment at the pro- posed CERN SPS Beam Dump Facility, Eur. Phys. J. C 82 (5) (2022) 486.arXiv:2112.01487, doi:10.1140/epjc/s10052-022-10346-5
Pith/arXiv arXiv 2022
-
[13]
Darmé, et al., UFO 2.0: the ‘Universal Feyn- man Output’ format, Eur
L. Darmé, et al., UFO 2.0: the ‘Universal Feyn- man Output’ format, Eur. Phys. J. C 83 (7) (2023) 631.arXiv:2304.09883,doi:10.1140/epjc /s10052-023-11780-9
Pith/arXiv arXiv 2023
-
[14]
C. Degrande, C. Duhr, B. Fuks, D. Grellscheid, O. Mattelaer, T. Reiter, UFO - The Universal FeynRules Output, Comput. Phys. Commun. 183 (2012) 1201–1214.arXiv:1108.2040,doi: 10.1016/j.cpc.2012.01.022
Pith/arXiv arXiv 2012
-
[15]
A. Buckley, P. Ilten, D. Konstantinov, L. Lönnblad, J. Monk, W. Pokorski, T. Przedzin- ski, A. Verbytskyi, The HepMC3 event record library for Monte Carlo event gen- erators, Comput. Phys. Commun. 260 (2021) 107310.arXiv:1912.08005, doi:10.1016/j.cpc.2020.107310
Pith/arXiv arXiv 2021
-
[16]
Bierlich, et al., A comprehensive guide to the physics and usage of PYTHIA 8.3, SciPost Phys
C. Bierlich, et al., A comprehensive guide to the physics and usage of PYTHIA 8.3, SciPost Phys. Codeb. 2022 (2022) 8.arXiv:2203.11601,doi: 10.21468/SciPostPhysCodeb.8
Pith/arXiv arXiv 2022
-
[17]
A. Alloul, N. D. Christensen, C. Degrande, C. Duhr, B. Fuks, FeynRules 2.0 - A complete toolbox for tree-level phenomenology, Comput. Phys. Commun. 185 (2014) 2250–2300.arXiv: 1310.1921,doi:10.1016/j.cpc.2014.04.01 2
Pith/arXiv arXiv 2014
-
[18]
HTCondor Team, HTCondor, Zenodo (2024).do i:10.5281/zenodo.11397217. 18
-
[19]
M. Cacciari, G. P. Salam, G. Soyez, The anti-kt jet clustering algorithm, JHEP 04 (2008) 063.arXi v:0802.1189,doi:10.1088/1126-6708/2008 /04/063
Pith/arXiv arXiv 2008
-
[20]
Navas, et al., Review of particle physics, Phys
S. Navas, et al., Review of particle physics, Phys. Rev. D 110 (3) (2024) 030001.doi:10.1103/Ph ysRevD.110.030001
doi:10.1103/ph 2024
-
[21]
Meurer, et al., Sympy: symbolic computing in python, PeerJ Computer Science 3 (2017) e103
A. Meurer, et al., Sympy: symbolic computing in python, PeerJ Computer Science 3 (2017) e103
2017
-
[22]
ANUBIS Collaboration, Projected sensitivity of the ANUBIS detector to heavy neutral leptons (6 2026).arXiv:2606.26862
Pith/arXiv arXiv 2026
-
[23]
UKRI under the Future Leaders Fellowship scheme (Grant number G107408)
T. Reymermier, P. Swallow, SET-ANUBIS: a modular pipeline for ANUBIS long-lived particle sensitivity studies, Software release v1.0.0 (2026). URLhttps://github.com/SET-ANUBIS/se t-anubis CRediT authorship contribution statement Anna Mullin:Methodology, Software (first imple- mentation of the event-selection component).ANU- BIS Collaboration:provision of c...
2026
-
[24]
FixΛ 5 from the inputα s(MZ) by inverting Equa- tion (2)
-
[25]
Determine the ordered thresholds (mu,m d,m s,m c,m (type) b ,m (type) t )
-
[26]
To evaluateα s(Q): step the flavour number from 5 ton f (Q) using Equation (B.1), then use Equa- tion (2) atn f (Q)
-
[27]
To run a quark massm(Q i)→m(Q f ): decom- pose the path into constant-n f segments and apply Equation (B.3) on each
-
[28]
Practical settings and caveats •Inputs:α s(MZ),M Z,m pole t , ¯mb, and ¯mu,d,s,c (all in GeV)
Convertm b orm t when needed using Equa- tions (B.4) and (B.5). Practical settings and caveats •Inputs:α s(MZ),M Z,m pole t , ¯mb, and ¯mu,d,s,c (all in GeV). •Threshold choice forb,t(pole or running) is con- figurable and only affects wheren f changes. •Matching usescontinuityofα s atQ=m h with- out higher–order decoupling constants; this is ade- quate f...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.