Optimizers for Stabilizing Likelihood-free Inference

Benjamin Nachman; Eva Silverstein; G. Bruno De Luca; Henry Zheng

REVIEW 4 major objections 5 minor 58 references

Optimizers for Stabilizing Likelihood-free Inference

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A physics-inspired optimizer stabilizes neural likelihood-ratio fits and beats Adam on collider benchmarks.

desk verdict The ALEPH metric is the soft spot; the jet benchmark shows a real but small effect, so the paper deserves revision, not rejection. read the letter →

arxiv 2501.18419 v2 pith:OIXBM5UA submitted 2025-01-30 hep-ph hep-th

G. Bruno De Luca , Benjamin Nachman , Eva Silverstein , Henry Zheng This is my paper · ORCID

classification hep-phhep-th

keywords energyconservingdescentlikelihood-ratioestimationneuralsimulation-basedinferenceoptimizerstabilityHamiltoniandynamicsparticlephysicsAdamcomparisonhyperparameterrobustness

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the dominant source of instability in neural likelihood-ratio estimation, the dependence of the trained network on its random initialization, can be suppressed by replacing the usual friction-based optimizers with an energy-conserving one. It develops a variant called $\mathrm{ECD}_{q=1}$, whose few hyperparameters are set by physical reasoning rather than expensive scanning, and tests it on two collider-physics reweighting tasks. In both, networks trained with $\mathrm{ECD}_{q=1}$ achieve lower mean absolute error than networks trained with Adam, with less spread across initializations and stable performance when the network width or depth is scaled. If the result holds, precision inference tasks that currently rely on large ensembles and extensive hyperparameter searches could be stabilized far more cheaply.

What carries the argument

The load-bearing object is the $\mathrm{ECD}_{q=1}$ optimizer, a discretized Hamiltonian dynamics on the space of network parameters whose update rules are $u \leftarrow u + \Delta t\, \dot u$ and $\Theta \leftarrow \Theta + \Delta t\, u$, with $\dot u = -\frac{\eta d}{2(d-1)}\, (F-F_0)\big(\nabla F - (u\cdot\nabla F)u\big)$ and $|u|=1$. Its theoretical effect is to make the distribution of outcomes after chaotic mixing proportional to $(F(\Theta)-F_0)^{-\eta d/2}$, which concentrates near the minimum of the loss $F$ as $\eta$ grows. This concentration is what reduces the spread of results across random initializations. The discrete scheme conserves a discretized energy, and the hyperparameters $\Delta t$, $F_0$, $\nu$, and $\eta$ are assigned narrow ranges from the step budget, the network width, and the requirement that $F_{\min}-F_0$ stay of order one.

What would settle it

Train the same classifier with $\mathrm{ECD}_{q=1}$ using full-batch gradients and using typical minibatch gradients while recording the discrete Hamiltonian at each step. If the minibatch run shows systematic energy drift, or if the full-batch run's insensitivity to initialization disappears once a small random bounce is removed, the conserved-energy mechanism is not what stabilizes the results, and the reported MAE gains would need to be re-explained as benchmark-specific artifacts.

Watch

Extended reading notes

Core claim

The central claim is that the $\mathrm{ECD}_{q=1}$ optimizer outperforms Adam for likelihood-ratio estimation in the regularization-dominated regime of neural network training. Concretely, on the ALEPH thrust-reweighting task the paper reports MAE $1.48(29)\times10^{-3}$ for $\mathrm{ECD}_{q=1}$, versus $2.59(27)\times10^{-3}$ for the best scanned Adam and $3.20(87)\times10^{-3}$ for default Adam, and $\mathrm{ECD}_{q=1}$ achieves the lower MAE on all six maximum-likelihood and binary cross-entropy jet parametrizations in the higher-dimensional example. The improvement comes without extra memory: ECD stores $2d$ variables per step versus Adam's $3d$, and it uses a narrow, theory-guided prior on its hyperparameters rather than a wide scan. The authors interpret the gain as the expected outcome of Hamiltonian dynamics that samples network parameters from a microcanonical measure concentrating near the loss minimum, so that initialization dependence is mixed away rather than tuned away.

Load-bearing premise

The argument depends on the discrete training dynamics actually conserving the Hamiltonian energy and mixing thoroughly across the network parameter space within the allowed number of steps; if stochastic minibatch gradients break energy conservation or the mixing is incomplete, the theory no longer predicts reduced initialization dependence.

Editorial extensions

If this is right

On the ALEPH $e^+e^-$ thrust-reweighting benchmark, default-parameter $\mathrm{ECD}_{q=1}$ attains MAE $1.48(29)\times10^{-3}$, beating both the best scanned Adam at $2.59(27)\times10^{-3}$ and default Adam at $3.20(87)\times10^{-3}$.
On the six MLC and BCE jet reweighting setups, $\mathrm{ECD}_{q=1}$ reports lower mean absolute error than Adam for every parametrization, with comparable or smaller uncertainties across 100 initializations.
Scaling the network width or depth by a factor of four leaves $\mathrm{ECD}_{q=1}$'s advantage over Adam intact.
Because $\mathrm{ECD}_{q=1}$ needs few hyperparameters with narrow physics-motivated ranges, it reduces the need for extensive hyperparameter scans and may lower the computational cost of ensembling for stability.
The optimizer stores fewer variables per step than Adam ($2d$ versus $3d$), so the stability gain is not bought with extra memory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the conservation-based mechanism is the real cause of the gain, $\mathrm{ECD}_{q=1}$ should transfer to other neural simulation-based inference tasks beyond the two benchmarks, such as anomaly detection or unfolding; the paper does not test these.
The theory assumes ergodic mixing within the training budget, so a natural check that does not appear in the paper is to measure energy drift along actual minibatch training, since stochastic gradients could break the conserved quantity.
With bounces set to zero in all experiments, the paper implicitly suggests that chaotic mixing is generic in these networks; if true, one could push $\eta$ higher or use narrower learning-rate priors in future work.
A natural extension is to combine $\mathrm{ECD}_{q=1}$ with existing ensembling strategies, where the optimizer's lower per-initialization variance should make ensembles converge faster, although the interaction is not quantified in the paper.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. The paper's headline that ECDq=1 beats Adam for likelihood-ratio estimation in particle physics is only as strong as its least solid benchmark, and that benchmark has a metric problem: the ALEPH MAE is defined only over bins with T ≤ 0.3, which is outside the range shown in the figure (the plot goes to 1−T = 0.5, i.e., T ≥ 0.5). No justification is given. That looks like a mistake or an indefensible restriction, and it undercuts the large reported gap (1.48 vs 2.59). The second benchmark, on six jet substructure parametrizations, is more solid: 100 initializations, full unbinned likelihood ratio, and a consistent, if small, improvement for ECDq=1 over best Adam. So there is a real empirical signal, just not as dramatic as the abstract suggests.

What is new: the optimization version of ECDq=1, with specific discrete update rules, physics-guided hyperparameter ranges, and a first application to likelihood-ratio stability. The code and hyperparameter tables are provided, which is reproducible and helpful. The idea of using a conserved quantity to reduce initialization dependence is worth taking seriously, even if the theory here is incomplete.

Soft spots, in proportion. The ALEPH metric issue is the load-bearing one. If the restriction is a typo, fine, but as written it makes the headline claim unsupported for that dataset. The theory side also has gaps: the paper admits it has no a priori mixing-time calculation, sets the bounce amplitude to zero, and does not analyze how minibatch stochastic gradients break energy conservation. The 'default' hyperparameters are not fully parameter-free—F0 is chosen empirically from the loss minimum (F0 = −0.3 because Fmin ≈ 0.68 for BCE). So the paper's claim of 'few free hyperparameters with physical reasoning' is overstated; some values come from looking at the training.

Who this is for: people using neural network likelihood-ratio estimators in high-energy physics or other data-limited sciences, who care about training stability. A serious referee should see it, but it needs revision. I would recommend asking the authors to recompute the ALEPH MAE over the full plotted thrust range, report full-range numbers, and either remove or justify the T ≤ 0.3 cutoff. I'd also want a discussion of how minibatch noise affects the conserved-energy assumption, or at least an honest caveat that the theoretical mechanism is unproven. The jet benchmark alone may justify a short paper, but the current version overclaims.

Referee Report

4 major / 5 minor

Summary. The paper introduces ECDq=1, an optimizer for neural-network-based likelihood ratio estimation, built from energy-conserving Hamiltonian dynamics. The authors argue that the optimizer's microcanonical measure concentrates near the loss minimum, which should reduce the dependence of trained networks on random initialization and lower the spread of results. They compare ECDq=1 with Adam on two particle-physics tasks: a one-dimensional ALEPH thrust reweighting and a six-dimensional jet-substructure reweighting with MLC and BCE losses. They report lower MAE for ECDq=1 in the ALEPH task and lower or comparable MAE across all six jet parametrizations, along with reduced sensitivity to network width and depth scaling. Code and hyperparameter tables are provided.

Significance. If the empirical claims hold, ECDq=1 would be a practically useful drop-in optimizer for NSBI tasks, with few hyperparameters and reduced need for ensembling. The paper is not circular: the central comparison against Adam is an external benchmark, and the code is publicly available, which strengthens reproducibility. The theoretical mechanism is plausible but is not the main evidence; the practical comparison is the load-bearing part. The main open issues are the restricted MAE definition in the ALEPH comparison and the fact that some default hyperparameters are partly chosen using information from the training runs.

major comments (4)

[III A] The ALEPH MAE is defined 'summed over the bins from Fig. 1 with T ≤ 0.3', but Fig. 1 plots 1−T from 0 to 0.5, which corresponds to T from 1 to 0.5. The range T ≤ 0.3 is therefore not even present in the figure, and the comparison uses only the low-density tail of the thrust distribution. Since the data-density weighting is dominated by high-thrust events, a metric restricted to T ≤ 0.3 may favor either optimizer arbitrarily. The paper never reports a full-range or plotted-range MAE, and no justification is given for the restriction. This is load-bearing for the headline claim, so the authors must report MAE over the full plotted range (and ideally over the full data range) and justify any sub-range restriction.
[II A] The theoretical rationale for reduced initialization dependence relies on the microcanonical measure p(Θ) ∝ (F(Θ)−F0)^(−ηd/2), which requires the discrete dynamics (16)-(18) to conserve energy and to mix ergodically within the training budget. The paper explicitly states, just before Eq. (11) and in Sec. II A 1, that there is no a priori calculation of the mixing time, and all experiments set ν = 0 (no bounces). The effect of minibatch gradient noise on energy conservation is not analyzed. Therefore the claim that ECDq=1 reduces initialization dependence is not established theoretically; it is an empirical observation. The authors should either provide a quantitative analysis of mixing and energy drift or explicitly reframe the contribution as an empirical optimizer study.
[II B and Table IV] The claim that ECDq=1 has 'few free hyperparameters with limited ranges guided by physical reasoning' is undermined by the way defaults are set in the experiments. In particular, F0 for the BCE loss is fixed to −0.3 because 'empirically we found Fmin ∼ 0.68' (Table IV caption), and η is derived from n_steps and the network width using an estimate that the Hessian scale F2 is O(1). These choices use information from the training problem and from the observed loss minimum, so the reported gains are not achieved with fully parameter-free defaults. The authors should state clearly which hyperparameters are fixed before seeing the data and which are adjusted after, and should test robustness to reasonable variations in F0 and η.
[III B and Table III] The conclusion that ECDq=1 'out-performs' Adam is stronger than the table supports for several parametrizations. For example, Table III gives MAE 0.401(4) for ECDq=1 vs 0.402(6) for Adam with exp(z), and 0.391(4) vs 0.398(5) for σ(z); these differences are at the 1–2 sigma level. Only the ALEPH comparison shows a large gap. The authors should either soften the wording or provide a statistical test (e.g., paired differences across initializations) to support the claim that ECDq=1 is better across all six parametrizations.

minor comments (5)

[III A / Fig. 1] Please clarify the horizontal axis: Fig. 1 shows 1−T on the x-axis, while the MAE definition in the text refers to T ≤ 0.3. This inconsistency makes it difficult to see which bins are included in the MAE.
[Table IV] The table caption contains a typo ('emperically' for 'empirically') and the notation for the rescaled learning rate is inconsistent (Δt̂ vs Δ̃t). Please unify the notation.
[II B, Eq. (27)] The recommended range for ν̂ is written as (0, 10^−6), but the text says 'no bounces are needed, so we can set ν̂ = 0'. Since 0 is not in the open interval, please state explicitly whether 0 is allowed and how the recommended range should be interpreted.
[References] Reference [27] is listed without author names; please provide the full citation for the Adam paper.
[III A] The MAE uncertainty is computed from the standard deviation across only N = 10 classifiers for ALEPH. Given the importance of this result, please report the per-bin uncertainties and consider increasing the number of initializations or bootstrapping the MAE estimate.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the headline ECD-vs-Adam comparison rests on external empirical benchmarks, and the theoretical measure is explicitly constructed rather than derived from its own conclusions.

full rationale

The paper's central claim is empirical: ECDq=1 achieves lower MAE than Adam on the ALEPH thrust-reweighting task and on jet reweighting with MLC/BCE losses. These MAEs are computed against external references (histogram ratios and normalizing-flow truth densities), so the headline result is not an output of the optimizer's own assumptions. The theoretical measure p(Theta) proportional to (F(Theta)-F0)^(-eta d/2) is constructed by choosing the Hamiltonian in Eqs. (5)-(11); the paper explicitly says 'we can readily obtain Hamiltonians designed so that this measure concentrates toward the bottom of the objective function.' This is an ansatz or design goal, not a prediction that reduces to its inputs by construction. The hyperparameters eta and F0 are set by heuristics and empirical estimates (e.g., F0=-0.3 from an empirical Fmin ~0.68 for BCE), but the comparison still gives Adam a scan and the reported numbers are measured, not derived from the hyperparameter choices. The theory section relies substantially on self-cited prior work [20,21,32,33], but the load-bearing numeric claims do not depend on those citations being correct; they are direct measurements. The restriction of the ALEPH MAE to bins with T <= 0.3 and the lack of an a priori mixing-time calculation are legitimate correctness and reporting concerns, but they are not circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The optimizer ECDq=1 is an algorithm, not a new physical entity; no new particles, forces, or conserved quantities are postulated beyond the mathematical Hamiltonian on parameter space. The free parameters are F0, eta, Deltat, and nu, with F0 and eta partly set using empirical information from the training losses. The theoretical claims rest on deterministic energy conservation, ergodic mixing, an order-one Hessian scale, and the fidelity of surrogate models used as ground truth.

free parameters (4)

F0 (loss offset) = -1 (ALEPH and MLC jet); -0.3 (BCE jet)
Advertised as a default, but for BCE loss it is set using an empirical estimate Fmin ~ 0.68 found during training (Table IV), so it carries information from the training run rather than being fixed purely a priori.
eta (concentration exponent) = 10^6 (ALEPH); 1.6 x 10^3 (jet)
Set via the heuristic eta ~ n_steps^2 / fanin with the Hessian scale assumed to be O(1) (Eqs. 28-30). It directly controls the claimed concentration of results and is not derived from first principles for the specific losses used.
Deltat_hat (rescaled learning rate) = 0.1 (ALEPH best); 1 (jet)
Scanned over [0.1, 0.5, 1, 2] and selected per problem; it controls step size and convergence, so the reported performance includes a small hyperparameter search.
nu (bounce amplitude) = 0
A chaos-inducing parameter set to zero in all experiments, with a recommended range (0, 10^-6). It is not fitted, but it is part of the ECD hyperparameter space and is not exercised here.

assumptions (5)

domain assumption Training follows deterministic, energy-conserving Hamiltonian dynamics, with minibatch gradient noise effectively negligible or not breaking the conserved energy.
The measure-concentration argument in Sec. II A relies on H = E; actual training uses minibatches, dropout, and early stopping, which inject noise and friction not modeled in Eqs. (5)-(18).
domain assumption Hamiltonian trajectories mix ergodically over the network parameter space within the training budget.
The paper states no a priori calculation of mixing time and sets nu=0 in all experiments, so the theoretical distribution of outcomes in Eq. (11) is assumed to be reached in practice.
ad hoc to paper The Hessian scale F2 of the loss at its minimum is O(1), used to estimate the maximal useful eta in Eq. (28).
The estimate eta ~ n_steps^2 / fanin depends on this assumed scale; no measurement of F2 is provided for the benchmark losses.
domain assumption The normalizing flows trained on Herwig and Pythia data provide exact surrogate densities for the six-dimensional jet benchmark.
The truth likelihood ratio used for MAE is the ratio of flow densities; if the flows are biased, the comparison inherits that bias.
domain assumption For the ALEPH benchmark, the binned histogram ratio is an adequate truth label for the classifier reweighting.
The MAE is measured against a histogram ratio with a bin-center density approximation (footnote 4), so finite-bin resolution adds error not included in the reported uncertainty.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Optimizers for Stabilizing Likelihood-free Inference." pith.science (2026). https://pith.science/paper/OIXBM5UA

@misc{pith2026250118419,
  author       = {Pith},
  title        = {Pith review of: Optimizers for Stabilizing Likelihood-free Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OIXBM5UA}},
  note         = {Machine review of arXiv:2501.18419}
}

abstract

A growing number of applications in particle physics and beyond use neural networks as unbinned likelihood ratio estimators applied to real or simulated data. Precision requirements on the inference tasks demand a high-level of stability from these networks, which are affected by the stochastic nature of training. We show how physics concepts can be used to stabilize network training through a physics-inspired optimizer. In particular, the Energy Conserving Descent (ECD) optimization framework uses classical Hamiltonian dynamics on the space of network parameters to reduce the dependence on the initial conditions while also stabilizing the result near the minimum of the loss function. We develop a version of this optimizer known as $ECD_{q=1}$, which has few free hyperparameters with limited ranges guided by physical reasoning. We apply $ECD_{q=1}$ to representative likelihood-ratio estimation tasks in particle physics and find that it out-performs the widely-used Adam optimizer. We expect that ECD will be a useful tool for wide array of data-limited problems, where it is computationally expensive to exhaustively optimize hyperparameters and mitigate fluctuations with ensembling.

Figures

Figures reproduced from arXiv: 2501.18419 by the authors.

**Figure 2.** FIG. 2. Corner plot of the normalizing flow dataset from [ [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 32 canonical work pages

[1]

A particular example: the q = 1 Hamiltonian In the context of sampling, a particular family of Hamiltonians parameterized by a number q appeared in Ref. [32]. Of these, we will develop the q = 1 Hamilto- nian for the purpose of precision optimization. This was applied for sampling in Ref. [33]. The Hamiltonian for q = 1 is given by, H(Θ, Π) = |Π| −e− L(Θ)...

work page
[2]

Monte Carlo

Adam in physical language It can be instructive to compare the update rules (16)- (18) with the corresponding update rules for Adam [27], which we will use as a baseline. Before doing so, let us start with the simpler example of Gradient Descent with Momentum (GDM) [28, 29]. Its update rules can be written as Π ← βΠ − ∇F (Θ), (19) Θ ← Θ + αΠ, (20) where β...

work page 1994
[3]

advantage is with a narrower range of hyperparameters overall, including the experiments in this section and the last

To fix η, we find that nsteps = epochs × batch size ∼ 800 such that η ∼ 2n2 steps/3n ∼ 1.6 × 103 according to equa- tion (30). advantage is with a narrower range of hyperparameters overall, including the experiments in this section and the last. We also report the hyperparameters for each exper- iment in Table IV. IV. CONCLUSIONS AND OUTLOOK In this paper...

work page
[4]

Cranmer, J

K. Cranmer, J. Brehmer, and G. Louppe, The frontier of simulation-based inference, Proceedings of the National Academy of Sciences 117 (May, 2020) 30055–30062

work page 2020
[5]

Y. Kahn, S. Roy, J. Shelton, and V. Riki, Systematic uncertainties and data complexity in normalizing flows , Neurips workshop (2024)

work page 2024
[6]

Andreev et al., Measurement of Lepton-Jet Correlation in Deep-Inelastic Scattering with the H1 Detector Using Machine Learning for Unfolding , Phys

H1 Collaboration, V. Andreev et al., Measurement of Lepton-Jet Correlation in Deep-Inelastic Scattering with the H1 Detector Using Machine Learning for Unfolding , Phys. Rev. Lett. 128 (2022), no. 13 132002, [arXiv:2108.12376]

arXiv 2022
[7]

Andreev et al., Unbinned deep learning jet substructure measurement in high Q2ep collisions at HERA , Phys

H1 Collaboration, V. Andreev et al., Unbinned deep learning jet substructure measurement in high Q2ep collisions at HERA , Phys. Lett. B 844 (2023) 138101, [arXiv:2303.13620]

arXiv 2023
[8]

Aad et al., Simultaneous Unbinned Differential Cross-Section Measurement of Twenty-Four Z+jets Kinematic Observables with the ATLAS Detector, Phys

A TLASCollaboration, G. Aad et al., Simultaneous Unbinned Differential Cross-Section Measurement of Twenty-Four Z+jets Kinematic Observables with the ATLAS Detector, Phys. Rev. Lett. 133 (2024), no. 26 261803, [arXiv:2405.20041]

arXiv 2024

Show all 58 references

[9]

rep., CERN, Geneva, 2024

CMS Collaboration, Measurement of event shapes in minimum bias events from pp collisions at 13 TeV, tech. rep., CERN, Geneva, 2024

2024
[10]

A TLASCollaboration, G. Aad et al., Measurement of off-shell Higgs boson production in the H ∗ → ZZ → 4ℓ decay channel using a neural simulation-based inference technique in 13 TeV pp collisions with the ATLAS detector, arXiv:2412.01548

arXiv
[11]

Aad et al., Dijet resonance search with weak supervision using √s = 13 TeV pp collisions in the ATLAS detector , Phys

A TLASCollaboration, G. Aad et al., Dijet resonance search with weak supervision using √s = 13 TeV pp collisions in the ATLAS detector , Phys. Rev. Lett. 125 (2020), no. 13 131801, [ arXiv:2005.02983]

2020 arXiv
[12]

Chekhovsky et al., Model-agnostic search for dijet resonances with anomalous jet substructure in proton-proton collisions at √s = 13 TeV , arXiv:2412.03747

CMS Collaboration, V. Chekhovsky et al., Model-agnostic search for dijet resonances with anomalous jet substructure in proton-proton collisions at √s = 13 TeV , arXiv:2412.03747

arXiv
[13]

Cranmer, J

K. Cranmer, J. Pavez, and G. Louppe, Approximating 10 Likelihood Ratios with Calibrated Discriminative Classifiers, arXiv:1506.02169

arXiv
[14]

K. Kong, K. T. Matchev, S. Mrenna, and P. Shyamsundar, New Machine Learning Techniques for Simulation-Based Inference: InferoStatic Nets, Kernel Score Estimation, and Kernel Likelihood Ratio Estimation, arXiv:2210.01680

arXiv
[15]

Brehmer, K

J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez, Constraining Effective Field Theories with Machine Learning, Physical Review Letters 121 (Sep, 2018)

2018
[16]

Brehmer, K

J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez, A Guide to Constraining Effective Field Theories with Machine Learning, Physical Review D 98 (Sep, 2018)

2018
[17]

R. T. D’Agnolo and A. Wulzer, Learning New Physics from a Machine , Physical Review D 99 (Jan, 2019)

2019
[18]

Nachman and J

B. Nachman and J. Thaler, E Pluribus Unum Ex Machina: Learning from Many Collider Events at Once , Physical Review D 103 (June, 2021)

2021
[19]

Stoye, J

M. Stoye, J. Brehmer, G. Louppe, J. Pavez, and K. Cranmer, Likelihood-free inference with an improved cross-entropy estimator, arXiv:1808.00973

arXiv
[20]

G. V. Moustakides and K. Basioti, Training Neural Networks for Likelihood/Density Ratio Estimation , arXiv e-prints (Nov., 2019) arXiv:1911.00405, [arXiv:1911.00405]

2019 arXiv
[21]

Jeffrey and B

N. Jeffrey and B. D. Wandelt, Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison, Mach. Learn. Sci. Tech. 5 (2024), no. 1 015008, [arXiv:2305.11241]

2024 arXiv
[22]

Rizvi, M

S. Rizvi, M. Pettee, and B. Nachman, Learning likelihood ratios with neural network classifiers , JHEP 02 (2024) 136, [ arXiv:2305.10500]

2024 arXiv
[23]

G. B. De Luca and E. Silverstein, Born-infeld (BI) for AI: Energy-conserving descent (ECD) for optimization , in Proceedings of the 39th International Conference on Machine Learning (K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, eds.), vol. 162 of Pr...

2022
[24]

G. B. De Luca, A. Gatti, and E. Silverstein, Improving Energy Conserving Descent for Machine Learning: Theory and Practice, arXiv:2306.00352

arXiv
[25]

M. P. Kuchera, R. Ramanujan, J. Z. Taylor, R. R. Strauss, D. Bazin, J. Bradt, and R. Chen, Machine Learning Methods for Track Classification in the AT-TPC, Nucl. Instrum. Meth. A 940 (2019) 156–167, [arXiv:1810.10350]

2019 arXiv
[26]

Chappell and L

A. Chappell and L. H. Whitehead, Application of Transfer Learning to Neutrino Interaction Classification, Eur. Phys. J. C 82 (2022), no. 12 1099, [arXiv:2207.03139]

2022 arXiv
[27]

F. A. Dreyer, R. Grabarczyk, and P. F. Monni, Leveraging universality of jet taggers through transfer learning, Eur.Phys.J.C 82 (3, 2022) 564, [arXiv:2203.06210]

2022 arXiv
[28]

Beauchesne, Z.-E

H. Beauchesne, Z.-E. Chen, and C.-W. Chiang, Improving the performance of weak supervision searches using transfer and meta-learning , arXiv:2312.06152

arXiv
[29]

Mikuni and B

V. Mikuni and B. Nachman, OmniLearn: A Method to Simultaneously Facilitate All Jet Physics Tasks , arXiv:2404.16091
[30]

D. P. Kingma, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)

2014 arXiv
[31]

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representations by back-propagating errors, nature 323 (1986), no. 6088 533–536

1986
[32]

Qian, On the momentum term in gradient descent learning algorithms, Neural networks : the official journal of the International Neural Network Society 12 1 (1999) 145–151

N. Qian, On the momentum term in gradient descent learning algorithms, Neural networks : the official journal of the International Neural Network Society 12 1 (1999) 145–151

1999
[33]

X. Chen, C. Liang, D. Huang, E. Real, K. Wang, Y. Liu, H. Pham, X. Dong, T. Luong, C.-J. Hsieh, Y. Lu, and Q. V. Le, Symbolic discovery of optimization algorithms, 2023

2023
[34]

Pagliardini, P

M. Pagliardini, P. Ablin, and D. Grangier, The ademamix optimizer: Better, faster, older , arXiv preprint arXiv:2409.03137 (2024)

2024 arXiv
[35]

Robnik, G

J. Robnik, G. B. D. Luca, E. Silverstein, and U. Seljak, Microcanonical hamiltonian monte carlo, Journal of Machine Learning Research 24 (2023), no. 311 1–34

2023
[36]

Robnik and U

J. Robnik and U. Seljak, Fluctuation without dissipation: Microcanonical Langevin Monte Carlo , arXiv:2303.18221

arXiv
[37]

Xiao, Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling , arXiv e-prints (Sept., 2024) arXiv:2409.15156, [arXiv:2409.15156]

L. Xiao, Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling , arXiv e-prints (Sept., 2024) arXiv:2409.15156, [arXiv:2409.15156]

2024 arXiv
[38]

Decamp et al., ALEPH: A detector for electron-positron annnihilations at LEP , Nucl

ALEPH Collaboration, D. Decamp et al., ALEPH: A detector for electron-positron annnihilations at LEP , Nucl. Instrum. Meth. A 294 (1990) 121–178. [Erratum: Nucl.Instrum.Meth.A 303, 393 (1991)]

1990
[39]

Sjostrand, P

T. Sjostrand, P. Eden, C. Friberg, L. Lonnblad, G. Miu, S. Mrenna, and E. Norrbin, High-energy physics event generation with PYTHIA 6.1 , Comput. Phys. Commun. 135 (2001) 238–259, [ hep-ph/0010017]

2001 arXiv
[40]

R. Brun, F. Bruyant, M. Maire, A. C. McPherson, and P. Zanarini, GEANT3,
[41]

Badea, A

A. Badea, A. Baty, P. Chang, G. M. Innocenti, M. Maggi, C. Mcginn, M. Peters, T.-A. Sheng, J. Thaler, and Y.-J. Lee, Measurements of two-particle correlations in e+e− collisions at 91 GeV with ALEPH archived data, Phys. Rev. Lett. 123 (2019), no. 21 212002, [arXiv:1906.00489]

2019 arXiv
[42]

Farhi, A QCD Test for Jets , Phys

E. Farhi, A QCD Test for Jets , Phys. Rev. Lett. 39 (1977) 1587–1588

1977
[43]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting , Journal of Machine Learning Research 15 (2014), no. 56 1929–1958

2014
[44]

Andreassen, P

A. Andreassen, P. T. Komiske, E. M. Metodiev, B. Nachman, and J. Thaler, OmniFold: A Method to Simultaneously Unfold All Observables , Physical Review Letters 124 (May, 2020)

2020
[45]

B¨ ahr, S

M. B¨ ahr, S. Gieseke, M. A. Gigg, D. Grellscheid, K. Hamilton, O. Latunde-Dada, S. Pl¨ atzer, P. Richardson, M. H. Seymour, A. Sherstnev, and B. R. Webber, Herwig++ Physics and Manual , The European Physical Journal C 58 (nov, 2008) 639–707

2008
[46]

Bellm, S

J. Bellm, S. Gieseke, D. Grellscheid, S. Pl¨ atzer, M. Rauch, C. Reuschle, P. Richardson, P. Schichtel, M. H. Seymour, A. Si´ odmok, A. Wilcock, N. Fischer, M. A. Harrendorf, G. Nail, A. Papaefstathiou, and D. Rauch, Herwig 7.0/Herwig++ 3.0 Release Note , The European Physical...

2016
[47]

Bellm, S

J. Bellm, S. Gieseke, D. Grellscheid, P. Kirchgaeßer, F. Loshaj, G. Nail, A. Papaefstathiou, S. Pl¨ atzer, R. Podskubka, M. Rauch, C. Reuschle, P. Richardson, 11 P. Schichtel, M. H. Seymour, A. Si´ odmok, and S. Webster, Herwig 7.1 Release Note , tech. rep., 2017. 7 pages, 7 f...

2017
[48]

Sj¨ ostrand, S

T. Sj¨ ostrand, S. Mrenna, and P. Skands,A Brief Introduction to PYTHIA 8.1 , Computer Physics Communications 178 (Jun, 2008) 852–867

2008
[49]

Sj¨ ostrand, S

T. Sj¨ ostrand, S. Mrenna, and P. Skands,PYTHIA 6.4 Physics and Manual , Journal of High Energy Physics 2006 (May, 2006) 026–026

2006
[50]

Sj¨ ostrand, S

T. Sj¨ ostrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Rasmussen, and P. Z. Skands, An Introduction to PYTHIA 8.2 , Computer Physics Communications 191 (Jun, 2015) 159–177

2015
[51]

de Favereau, C

J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lema ˆ ıtre, A. Mertens, M. Selvaggi, and T. D. . collaboration, Delphes 3: a modular framework for fast simulation of a generic collider experiment , Journal of High Energy Physics 2014 (Feb, 2014) 57

2014
[52]

A. J. Larkoski, S. Marzani, G. Soyez, and J. Thaler, Soft Drop, JHEP 05 (2014) 146, [ arXiv:1402.2657]

2014 arXiv
[53]

Thaler and K

J. Thaler and K. Van Tilburg, Identifying Boosted Objects with N-subjettiness , JHEP 03 (2011) 015, [arXiv:1011.2268]

2011 arXiv
[54]

Thaler and K

J. Thaler and K. Van Tilburg, Maximizing Boosted Top Identification by Minimizing N-subjettiness , JHEP 02 (2012) 093, [ arXiv:1108.2701]

2012 arXiv
[55]

Jimenez Rezende and S

D. Jimenez Rezende and S. Mohamed, Variational Inference with Normalizing Flows , arXiv e-prints (May,
[57]

R. T. D’Agnolo and A. Wulzer, Learning New Physics from a Machine , Phys. Rev. D 99 (2019), no. 1 015014, [arXiv:1806.02350]

2019 arXiv
[58]

Nachman and J

B. Nachman and J. Thaler, Learning from many collider events at once , Phys. Rev. D 103 (2021), no. 11 116013, [arXiv:2101.07263]

2021 arXiv
[2015]

arXiv:1505.05770, [ arXiv:1505.05770]

arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

A particular example: the q = 1 Hamiltonian In the context of sampling, a particular family of Hamiltonians parameterized by a number q appeared in Ref. [32]. Of these, we will develop the q = 1 Hamilto- nian for the purpose of precision optimization. This was applied for sampling in Ref. [33]. The Hamiltonian for q = 1 is given by, H(Θ, Π) = |Π| −e− L(Θ)...

work page

[2] [2]

Monte Carlo

Adam in physical language It can be instructive to compare the update rules (16)- (18) with the corresponding update rules for Adam [27], which we will use as a baseline. Before doing so, let us start with the simpler example of Gradient Descent with Momentum (GDM) [28, 29]. Its update rules can be written as Π ← βΠ − ∇F (Θ), (19) Θ ← Θ + αΠ, (20) where β...

work page 1994

[3] [3]

advantage is with a narrower range of hyperparameters overall, including the experiments in this section and the last

To fix η, we find that nsteps = epochs × batch size ∼ 800 such that η ∼ 2n2 steps/3n ∼ 1.6 × 103 according to equa- tion (30). advantage is with a narrower range of hyperparameters overall, including the experiments in this section and the last. We also report the hyperparameters for each exper- iment in Table IV. IV. CONCLUSIONS AND OUTLOOK In this paper...

work page

[4] [4]

Cranmer, J

K. Cranmer, J. Brehmer, and G. Louppe, The frontier of simulation-based inference, Proceedings of the National Academy of Sciences 117 (May, 2020) 30055–30062

work page 2020

[5] [5]

Y. Kahn, S. Roy, J. Shelton, and V. Riki, Systematic uncertainties and data complexity in normalizing flows , Neurips workshop (2024)

work page 2024

[6] [6]

Andreev et al., Measurement of Lepton-Jet Correlation in Deep-Inelastic Scattering with the H1 Detector Using Machine Learning for Unfolding , Phys

H1 Collaboration, V. Andreev et al., Measurement of Lepton-Jet Correlation in Deep-Inelastic Scattering with the H1 Detector Using Machine Learning for Unfolding , Phys. Rev. Lett. 128 (2022), no. 13 132002, [arXiv:2108.12376]

arXiv 2022

[7] [7]

Andreev et al., Unbinned deep learning jet substructure measurement in high Q2ep collisions at HERA , Phys

H1 Collaboration, V. Andreev et al., Unbinned deep learning jet substructure measurement in high Q2ep collisions at HERA , Phys. Lett. B 844 (2023) 138101, [arXiv:2303.13620]

arXiv 2023

[8] [8]

Aad et al., Simultaneous Unbinned Differential Cross-Section Measurement of Twenty-Four Z+jets Kinematic Observables with the ATLAS Detector, Phys

A TLASCollaboration, G. Aad et al., Simultaneous Unbinned Differential Cross-Section Measurement of Twenty-Four Z+jets Kinematic Observables with the ATLAS Detector, Phys. Rev. Lett. 133 (2024), no. 26 261803, [arXiv:2405.20041]

arXiv 2024