Pith. sign in

REVIEW 3 major objections 3 minor 45 references

Dividing the likelihood-ratio statistic by a goodness-of-fit score stops confidence intervals from shrinking to zero when the model is wrong.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 09:34 UTC pith:LU6ZG7WG

load-bearing objection A clean, useful construction with a real caveat: self-limitation only kicks in if the GOF test can actually see the misspecification. the 3 major comments →

arxiv 2607.20718 v1 pith:LU6ZG7WG submitted 2026-07-22 stat.ME hep-exhep-phphysics.data-an

The Well-Tempered Likelihood: Honest Confidence Intervals for Misspecified Models

classification stat.ME hep-exhep-phphysics.data-an MSC 62F1062F2562G10
keywords misspecified modelsconfidence intervalslikelihood ratiogoodness-of-fitself-limiting intervalseffective sample sizeclassifier two-sample testWilks theorem
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes a simple fix for a deep problem in likelihood-based inference: when the model does not perfectly describe the data, standard confidence intervals keep shrinking as more data arrive, eventually becoming misleadingly precise and centered on a biased value. The fix, dubbed the well-tempered likelihood, is to divide the usual likelihood-ratio test statistic by a goodness-of-fit statistic evaluated at the best-fit point. Under a correct model the divisor is about one and standard inference is unchanged; under a misspecified model both numerator and denominator grow with the dataset size, so their ratio stays bounded and the interval self-limits at a floor set by the model's inadequacy. The paper demonstrates the behavior in a Gaussian example where the floor can be computed analytically and in a particle-physics measurement of the strong coupling constant using classifier-based goodness-of-fit. The key claim is that this construction honestly reports 'the model cannot distinguish parameter values differing by less than this' rather than a spuriously precise number.

Core claim

The paper claims that the statistic T(θ) = Λ(θ)/γ̂, where Λ is the usual likelihood-ratio test statistic and γ̂ is a goodness-of-fit statistic per degree of freedom evaluated at the best fit, gives confidence intervals that are honest under misspecification. When the model is correct, γ̂ tends to one and T recovers standard Wilks-based inference. When the model is wrong, both Λ and γ̂ grow linearly with the number of data points, so T remains O(1) and the resulting interval width plateaus at a constant floor rather than collapsing at the 1/√N rate. The floor corresponds to an effective sample size N_eff = N/γ̂ that saturates at a value set by the severity of the misspecification and the reso

What carries the argument

The central object is the ratio T(θ) = Λ(θ)/γ̂, formed by dividing the standard likelihood-ratio test statistic by a goodness-of-fit statistic evaluated at the best-fit point. The scaling property — γ̂ is O(1) under correct specification and O(N) under misspecification, growing in lockstep with Λ — is what converts an ever-shrinking interval into a self-limiting one. For the binned case the GOF is Pearson's chi-squared per degree of freedom, giving T that follows an F-distribution when the model is correct. For the unbinned case the GOF is a classifier two-sample test statistic converted to a shifted z-score, with per-N null parameters calibrated by permutation. The numerator and denominator

Load-bearing premise

The self-limiting property requires that the goodness-of-fit statistic actually grows at the same O(N) rate as the likelihood under the specific model defect present; if the GOF test is too weak to detect that defect, the tempering is inert and the interval keeps shrinking.

What would settle it

Generate data from a model with a known, small misspecification (e.g., a Gaussian with variance 1.3 fit to unit variance), bin at a very coarse level so the Pearson test has low power, and check whether the well-tempered interval still self-limits or instead keeps shrinking like 1/√N. A second test: train a deliberately low-capacity classifier (e.g., a shallow decision tree) on a high-dimensional misspecified dataset and compare the resulting N_eff plateau to that obtained with a high-capacity classifier; if the plateau disappears or shifts substantially, the safety property is training-depend

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Under correct specification the well-tempered interval is essentially identical to the standard likelihood-ratio interval, so adopting it costs no nominal statistical power when the model is adequate.
  • Under misspecification the interval width saturates at a constant floor proportional to √(D_χ²/(B−2)), so the method refuses to report precision beyond what the goodness-of-fit test can resolve.
  • The effective sample size N_eff = N/γ̂ gives a data-driven measure of how much information the model actually supports, which could inform analysis design and systematic-error reporting.
  • The method is agnostic to the source of misspecification — a classifier two-sample test detects any shape discrepancy between data and model in high-dimensional feature spaces.
  • The framework extends beyond particle physics to any likelihood-based inference in cosmology, gravitational waves, or climate modeling where model error is a concern.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The self-limiting guarantee is only as strong as the GOF test's power: if the chosen test is blind to a particular defect, γ̂ stays O(1) and the interval collapses at the standard rate while wearing the appearance of protection. The paper acknowledges this in Sec. V, saying a weak test is 'more permissive.'
  • The floor on interval width is set by the resolution of the GOF test, meaning that a more powerful classifier yields a wider, more conservative interval. This makes the reported uncertainty depend on the analyst's choice of test architecture, which is a modeling decision rather than an objective property of the data.
  • One could test the method's behavior against a truly adversarial misspecification, such as a defect that a low-capacity classifier misses entirely, to see whether the self-limiting floor is robust or merely an artifact of the test's sensitivity.
  • The duster form could be adapted for hypothesis testing, not just parameter estimation, to prevent overconfident exclusions of physically motivated models based on misspecified backgrounds.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes a modification of likelihood-based inference, the 'well-tempered likelihood', defined by dividing the standard likelihood-ratio statistic Λ(θ) by a goodness-of-fit statistic γ̂ evaluated at the best-fit point: T(θ)=Λ(θ)/γ̂. The authors argue that under correct specification γ̂→1 and standard Wilks inference is recovered, while under misspecification both Λ and γ̂ grow as O(N), so T remains O(1) and confidence intervals self-limit at a floor set by the model's inadequacy. Two concrete GOF choices are developed: a binned Pearson chi-squared statistic and an unbinned classifier two-sample statistic. The method is illustrated on a Gaussian location problem with an analytic floor formula and on a particle-physics-like measurement of α_s using a DCTR surrogate likelihood with a gradient-boosted decision tree GOF. Code and data are provided.

Significance. If the main scaling claim were unconditional, the well-tempered likelihood would be a valuable, simple safeguard against overconfident intervals under misspecification, with clear relevance to high-energy physics and beyond. The Gaussian proof of concept is explicit and internally consistent: the floor w_WT≈0.084 for σ0=1.3, B=50 follows from Dχ2(1.3)≈0.34 and N_eff≈140, and the analytical formulas are checkable. The public code and reproduction data are a definite strength. However, the central guarantee is conditional on the GOF test actually detecting the misspecification. The manuscript acknowledges this only in Sec. V, while the abstract and Eq. (10) present the O(N) scaling as unconditional. As a result, the advertised 'self-limiting guarantee' is overstated relative to what is proved.

major comments (3)
  1. [Abstract; Sec. II.B, Eq. (10)] Eq. (10) and the abstract state that under misspecification γ̂=O(N) unconditionally. This is false for a fixed GOF. For example, with fixed B bins, any misspecification that leaves all bin probabilities at the pseudo-true parameter unchanged (e.g., a within-bin rearrangement of the conditional density) gives Gbin(θ̂MLE)/(B−k−1)→1 under the model, so γ̂=O(1) and T(θ)=Λ(θ)/γ̂ inherits the standard O(N) growth and the interval collapses as 1/√N. The property in Eq. (10) is a property of the pair (misspecification, GOF), not of misspecification alone. The remark in Sec. V that 'a weak or underpowered test is more permissive' does not rescue the abstract's unconditional claim. Because self-limitation is the main advertised result, this conditionality must be stated as a formal hypothesis in Eq. (10), the abstract, and the summary of guarantees.
  2. [Sec. II.C, Eq. (13)] The statement 'T(θ0)/k follows an F_{k,B−k−1} distribution' is not correct as written, because γ̂ is clipped at 1 in Eq. (12). Unconditionally, T/k equals Λ/k on the event Gbin/(B−k−1)<1, and equals (Λ/k)/(Gbin/(B−k−1)) on the complement. The law is therefore a mixture, not a single F distribution. A conditional statement on Gbin/(B−k−1)>1 would be correct, but that is not what is written. This matters for finite-sample calibration, and it is inconsistent with Eq. (21), which uses the χ² quantile rather than the F quantile that the text advertises as a finite-sample correction.
  3. [Sec. II.D, Eq. (16)] The unbinned construction rests on the claim that the null standard deviation σ(N)=std[G_cls]_0 is approximately constant in N. This is load-bearing: G_cls is a sum over 2N event-level terms, so for a raw sum the null spread would grow as √N. A constant σ requires a nontrivial cancellation, e.g., classifier estimation error of order N^{−1/2}; the text only says 'Empirically, ν(N) is negative and approximately constant in N, and σ(N) is positive ... and likewise approximately constant.' If instead σ(N)∼√N (or any growing rate), then under misspecification z=(G_cls−ν)/σ=O(√N), so γ̂=O(√N) and the interval width scales as N^{−1/4}, not the advertised constant floor. The unbinned self-limiting claim therefore needs either a proof of the variance cancellation or a careful empirical scaling study of σ(N) over a wide range of N.
minor comments (3)
  1. [Sec. II.C, Eq. (12) vs. Sec. II.D, Eq. (16)] The two γ̂ constructions are structurally different: the binned version uses a scaled Pearson statistic directly, while the unbinned version uses a shifted z-score. This makes the analogy less direct than the text suggests. Consider unifying the notation or explicitly stating that the binned case is a special case of a more general 'GOF per degree of freedom'.
  2. [Sec. IV.D, Fig. 2c] The misspecified N_eff curve rises from ≈180 at N=500 to ≈480 at N=25000; this is far from the asymptotic plateau of the Gaussian example. The label 'self-limits' is an asymptotic claim; the finite-sample behavior shown is consistent with a slow approach to a floor, but the figure does not yet demonstrate saturation. The text should distinguish between 'approaching a floor' and 'at the floor'.
  3. [Sec. III.C, Eq. (21)] For moderate B, the F distribution has heavier tails than χ²_k/k, so using χ² quantiles in Eq. (21) is a further approximation that is not quantified. If the F-statistic claim is retained as a finite-sample correction, the numerical coverage studies should show the effect of clipping and the choice of quantile.

Circularity Check

0 steps flagged

No significant circularity: the O(N) scaling and self-limiting floor derive from separate asymptotic results; the underpowered-GOF caveat is an acknowledged scope condition, not a fit or self-definition.

full rationale

The derivation chain is self-contained. The well-tempered statistic is defined in Eq. (1) and concretized in Eqs. (12) and (16); it is not fitted. The O(N) growth of Lambda under misspecification follows from standard misspecified-MLE theory (White, Ref. [8]), and the O(N) growth of the binned Pearson GOF follows from the noncentrality calculation in Eq. (19), gamma_hat -> 1 + N D_chi2/(B-2). The claimed floor in Eq. (22), the Neff saturation in Eq. (23), and the Gaussian plateau w_WT ~ 0.084 are derived analytically and then checked against Monte Carlo, not obtained by fitting a parameter to the simulated plateau. In the unbinned case, nu(N) and sigma(N) are empirical null-calibration quantities, not fits to the misspecification signal; gamma_hat remains a test statistic. The paper explicitly flags the conditionality of the O(N) guarantee in Sec. V: 'a weak or underpowered test is more permissive, allowing narrower intervals.' That is a scope condition and correctness risk, not a circular step. The author-overlapping citations (DCTR, Ref. [26]; quark/gluon systematics, Ref. [25]) are tool and observable references used in the physics application; they are not used to prove the central scaling, to import a uniqueness theorem, or to smuggle in an ansatz. The construction is an engineered ratio with explicitly stated asymptotic ingredients, and no step reduces by definition to its own inputs.

Axiom & Free-Parameter Ledger

6 free parameters · 7 axioms · 1 invented entities

The central construction rests on standard misspecified-MLE scaling (Λ = O(N)), a GOF statistic with the matching O(N) scaling (the paper's key premise, contingent on GOF power), and, for the binned null, independence of Λ and G_bin. No physical entities are invented; N_eff is a defined summary, not an independent handle. Method hyperparameters (B, bin span, the 1+z map, K, M, classifier architectures) shape the saturation floor and are user choices rather than values fitted to data.

free parameters (6)
  • B (number of bins) = 50 (Gaussian example)
    Chosen by hand; sets the saturation floor N_eff → (B−2)/D_χ² and hence the self-limiting interval width. The floor scales with B, so the reported 'model inadequacy floor' depends on a user choice.
  • bin span = [μ̂−5, μ̂+5]
    Hand-chosen range for the Gaussian binning; bin edges depend on the fitted mean, so the χ²_{B−2} null is asserted for data-dependent bins.
  • z-score shift form of γ̂ = γ̂ = max(1, 1 + z)
    The '1 + z' mapping from GOF significance to the tempering factor is a modeling choice; alternative monotone maps (e.g., exp(z)) would give different floors and different N_eff values.
  • K (cross-validation folds) = 5
    Standard choice for out-of-fold classifier predictions; authors verified insensitivity in the range 3–10.
  • M (permutations for null calibration) = 20
    Number of permutation splits used to estimate ν(N) and σ(N); the calibration accuracy of the unbinned γ̂ depends on M.
  • GOF classifier / DCTR architecture choices = GBDT for GOF; NN (256,256,256,128), GELU, dropout 0.05, LR 3e-4, AdamW for DCTR
    Capacity and architecture of the classifiers determine detection power and hence the N_eff plateau; the authors acknowledge this as a modeling choice in Sec. V.
axioms (7)
  • standard math Wilks' theorem: under correct specification Λ(θ₀) → χ²_k
    Invoked in Sec. II.A (Eq. 6) as the baseline null that the well-tempered statistic must recover under correct specification.
  • domain assumption Under misspecification, θ̂_MLE → θ* = argmin D_KL(P‖f_θ) and Λ(θ*) = O(N)
    Standard misspecified-MLE theory (Huber 1967, White 1982, Refs. [7,8]); the entire self-limiting argument requires Λ to grow linearly in N.
  • domain assumption Existence of GOF statistics with γ̂ = O(1) under correct specification and O(N) under misspecification (Eq. 10)
    The load-bearing scaling premise. Verified for the Pearson statistic and argued for the classifier two-sample test, but the guarantee is conditional on the GOF test having power against the actual misspecification.
  • domain assumption Independence of Λ(θ₀) and G_bin(θ̂_MLE) for the F_{k,B−k−1} null
    Exact in the Gaussian location family by Cochran's theorem; asserted asymptotically in general via orthogonality of the efficient score and residual GOF components (Ref. [13], Sec. II.C).
  • domain assumption Classifier convergence: ĉ/(1−ĉ) → p/f under universal approximation and sufficient capacity (Eq. 14)
    Stated in footnote 5; the O(N) growth of G_cls depends on the classifier approaching the optimal density ratio.
  • domain assumption DCTR surrogate ℓ(θ)−ℓ(θ₀) equals the true log-likelihood ratio (Eq. 25)
    The physics example builds Λ from a trained parametrized classifier (Ref. [26]); the inference inherits any surrogate error.
  • domain assumption Pythia8.317/Monash faithfully forward-models e+e− → Z → hadrons for the study
    Both 'data' and model are Pythia-based; the misspecification is engineered by changing nuisance parameters, so the demonstration tests the method on a controlled, simulated version of reality.
invented entities (1)
  • N_eff (effective sample size) no independent evidence
    purpose: Summarizes the self-limiting information rate, claimed to characterize the sample size below which the GOF cannot distinguish data from model.
    A derived scalar summary of the construction, not a new physical or statistical entity; the authors themselves note it is 'best understood as a convenient summary ... rather than a true effective sample size in the importance-sampling sense.' It admits no handle outside the method.

pith-pipeline@v1.3.0-alltime-deepseek · 15140 in / 25256 out tokens · 215345 ms · 2026-08-01T09:34:27.961847+00:00 · methodology

0 comments
read the original abstract

Likelihood-based inference in particle physics, and in the physical sciences more broadly, relies on the assumption that the model accurately describes the data. When the model is misspecified, though, standard confidence intervals shrink to zero width with increasing data, producing overconfident and potentially misleading constraints. We propose the well-tempered likelihood, which divides the likelihood-ratio test statistic by a goodness-of-fit (GOF) statistic evaluated at the best-fit point. Under correct specification, the GOF is $\mathcal{O}(1)$ and standard inference is recovered. Under misspecification, both the likelihood and the GOF scale as $\mathcal{O}(N)$ for $N$ data points, so their ratio remains $\mathcal{O}(1)$ and the resulting well-tempered confidence interval self-limits at a floor determined by the model's inadequacy, essentially reducing the effective sample size. In other words, \textit{all models are correct, as long as your dataset is small enough}. We present binned and unbinned formulations of the well-tempered likelihood---the latter based on a classifier two-sample test---and demonstrate our method on a Gaussian example and on a measurement of the strong coupling constant using synthetic electron-positron collisions.

Figures

Figures reproduced from arXiv: 2607.20718 by Benjamin Nachman, Jesse Thaler.

Figure 1
Figure 1. Figure 1: FIG. 1. Gaussian example with the model [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2. Particle physics example using DCTR-based likelihood inference for [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 23 linked inside Pith

  1. [1]

    Note that under variance misspecification, the mean is still unbiased:θ ∗ = θ0 since ¯X→µ 0 regardless ofσ 0

    Ifσ 0 >1 the standard interval undercovers, and ifσ 0 <1 it overcovers. Note that under variance misspecification, the mean is still unbiased:θ ∗ = θ0 since ¯X→µ 0 regardless ofσ 0. This Gaussian proof of concept focuses on coverage/width distortion, while Sec. IV below illustrates both bias and coverage issues. A. Binned W ell-T empered Statistic To demo...

  2. [2]

    the model cannot distinguishα s values differing by less than δ

    usingB= 50 bins. Shaded bands show 16th–84th percentile ranges over 2000 pseudoexperiments.(a)68% CL interval half-width onµversusN. The gray dashed line shows the standard likelihood-ratio interval (∝1/ √ N); solid lines show the well-tempered intervals. Under correct specification (σ 0 = 1, blue) both agree. Under misspecification (σ 0 = 1.3, red), the ...

  3. [3]

    Cranmer, J

    K. Cranmer, J. Brehmer, and G. Louppe, The frontier of simulation-based inference, Proc. Nat. Acad. Sci.117, 30055 (2020), arXiv:1911.01429 [stat.ML]

  4. [4]

    Barlow, Systematic errors: Facts and fictions, inCon- ference on Advanced Statistical Techniques in Particle Physics(2002) pp

    R. Barlow, Systematic errors: Facts and fictions, inCon- ference on Advanced Statistical Techniques in Particle Physics(2002) pp. 134–144, arXiv:hep-ex/0207026

  5. [5]

    Aadet al.(ATLAS), Jet energy scale and resolution measured in proton–proton collisions at √s= 13 TeV with the ATLAS detector, Eur

    G. Aadet al.(ATLAS), Jet energy scale and resolution measured in proton–proton collisions at √s= 13 TeV with the ATLAS detector, Eur. Phys. J. C81, 689 (2021), arXiv:2007.02645 [hep-ex]

  6. [6]

    Khachatryanet al.(CMS), Jet energy scale and reso- lution in the CMS experiment in pp collisions at 8 TeV, JINST12(02), P02014, arXiv:1607.03663 [hep-ex]

    V. Khachatryanet al.(CMS), Jet energy scale and reso- lution in the CMS experiment in pp collisions at 8 TeV, JINST12(02), P02014, arXiv:1607.03663 [hep-ex]

  7. [7]

    Szewc, Defining a Minimum Resolution for Unbinned Analyses, (2026), arXiv:2606.28498 [hep-ph]

    M. Szewc, Defining a Minimum Resolution for Unbinned Analyses, (2026), arXiv:2606.28498 [hep-ph]

  8. [8]

    Navaset al.(Particle Data Group), Review of particle 11 physics, Phys

    S. Navaset al.(Particle Data Group), Review of particle 11 physics, Phys. Rev. D110, 030001 (2024)

  9. [9]

    P. J. Huber, The Behavior of Maximum Likelihood Esti- mates under Nonstandard Conditions, Proc. Fifth Berke- ley Symp. Math. Stat. Prob.1, 221 (1967)

  10. [10]

    White, Maximum Likelihood Estimation of Misspeci- fied Models, Econometrica50, 1 (1982)

    H. White, Maximum Likelihood Estimation of Misspeci- fied Models, Econometrica50, 1 (1982)

  11. [11]

    Gr¨ unwald and T

    P. Gr¨ unwald and T. van Ommen, Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It, Bayesian Analysis12, 1069 (2017)

  12. [12]

    J. W. Miller and D. B. Dunson, Robust Bayesian In- ference via Coarsening, J. Am. Stat. Assoc.114, 1113 (2019)

  13. [13]

    S. S. Wilks, The Large-Sample Distribution of the Like- lihood Ratio for Testing Composite Hypotheses, Annals Math. Statist.9, 60 (1938)

  14. [14]

    W. G. Cochran, The distribution of quadratic forms in a normal system, with applications to the analysis of covariance, Math. Proc. Cambridge Phil. Soc.30, 178 (1934)

  15. [15]

    T. S. Ferguson,A Course in Large Sample Theory(Chap- man & Hall, 1996)

  16. [16]

    Williams, How good are your fits? Unbinned mul- tivariate goodness-of-fit tests in high energy physics, JINST5, P09004, arXiv:1006.3019 [hep-ex]

    M. Williams, How good are your fits? Unbinned mul- tivariate goodness-of-fit tests in high energy physics, JINST5, P09004, arXiv:1006.3019 [hep-ex]

  17. [17]

    J. H. Friedman, On multivariate goodness of fit and two sample testing, eConfC030908, THPD002 (2003)

  18. [18]

    Lopez-Paz and M

    D. Lopez-Paz and M. Oquab, Revisiting Classifier Two- Sample Tests, inInternational Conference on Learning Representations(2017) arXiv:1610.06545 [stat.ML]

  19. [19]

    Neyman and E

    J. Neyman and E. S. Pearson, On the Problem of the Most Efficient Tests of Statistical Hypotheses, Phil. Trans. Roy. Soc. Lond. A231, 289 (1933)

  20. [20]

    J. L. W. V. Jensen, Sur les fonctions convexes et les in´ egalit´ es entre les valeurs moyennes, Acta Mathematica 30, 175 (1906)

  21. [21]

    Bierlichet al., A comprehensive guide to the physics and usage of PYTHIA 8.3, SciPost Phys

    C. Bierlichet al., A comprehensive guide to the physics and usage of PYTHIA 8.3, SciPost Phys. Codeb.2022, 8 (2022), arXiv:2203.11601 [hep-ph]

  22. [22]

    Skands, S

    P. Skands, S. Carrazza, and J. Rojo, Tuning PYTHIA 8.1: the Monash 2013 Tune, Eur. Phys. J. C74, 3024 (2014), arXiv:1404.5630 [hep-ph]

  23. [23]

    Cacciari, G

    M. Cacciari, G. P. Salam, and G. Soyez, The anti-k t jet clustering algorithm, JHEP04, 063, arXiv:0802.1189 [hep-ph]

  24. [24]

    A. Roy, J. Pivarski, and C. W. Freer, An array-oriented Python interface for FastJet, J. Phys. Conf. Ser.2438, 012011 (2023), arXiv:2202.03911 [hep-ex]

  25. [25]

    Cacciari, G

    M. Cacciari, G. P. Salam, and G. Soyez, FastJet User Manual, Eur. Phys. J. C72, 1896 (2012), arXiv:1111.6097 [hep-ph]

  26. [26]

    CMS Collaboration, Performance of quark/gluon dis- crimination in 8 TeV pp data, CMS-PAS-JME-13-002 (2013)

  27. [27]

    P. Gras, S. H¨ oche, D. Kar, A. Larkoski, L. L¨ onnblad, S. Pl¨ atzer, A. Si´ odmok, P. Skands, G. Soyez, and J. Thaler, Systematics of quark/gluon tagging, JHEP07, 091, arXiv:1704.03878 [hep-ph]

  28. [28]

    Andreassen and B

    A. Andreassen and B. Nachman, Neural Networks for Full Phase-space Reweighting and Parameter Tuning, Phys. Rev. D101, 091901 (2020), arXiv:1907.08209 [hep- ph]

  29. [29]

    Cranmer, J

    K. Cranmer, J. Pavez, and G. Louppe, Approximating Likelihood Ratios with Calibrated Discriminative Classi- fiers, (2015), arXiv:1506.02169 [stat.AP]

  30. [30]

    Baldi, K

    P. Baldi, K. Cranmer, T. Faucett, P. Sadowski, and D. Whiteson, Parameterized neural networks for high- energy physics, Eur. Phys. J. C76, 235 (2016), arXiv:1601.07913 [hep-ex]

  31. [31]

    K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition 10.1109/CVPR.2016.90 (2015), arXiv:1512.03385 [cs.CV]

  32. [32]

    J. L. Ba, J. R. Kiros, and G. E. Hinton, Layer Normal- ization, (2016), arXiv:1607.06450 [stat.ML]

  33. [33]

    Hendrycks and K

    D. Hendrycks and K. Gimpel, Gaussian Error Linear Units (GELUs), (2016), arXiv:1606.08415 [cs.LG]

  34. [34]

    Srivastava, G

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A Simple Way to Prevent Neural Networks from Overfitting, J. Machine Learning Res.15, 1929 (2014)

  35. [35]

    D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, inInternational Conference on Learning Representations(2014) arXiv:1412.6980 [cs.LG]

  36. [36]

    Loshchilov and F

    I. Loshchilov and F. Hutter, Decoupled Weight Decay Regularization (2017) arXiv:1711.05101 [cs.LG]

  37. [37]

    R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, A Limited Memory Algorithm for Bound Constrained Optimiza- tion, SIAM J. Sci. Comput.16, 1190 (1995)

  38. [38]

    C. Zhu, R. H. Byrd, P. Lu, and J. Nocedal, Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization, ACM Trans. Math. Softw.23, 550 (1997)

  39. [39]

    Virtanenet al., SciPy 1.0–Fundamental Algorithms for Scientific Computing in Python, Nature Meth.17, 261 (2020), arXiv:1907.10121 [cs.MS]

    P. Virtanenet al., SciPy 1.0–Fundamental Algorithms for Scientific Computing in Python, Nature Meth.17, 261 (2020), arXiv:1907.10121 [cs.MS]

  40. [40]

    Paszkeet al., PyTorch: An Imperative Style, High-Performance Deep Learning Library, (2019), arXiv:1912.01703 [cs.LG]

    A. Paszkeet al., PyTorch: An Imperative Style, High-Performance Deep Learning Library, (2019), arXiv:1912.01703 [cs.LG]

  41. [41]

    J. H. Friedman, Greedy function approximation: A gra- dient boosting machine., Annals Statist.29, 1189 (2001)

  42. [42]

    Chen and C

    T. Chen and C. Guestrin, XGBoost: A Scalable Tree Boosting System 10.1145/2939672.2939785 (2016), arXiv:1603.02754 [cs.LG]

  43. [43]

    Hastie, R

    T. Hastie, R. Tibshirani, and J. Friedman,The Elements of Statistical Learning, 2nd ed., Springer Series in Statis- tics (Springer, 2009)

  44. [44]

    Gretton, K

    A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Sch¨ olkopf, and A. Smola, A Kernel Two-Sample Test, J. Machine Learning Res.13, 723 (2012)

  45. [45]

    G. J. Sz´ ekely and M. L. Rizzo, Energy statistics: A class of statistics based on distances, J. Statist. Plann. Infer- ence143, 1249 (2013)