Machine Learning-based Unfolding for Cross Section Measurements in the Presence of Nuisance Parameters

Benjamin Nachman; Huanbiao Zhu; Krish Desai; Larry Wasserman; Mikael Kuusela; Vinicius Mikuni

REVIEW 3 major objections 4 minor 1 cited by

Machine Learning-based Unfolding for Cross Section Measurements in the Presence of Nuisance Parameters

T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces Profile OmniFold, a machine-learning-based unfolding method that simultaneously reweights simulated events and profiles nuisance parameters in the detector response, recovering the true particle-level distribution when

desk verdict A genuinely useful extension of OmniFold that profiles nuisance parameters in unbinned unfolding; the central claim holds in the demonstrated experiments, but the heuristic initialization-selection rule (V) is the load-bearing soft spot and should be probed in review. read the letter →

arxiv 2512.07074 v3 pith:DN7DFTR6 submitted 2025-12-08 stat.AP hep-exhep-phphysics.data-anstat.ML

Huanbiao Zhu , Krish Desai , Mikael Kuusela , Vinicius Mikuni , Benjamin Nachman , Larry Wasserman This is my paper · ORCID

classification stat.APhep-exhep-phphysics.data-anstat.ML

keywords unfoldingnuisanceparametersOmniFoldProfileclassifier-baseddensityratioestimationexpectation-maximizationsimulation-basedinferencecrosssectionmeasurement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Profile OmniFold (POF), an extension of the OmniFold algorithm that performs unbinned, simulation-based unfolding of detector-smeared data while simultaneously profiling nuisance parameters in the forward model. POF iteratively reweights simulated particle-level events to match experimental data and updates the nuisance parameters by maximizing the population log-likelihood, using classifiers to estimate all required density ratios. The authors demonstrate, on a Gaussian example and on simulated CMS jet data, that when the detector simulation's response kernel is misspecified, POF recovers the true particle-level distribution, while standard OmniFold yields biased results. The paper also highlights a practical caveat: POF can converge to local maxima depending on the initialization of the nuisance parameter, so the authors propose a classifier-based goodness-of-fit statistic V to select among multiple initializations.

What carries the argument

The central object is the conditional density ratio w(y,x,θ)=p(y|x,θ)/q(y|x), which reweights the Monte Carlo response kernel to account for nuisance parameters. POF is an EM algorithm whose Q-function separates into a ν-dependent term and a θ-dependent term, allowing the particle-level reweighting and the nuisance-parameter update to be performed independently each iteration. Density ratios for both the detector-level reweighting and for w itself are estimated with binary classifiers (Proposition 3 factors w into two classifier ratios). To pick among multiple initializations, the method uses the heuristic statistic V, derived from the step-1 classifier's validation accuracy, which is near 1

What would settle it

Take a simulation with a known true θ* and run POF from a grid of initializations spread across the parameter space; if the solution with the highest V has θ̂ distant from θ* (while a lower-V run is closer to θ*), the selection rule is contradicted. A sharper version: in the Gaussian example with analytic w, if the V-maximizing run ever yields an unfolded density that is further from the true density (by KS distance or L1) than a lower-V run, the central claim that the selected POF solution recovers the truth fails.

Watch

Extended reading notes

Core claim

Profile OmniFold treats the unfolding inverse problem as a joint maximum-likelihood estimation of the particle-level reweighting function ν(x) and the nuisance parameter θ in the forward model p(y|x,θ). At each EM iteration, it (1) reweights detector-level simulation to match data, (2) pulls the ratio back to particle level, and (3) updates θ by maximizing a Q-function term that depends on the learned conditional density ratio w(y,x,θ)=p(y|x,θ)/q(y|x). The authors prove (Proposition 2) that the ν and θ updates separate, and (Proposition 3) that w can be estimated as a product of two classifier-based density ratios. In experiments, POF with correctly estimated w recovers the true particle-lev

Load-bearing premise

The algorithm's success rests on the untested heuristic that the V-statistic—the step-1 classifier's weighted accuracy—identifies the global maximum of the likelihood; the CMS results show V fails to flag a bad local optimum when initialization is poor, so a scenario with only poor initializations would break the method.

Editorial extensions

If this is right

With a correctly estimated w, POF recovers the true particle-level distribution in the Gaussian and CMS studies, while OmniFold’s solution is visibly biased.
POF performs unbinned profiling at both detector and particle level, unlike the earlier profiled unfolding approach that required binned detector-level data.
Because POF preserves OmniFold’s classifier-based density-ratio estimation, it can be implemented as a drop-in extension of existing OmniFold software.
The reported sensitivity to initialization implies that users must run multiple starting values and use the V-statistic to select the solution.
The EM separation result (Proposition 2) provides a foundation for adding nuisance-parameter profiling to other simulation-based unfolding methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If V consistently ranks solutions, it could also serve as a model-misspecification diagnostic for standard OmniFold, flagging runs where the reweighted detector-level distribution never matches data.
The classifier-based factorization of w suggests a block-coordinate extension to multiple nuisance parameters; the scalar demonstration leaves that as a natural next step.
The observed sensitivity of θ̂ to classifier training suggests that ensembles over w, rather than a single fit, may be needed for reliable profiling; the paper notes sensitivity but does not quantify it.
POF could be paired with profile-likelihood or bootstrap techniques to attach uncertainties to both the unfolded density and the nuisance parameter, closing the current gap in uncertainty quantification.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read the POF paper. Short version: it does what it says—extends OmniFold to profile a nuisance parameter in the forward model, fully unbinned, with coherent EM machinery and honest experiments. I'd send it to peer review, expecting moderate revision.

What's new: OmniFold assumes the detector response is exactly known, and Chan & Nachman's profiled unfolding requires binned detector-level data. POF is the first fully unbinned classifier-based EM that updates the reweighting function and the nuisance parameter jointly, using a learned response ratio w(y,x,θ)=p(y|x,θ)/q(y|x). I checked the supplement: the proofs of Props 1–3 are sound, and Prop 3 formalizes a classifier trick Chan & Nachman used without proving. The experiments earn their keep: with the analytic w, the Gaussian example recovers θ̂=1.48 vs true 1.5, and the unfolded density tracks truth while OmniFold is visibly biased. In the CMS jet-resolution study (truth θ=1.7), POF gives θ̂=1.62 and a particle-level result that matches truth. Code and data are posted openly. The paper is also upfront about its limitations—θ̂ sensitivity to w training, no uncertainty quantification, local optima—in Sections 5 and 6.

Soft spots, in proportion. The main one is the selection rule. The algorithm is run from multiple θ(0) and the reported solution is whichever maximizes V, a goodness-of-fit statistic built from the step-1 classifier's validation accuracy. The paper calls V a heuristic and does not study it. In the CMS example, initializations at 1.0 and 1.1 fall into a local optimum θ̂≈1.35 with V<1, and V picks the better run—so the headline demonstration literally depends on V doing its job. The concern that V is detector-level and might not rank particle-level accuracy is fair but not fatal: detector-level fit is the only thing the observed data can constrain, and in the presented cases V selects the solution that is also right at particle level. Still, a referee should ask for a check of the selection rule against known-truth synthetic data, and for reporting of what happens without selection.

Second, θ̂ is fragile with respect to the estimated w: Gaussian θ̂=1.42 vs 1.5, and the paper notes shifts with w training range and seed. The unfolded density is more robust, which is the primary output, but the nuisance-parameter estimate being this sensitive matters for a method whose selling point is profiling.

Third, there is no uncertainty quantification of any kind. The authors acknowledge this; for a measurement-oriented community it limits immediate deployment.

Who this is for: HEP physicists and statisticians working on simulation-based unfolding. A solid methods paper, not a field-reshaping one. My recommendation: engage with it, send it out, and make the review center on whether V's selection is reliable enough to carry the empirical claims.

Referee Report

3 major / 4 minor

Summary. The paper proposes Profile OmniFold (POF), an extension of the classifier-based OmniFold algorithm for unbinned unfolding that jointly estimates the particle-level reweighting function ν(x) and a scalar nuisance parameter θ entering the detector response kernel. The authors formulate a population-level likelihood with a prior/penalty on θ, derive EM updates (Propositions 1–3), and implement the algorithm using neural-network density-ratio estimators. They validate POF on a two-dimensional Gaussian example with analytic and estimated w-functions, and on a CMS Open Data simulation for inclusive jet-pT spectra, comparing against the original OmniFold. The paper introduces a heuristic goodness-of-fit statistic V, Eq. (25), to select among runs with different θ initializations. It acknowledges that POF lacks convergence guarantees, that θ estimates are sensitive to the estimated w-function, and that no uncertainty quantification is provided.

Significance. If established, POF would fill a genuine gap: extending simulation-based unbinned unfolding to the practically important case where the forward model depends on nuisance parameters, thereby avoiding the expensive 'repeat the measurement under systematic variations' paradigm. The paper's formal EM derivation, the classifier-based construction of the w-function, and the use of public CMS simulation data are strengths, as are the clear statements of limitations. However, the central empirical claim—that POF accurately recovers the true particle-level distribution under a misspecified forward model—is presently demonstrated only after selecting among multiple initializations using an unvalidated heuristic. Because the inverse problem is ill-posed, detector-level agreement does not by itself imply particle-level accuracy, so the selection rule needs justification. The lack of uncertainty quantification further tempers the 'accurate' claim. The methodology is promising and the derivations appear sound, but the current evidence is conditional.

major comments (3)

[Sec. 3.1, Eq. (25); Sec. 5.3, Fig. 10] The central empirical claim relies on the heuristic statistic V to select among multiple initialization runs, but V is not validated as a ranking of particle-level accuracy. The paper states in Sec. 3.1 that V 'is a heuristic statistic' whose statistical properties have not been investigated. In the CMS study (Sec. 5.3, Fig. 10), initializations θ^(0)=1.0 and 1.1 converge to θ̂≈1.35 with V<1, while the reported result is the run with the highest V. Since V is based on the validation accuracy of a detector-level classifier, and since the unfolding problem is ill-posed, there is no guarantee that a high-V solution is more accurate at the particle level. Please provide either a theoretical justification or a controlled experiment (e.g., many simulated truths with varying ν and θ, checking that argmax V selects the solution closest to the truth in a particle-level metric) before the headline
[Sec. 6; Sec. 4.3; Sec. 5.3] No uncertainty quantification is provided for either the unfolded distribution or the nuisance parameter, as the paper acknowledges in Sec. 6. The observed point estimates with the estimated w-function (Gaussian θ̂=1.42 vs true 1.5; CMS θ̂=1.62 vs true 1.7) and the stated sensitivity of θ̂ to classifier training (Sec. 4.3) make it impossible to judge whether these differences are statistical fluctuations or systematic biases. Even a bootstrap or repeated-simulation variability assessment would materially strengthen the claim that POF 'accurately estimates' the true distribution. Without such quantification, the empirical support for the central claim is incomplete.
[Sec. 3.1; Sec. 5.3] The lack of convergence guarantees is a practical issue that interacts with the selection rule. The paper notes that the likelihood is not concave and that POF can converge to a local maximum (Sec. 5.3). The proposed remedy—multiple initializations plus the V heuristic—is not accompanied by any guidance on how many initializations are needed, how to choose their range, or how to detect when the V-based selection has failed. This is not a fatal flaw, but it is a load-bearing aspect of the method's applicability and should be addressed, at least empirically, in the revision.

minor comments (4)

[Eq. (25)] Please define w_i explicitly: is it the detector-level weight assigned in step 1 of the current iteration? The notation is currently ambiguous.
[Sec. 5.1] The dataset description says events are used as both 'simulation' and 'data'; clarify that the 'data' are simulated events from the same detector simulation, and that 'data' is produced by reweighting the MC sample. The distinction between the constructed θ=1.7 'truth' and the nominal θ=1.0 simulation should be stated more prominently.
[References] There are several typographical issues in the reference list, e.g., 'Physical Reivew Letters' in Andreassen et al. (2020). A careful proofread is needed.
[Figs. 4, 7, 10] The top/bottom plot labels ('Updated estimates θ̂' and 'Goodness-of-fit statistic') are useful; consider adding them as axis labels in the figures for clarity.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the POF updates are derived from the likelihood/Q-function, the w-function is estimated from independent synthetic forward-model samples with a supplied proof, and the claimed particle-level estimates are not equal by construction to any fitted input.

full rationale

I walked the derivation chain. Proposition 2 (Sec. 3.1) derives the POF updates for ν and θ by maximizing the Q-function in Eq. (21); the proof is reproduced in Supp. B.2 and does not assume the conclusion. Proposition 3 is likewise proved in Supp. B.3: the classifier-ratio factorization of w(y,x,θ) is derived from Bayes' rule, not imported as an unverified ansatz. The OmniFold EM structure is re-derived in Sec. 2.3 (Eq. 19), so reliance on the OmniFold citation is not load-bearing for the paper's own derivation. The nuisance parameter θ is explicitly fitted by profiling, which is the intended structure, and the unfolded distribution ν is not obtained by fitting to the target truth. The Gaussian and CMS studies use simulated data with known truth, so the empirical claim is externally checkable. The heuristic V used for initialization selection is explicitly flagged by the authors: 'Currently, V is a heuristic statistic... we have not yet established a principled investigation of its statistical properties' (Sec. 3.1), and the CMS case shows initialization sensitivity (Fig. 10). This is a validation/robustness limitation, not circularity, because V is a detector-level goodness-of-fit proxy and is not the particle-level quantity being predicted; no equation reduces the particle-level estimate to V or to the fitted θ by construction. The paper does cite the authors' own earlier workshop paper (Zhu et al. 2024) and Chan & Nachman (2023) with overlapping authorship, but these citations are not load-bearing since the needed derivations are included in the manuscript. I therefore find no circular step and assign a low score reflecting only minor self-citation.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a known-form forward model, identifiability, calibrated classifiers, and the V-selection heuristic. The method's own free parameters are θ and the w-training range; no invented physical entities are posited.

free parameters (3)

Nuisance parameter θ (noise scale / jet energy resolution multiplier) = Gaussian: 1.48 (analytic w), 1.42 (estimated w), true 1.5; CMS: 1.62, true 1.7
Central profiling parameter estimated by maximizing Q2 (Eq. 23); the whole method is designed to fit it.
Training range for θ in w-function classifiers = [0.5, 2.0] in main text; [0.5, 1.5] in CMS supplement
Chosen by hand; the paper reports θ̂ is sensitive to this range and to other w-training details (Sec. 4.3 and Sec. 6).
Prior/penalty log p0(θ) = unspecified in the experiments
Eq. (20) includes a prior, but the demonstrations never state its form or σ0; presumably a flat prior is used. This affects the θ estimate.

assumptions (5)

domain assumption Forward model p(y|x,θ) is known up to scalar θ and can be sampled for arbitrary θ in a training range.
Sec. 3.2 and 5.1: w is trained on synthetic data generated from Eqs. (28) and (30) with varied θ.
domain assumption The pair (ν, θ) is identifiable from detector-level data.
Gaussian identifiability is argued via characteristic functions (Eq. 29); the CMS case is asserted from symmetry but not proven.
standard math Bayes-optimal classifiers give calibrated density ratios.
Sec. 2.4 and Prop. 3 rely on Bayes optimality to turn classifier outputs into density ratios.
domain assumption MC and data distributions have overlapping support with finite density ratios.
Sec. 5.3 acknowledges support mismatches and unstable weights around peaks, yet the method still requires finite ratios to reweight.
ad hoc to paper The V heuristic selects the global optimum among initializations.
Sec. 3.1: V has no established statistical properties; CMS results show it does not flag θ≈1.35 as a poor solution for some initializations (Fig. 10).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Machine Learning-based Unfolding for Cross Section Measurements in the Presence of Nuisance Parameters." pith.science (2026). https://pith.science/paper/DN7DFTR6

@misc{pith2026251207074,
  author       = {Pith},
  title        = {Pith review of: Machine Learning-based Unfolding for Cross Section Measurements in the Presence of Nuisance Parameters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DN7DFTR6}},
  note         = {Machine review of arXiv:2512.07074}
}

read the original abstract

Statistically correcting measured cross sections for detector effects is an important step across many applications. In particle physics, this inverse problem is known as unfolding. In cases with complex instruments, the distortions they introduce are often known only implicitly through simulations of the detector. Modern machine learning has enabled efficient simulation-based approaches for unfolding high-dimensional data. Among these, one of the first methods successfully deployed on experimental data is the OmniFold algorithm, a classifier-based Expectation-Maximization procedure. In practice, however, the forward model is only approximately specified, and the corresponding uncertainty is encoded through nuisance parameters. Building on the well-studied OmniFold algorithm, we show how to extend machine learning-based unfolding to incorporate nuisance parameters. Our new algorithm, called Profile OmniFold, is demonstrated using a Gaussian example as well as a particle physics case study using simulated data from the CMS Experiment at the Large Hadron Collider.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Reweighting Adversarial Networks for Unbinned Unfolding
hep-ph 2026-06 unverdicted novelty 7.0 of 10

RANs generalize moment unfolding to full phase-space unbinned unfolding via detector-level Wasserstein critics without requiring support overlap or multiple iterations.

Reference graph

Works this paper leans on

2 extracted references · cited by 1 Pith paper

[1]

ABADI, M., BARHAM, P., CHEN, J., CHEN, Z., DAVIS, A., DEAN, J., DEVIN, M., GHEMAWAT, S., IRVING, G., ISARD, M. et al. (2016). TensorFlow: A System for Large-Scale Machine Learning. InOSDI16265–283. AGOSTINELLI, S. et al. (2003). GEANT4–a simulation toolkit.Nucl. Instrum. Meth. A506250–303. https: //doi.org/10.1016/S0168-9002(03)01368-8 ALLISON, J. et al. ...

arXiv 2016
[2]

Therefore, the stationary point satisfies f(x) = Z p(y|x)f (k)(x)R p(y|x′)f (k)(x′)dx′ p(y)dy. Moreover, the second order derivative δ ˜Q δf(x)δf(x ′) satisfies Z Z δ ˜Q δf(x)δf(x ′) ϕ(x)ϕ(x′)dxdx′ = d2 dϵ2 ˜Q(f+ϵϕ) ϵ=0 = d dϵ Z p(y) Z p(x|y, f(k)) ϕ(x) (f(x) +ϵϕ(x)) dxdy−λ Z ϕ(x)dx ϵ=0 = − Z p(y) Z p(x|y, f(k)) ϕ2(x) [(f(x) +ϵϕ(x))] 2 dxdy ϵ=0 = Z ϕ2(x) ...

2020

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

ABADI, M., BARHAM, P., CHEN, J., CHEN, Z., DAVIS, A., DEAN, J., DEVIN, M., GHEMAWAT, S., IRVING, G., ISARD, M. et al. (2016). TensorFlow: A System for Large-Scale Machine Learning. InOSDI16265–283. AGOSTINELLI, S. et al. (2003). GEANT4–a simulation toolkit.Nucl. Instrum. Meth. A506250–303. https: //doi.org/10.1016/S0168-9002(03)01368-8 ALLISON, J. et al. ...

arXiv 2016

[2] [2]

Therefore, the stationary point satisfies f(x) = Z p(y|x)f (k)(x)R p(y|x′)f (k)(x′)dx′ p(y)dy. Moreover, the second order derivative δ ˜Q δf(x)δf(x ′) satisfies Z Z δ ˜Q δf(x)δf(x ′) ϕ(x)ϕ(x′)dxdx′ = d2 dϵ2 ˜Q(f+ϵϕ) ϵ=0 = d dϵ Z p(y) Z p(x|y, f(k)) ϕ(x) (f(x) +ϵϕ(x)) dxdy−λ Z ϕ(x)dx ϵ=0 = − Z p(y) Z p(x|y, f(k)) ϕ2(x) [(f(x) +ϵϕ(x))] 2 dxdy ϵ=0 = Z ϕ2(x) ...

2020