REVIEW 3 major objections 4 minor 29 references
Asymmetric Errors
T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Asymmetric measurement errors are four distinct problems, and the usual quadrature fix is wrong.
desk verdict A genuinely useful, software-backed framework for asymmetric errors, honest about its own scope; the model-dependence caveat is real but the paper does not oversell what it cannot know. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing objects are two families of three-parameter, near-Gaussian models. For pdf errors, the paper uses distributions such as the dimidiated Gaussian (two half-Gaussians from a one-parameter-at-a-time, 'OPAT', systematic variation approximated by two straight lines) and the distorted Gaussian (a parabolic OPAT dependence), whose first three moments — mean, variance, and unnormalised skewness — add exactly under convolution; errors are combined by summing moments and then converting back to the quantile parameters $M\,{+\sigma_+\atop-\sigma_-}$. For likelihood errors, the machinery is the variable-width Gaussian log-likelihood, in either the linear-$\sigma$ form $\ln L(a)=-\tfrac12[(a-\hat a)/(\sigma+\sigma'(a-\hat a))]^2$ or the linear-variance form $\ln L(a)=-\tfrac12(a-\hat a)^2/[V+V'(a-\hat a)]$, with parameters fixed by the three quoted points; results are combined by summing such log-likelihoods, the combined value being found by the iterative weighted equations (11) and (12), and the combined errors by root-finding where $\Delta\ln L=-\tfrac12$.
What would settle it
Take a measurement whose true distribution is strongly skewed or bimodal (for example a chi-squared variable with one degree of freedom, or a mixture of two well-separated Gaussians), quote it in the form $R\,{+\sigma_+\atop-\sigma_-}$, combine several independent copies with the paper's recommended linear-variance and dimidiated models, and compare with the exact convolution or product likelihood. If the model-based 68% intervals miss the exact intervals by more than the spread between the two recommended models, the paper's claim that two models suffice for a robustness check is refuted; the paper itself passes this test for a Poisson with mean 5.
Extended reading notes
Core claim
The paper's central claim is that a measurement quoted as $R\,{+\sigma_+\atop-\sigma_-}$ is not a single kind of object, and that the apparent lack of a consistent procedure comes from conflating four cases: pdf versus likelihood errors, and combination of errors versus combination of results. Under pdf errors the quoted $\sigma_\pm$ are properties of a probability density, so combining errors means convolving densities and adding moments; under likelihood errors the quoted $\sigma_\pm$ are the $\Delta\ln L=-\tfrac12$ points of a log-likelihood, so combining results means multiplying likelihoods and finding the peak of the sum of log-likelihoods. A directly testable embodiment is the Poisson example: combining the measurement $5\,{+2.581\atop-1.916}$ with itself using the linear-variance likelihood model yields $5.000\,{+1.748\atop-1.415}$, close to the exact combined answer $5.000\,{+1.752\atop-1.419}$. The paper also claims the common recipe of adding the $\sigma_+$ values in quadrature separately from the $\sigma_-$ values is wrong because it preserves shape under many additions and therefore contradicts the Central Limit Theorem.
Load-bearing premise
The whole machinery rests on the assumption that the true distribution behind any quoted asymmetric error is well approximated by a three-parameter, single-peaked, near-Gaussian family, and that the errors being combined are statistically independent; the paper itself notes that the models are not 'correct' and lose reliability for large asymmetries.
Editorial extensions
If this is right
- Combining systematic uncertainties should be done as pdf errors: convert each $\sigma_\pm$ to moments, add the moments, convert back — the central value shifts, and the asymmetry shrinks as more independent errors are combined.
- Combining best-measurement results should be done as likelihood errors: model each quoted peak and error with a linear-sigma or linear-variance log-likelihood, sum the log-likelihoods, and read the combined value and the 68% errors from the summed curve.
- The usual 'add positive errors in quadrature, add negative errors in quadrature, then quote a split Gaussian' recipe is not a small approximation error; it is structurally wrong and can be seriously discrepant, as in the lifetime combination example.
- Whenever the asymmetry is even moderate, using at least two different models (e.g., dimidiated and distorted for pdfs, linear sigma and linear variance for likelihoods) is necessary to know how much the answer can be trusted; for large asymmetries the models disagree and no definite accuracy is claimed.
- If both OPAT deviations go the same way ('flipped'), the paper's advice is to replace the flipped result by a moment-matched ordinary dimidiated distribution rather than build a special flipped model, unless the effect is important enough to demand a full reanalysis.
Reading between the lines
- If the four-way classification is accepted, publication practice should change: a result quoted as $R\,{+\sigma_+\atop-\sigma_-}$ is under-specified, and collaborations or journals should state whether the errors are pdf or likelihood (ideally supplying the full likelihood), otherwise a meta-analyst must guess.
- The same recipes should transfer to any field quoting asymmetric uncertainties — metrology, astronomy, economics — and the coverage of the two-model interval could be tested on known skewed distributions (Poisson, chi-square, log-normal) to calibrate how wide the model family really is.
- A practical decision rule suggested by the paper's large-asymmetry warnings, though not stated as such, is to refuse three-number summaries when $\sigma_+$ and $\sigma_-$ differ by more than roughly a factor of two or three; in that regime the spread between models is the dominant uncertainty.
- The paper's linear-variance success on Poisson examples suggests a testable extension: for counting experiments, replacing each asymmetric result by a generalised-Poisson log-likelihood with the same peak and errors may give near-exact meta-analysis even for very small counts, without needing full likelihoods.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses the practical problem of handling measurements quoted as R +σ+ −σ−. Its central contribution is a taxonomy: asymmetric errors can be pdf errors (rms spread of a probability distribution) or likelihood errors (68% central interval from ΔlnL = −1/2), and the operation can be combination of errors or combination of results. For each cell of this taxonomy the paper proposes families of three-parameter near-Gaussian models, derives conversions between quantile, moment, and model parameters, and gives algorithms for convolution of pdfs and for multiplication/profiling of likelihoods. The methods are validated in cases where the exact answer is known (Poisson counts, exponential lifetimes, transformed Gaussians) and are accompanied by open-source implementations in C++/Python and R. The paper also argues that the common recipe of adding positive and negative errors separately in quadrature is inconsistent with the Central Limit Theorem and that a shift in the central value is generally needed when combining asymmetric errors.
Significance. If the claims hold, this is a genuinely useful contribution for experimental particle physics and metrology. The explicit separation of pdf errors from likelihood errors, the warning that separate-quadrature combination violates the CLT, and the demonstration that the median shifts under convolution are all valuable and likely to influence practice. The algebraic appendices are careful, the software is a concrete deliverable, and the validation against exact Poisson and exponential cases provides real evidence that the recommended Bartlett models work well for smooth, near-Gaussian, moderate asymmetries. The honest acknowledgement that no model is ‘correct’ and that large asymmetries require caution is a strength, not a defect. The main weakness is that the model-family dependence is acknowledged but not quantified: for large asymmetries the spread across models can be comparable to the quoted errors, and the paper does not give a threshold or a calibration for when the procedure should be trusted.
major comments (3)
- [Section 3.4, Eq. (13)] The displayed expression for w_i in the linear-variance model is algebraically wrong in the Gaussian limit. With V'_i = 0 the log likelihood is −(1/2)a_i^2/V_i, so ∂lnL_i/∂a_i = −a_i/V_i and the definition w_i = −(a_i^{-1}∂lnL_i/∂a_i)^{-1} gives w_i = V_i. The printed formula gives V_i/2. For V'_i ≠ 0 the correct local weight is 2(V_i + a_i V'_i)^2/(2V_i + a_i V'_i), not (V_i + a_i V'_i)^2/(2V_i + a_i V'_i). The missing factor cancels in the purely parabolic case, so the symmetric Gaussian examples are unaffected, but it does not cancel in general and changes the profile path for asymmetric inputs. Please correct the formula and confirm that the numerical results in Section 4.1.1 were produced with the corrected weight.
- [Sections 2.2 and 3.1; Table 3] The central claim of a ‘consistent procedure’ is model-dependent in an unquantified way. Table 3 shows that for inputs σ_− = 0.5, σ_+ = 1.5 combined with 0.5/1.5, the predicted combined σ_+ ranges roughly from 1.93 to 2.07 (2.42 for the log-normal model) and σ_− from 0.91 to 1.13 across models that can represent the asymmetry. Section 3.1 similarly states that ‘a wide range of models can potentially give a wide range of outcomes.’ Because no calibration or validity domain is given, the recommendation to use two models does not by itself bound the true answer: the models can agree with each other while both being far from the exact combination for skewed, boundary-truncated, or mixture-like likelihoods. The manuscript should either quantify the asymmetry range within which model spread is below a stated tolerance, add a coverage study on non-smooth cases, or explicitly present the method as an approximate heuristic with a defined scope.
- [Section 4.1.3, Table 10] The Poisson validation is less convincing than the 5+5 row suggests. For two samples split as 8+2, the linear-variance method gives 5.054 +1.856/−1.516 instead of the exact 5.000 +1.752/−1.419, and for 9+1 it gives 5.201 +1.942/−1.605 instead of 5.000 +1.752/−1.419. These deviations are much larger than the 5+5 case and are not negligible for published Poisson measurements, even if a 9+1 split is not the most likely outcome for mean 5. The text dismisses these rows as ‘unlikely experimental circumstances’ with ‘poor goodness of fit,’ but the paper does not provide a threshold for declaring a poor fit or a fallback procedure when the fit is poor. Please state the conditions under which the ‘excellent match’ claim holds and show the corresponding goodness-of-fit values for the rows in Table 10.
minor comments (4)
- [References] Reference [9] contains a typo in the arXiv identifier (‘physis’ instead of ‘physics’) and the citation ‘Merkat A Possolo and O Biodnar’ is inconsistently formatted; please correct the bibliography entries.
- [Table 4] The two inputs in each block of Table 4 are assigned the same asymmetric errors even though their central values differ; the caption should explain that this happens because the underlying Gaussian is the same and only the sampled x value changes.
- [Appendix D] The code example labelled ‘Likelihood and Pdf Errors’ refers to a ‘combined results of Example 5.3’ but the section number corresponds to Section 5.3, and the code block contains a duplicated phi/plo pair and a ‘readline’ prompt that may confuse readers; please check the displayed code against the released package.
- [Figures 6 and 7] The statement that the models ‘coincide’ for moderate asymmetries in the upper rows of Figures 6 and 7 is only qualitative; a sentence giving a numerical tolerance (e.g., agreement in the 68% interval to two significant figures) would make the claim easier to verify.
Circularity Check
No circular derivation; only minor non-load-bearing self-citations.
full rationale
Walking the derivation chain, the paper's three-number inputs (R, sigma+, sigma-) are used to fit three-parameter pdf/likelihood models, and the combined outputs are computed from summed moments (Section 2.2) or summed log-likelihoods (Section 3.2). These operations are nontrivial functions of the inputs; no output equation reduces to an input by construction. The validation examples are genuinely external: Section 4.1.2 compares model combinations of lifetime measurements against the full exponential likelihood product, and Section 4.1.3 compares the linear-variance combination of two Poisson 5-count results (5.000 +1.748/-1.415) with the exact combined answer (5.000 +1.752/-1.419), matching to four significant figures. Neither exact answer is used to set model constants. The paper's own caveats - 'These forms are not correct: there can be no such guarantee' (Section 2.1), 'for large asymmetries... the accuracy should not be considered as being definite' (Section 2.2), and 'This treatment assumes that the errors being considered are independent' (Section 6) - are scope limitations, not circularity. The only self-citations are refs [7], [8] (prior work by the first author on asymmetric errors) and [13] (a technical note on omitting the -ln sigma term in the Bartlett model). These are not load-bearing: the linear sigma/variance models are attributed to Bartlett [16,17], and the paper independently demonstrates their accuracy against exact benchmarks. No uniqueness theorem or prior author result is invoked to forbid alternative models; the paper explicitly recommends using at least two models as a robustness check. Therefore the central claim has independent content and there is no significant circularity.
Assumptions & free parameters
free parameters (3)
- Railway Gaussian transition widths h_l, h_r
- Symmetric beta Gaussian shape parameters p, h
- Conservative spline curvature bound kappa
assumptions (4)
- domain assumption Errors being combined are independent and the final result is a single quantity.
- domain assumption The true distribution underlying an asymmetric error is effectively described by a three-parameter near-Gaussian family.
- domain assumption Quoted asymmetric errors carry no information beyond the three numbers; the origin (pdf or likelihood) is known or must be assumed.
- domain assumption Wilks' theorem provides a usable approximation for the goodness-of-fit statistic when combining likelihoods.
invented entities (2)
-
Three-parameter asymmetric pdf models (dimidiated, distorted, railway, double cubic, symmetric beta Gaussian, QVW, etc.)
independent evidence
-
Three-parameter asymmetric log-likelihood models (linear sigma, linear variance, logarithmic, PDG, etc.)
independent evidence
Cite this review
Pith. "Pith review of Asymmetric Errors." pith.science (2026). https://pith.science/paper/XSRD4RS3
@misc{pith2026241115499,
author = {Pith},
title = {Pith review of: Asymmetric Errors},
year = {2026},
howpublished = {\url{https://pith.science/paper/XSRD4RS3}},
note = {Machine review of arXiv:2411.15499}
}
read the original abstract
We present a procedure for handling asymmetric errors. Many results in particle physics are presented as values with different positive and negative errors, and there is no consistent procedure for handling them. We consider the difference between errors quoted using pdfs and using likelihoods, and the difference between the rms spread of a measurement and the 68\% central confidence region. We provide a comprehensive analysis of the possibilities, and software tools to enable their use.
Figures
Figures from the paper (24 more)
Reference graph
Works this paper leans on
-
[1]
Higgs boson mass and width measurement with the ATLAS detector
S Manzoni. Higgs boson mass and width measurement with the ATLAS detector. In Proc. EPS-HEP 2023, Venice, 2023
work page 2023
-
[2]
Higgs boson properties (mass/width) at CMS
F Errico. Higgs boson properties (mass/width) at CMS. In Proc. EPS-HEP 2023, Venice, 2023
work page 2023
-
[3]
The latest three-flavor neutrino oscillation results from NOvA
M Frank. The latest three-flavor neutrino oscillation results from NOvA. In Proc. EPS-HEP 2023, Venice, 2023
work page 2023
-
[4]
Recent Belle II results on hadronic B decays
M Reif. Recent Belle II results on hadronic B decays. In Proc. EPS-HEP 2023, Venice, 2023
work page 2023
-
[5]
Time-dependent cp violation measurements in b decays
V Chobanova. Time-dependent cp violation measurements in b decays. In Proc. EPS-HEP 2023, Venice, 2023
work page 2023
-
[6]
Averaging Measurements with Hidden Correlations and Asymmetric Errors
M Schmelling. Averaging measurements with hidden correlations and asymmetric errors. arXiv:hep-ex/0006004v1, 2000. 69
work page Pith review arXiv 2000
-
[7]
R J Barlow. Asymmetric systematic errors. arXiv:physics/0306138v1, 2003
arXiv 2003
-
[8]
R J Barlow. Asymmetric statistical errors. arXiv:physics/0406120v1, 2004
arXiv 2004
Show all 29 references
-
[9]
Asymmetric uncertainties: Sources, treatment and potential dangers
G d’Agostini. Asymmetric uncertainties: Sources, treatment and potential dangers. arXiv:physis/0403086.v2, 2004
2004
-
[10]
Asymmetrical uncertainties
C Merkat A Possolo and O Biodnar. Asymmetrical uncertainties. Metrologia, 56:045009, 2019
2019
-
[11]
Everything you always wanted to know about pulls
L Demortier and L Lyons. Everything you always wanted to know about pulls. CDF Note, CDF/ANAL/PUBLIC/5776 (v3), 2008
2008
-
[12]
(Particle Data Group) Review of Particle Properties
S Navas et al. (Particle Data Group) Review of Particle Properties. Phys. Rev. D 110, 030001 (2024)
2024
-
[13]
A note on ∆ lnl=− 1 2 errors
R J Barlow. A note on ∆ lnl=− 1 2 errors. arXiv/physics/0403046, 2004
2004 arXiv
-
[14]
Coverage of Error Bars for Poisson Data CDF report CDF-6438 2003
J G Heinrich. Coverage of Error Bars for Poisson Data CDF report CDF-6438 2003
2003
-
[15]
Fiducial limits for the Poisson distribution
F Garwood. Fiducial limits for the Poisson distribution. Biometrika 28 437–442, 1936
1936
-
[16]
On the statistical estimation of mean lifetimes
M S Bartlett. On the statistical estimation of mean lifetimes. Phil. Mag, 44:244, 1953
1953
-
[17]
Estimation of mean lifetimes from multiple plate cloud chamber tracks
M S Bartlett. Estimation of mean lifetimes from multiple plate cloud chamber tracks. Phil. Mag, 44:1407, 1953
1953
-
[18]
Aaj et al)
The LHCb Collaboration (R. Aaj et al). Amplitude analysis of the decay Λ 0 b →pK −γ. arXiv:2403.03710, 2024
2024
-
[19]
The Bootstrap and Edgeworth Expansion
P Hall. The Bootstrap and Edgeworth Expansion. Springer, 1992
1992
-
[20]
Theoretical comparison of bootstrap confidence intervals
P Hall. Theoretical comparison of bootstrap confidence intervals. Ann. Stat., 16:927, 1988
1988
-
[21]
Keelin and B.W
T.W. Keelin and B.W. Powley. Quantile-parameterized distributions. Decision Analysis, 8:206, 2011
2011
-
[22]
A class of distributions which includes the normal ones
A Azzalini. A class of distributions which includes the normal ones. Scandinavian Journal of Statistics, 12:181, 1985
1985
-
[23]
N.L. Johnson. Systems of frequency curves generated by methods of translation. Biometrika, 36:149, 1949
1949
-
[24]
Elderton and N.L
W.P. Elderton and N.L. Johnson. Systems of Frequency Curves. Cambridge University Press, 1969
1969
-
[25]
Hahn and S.S
G.J. Hahn and S.S. Shapiro. Statistical Models in Engineering. Wiley, 1994
1994
-
[26]
Hill I.D
R. Hill I.D. Hill and R. L. Holder. Algorithm AS 99: fitting Johnson curves by moments. Applied Statistics, 25:180, 1976
1976
-
[27]
Kotz N.L
S. Kotz N.L. Johnson and N. Balakrishnan. Continuous Univariate Distributions, Vol. 2. Wiley, 1995
1995
-
[28]
Private communication, 2004
Wei-Ming Jao. Private communication, 2004
2004
-
[29]
Simplified Wrapper and Interface Generator.https://www.swig.org. 70
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.