pith. sign in

arxiv: 2509.02852 · v3 · submitted 2025-09-02 · ⚛️ physics.data-an

Confidence intervals for the Poisson distribution

Frank C. Porter This is my paper

Pith reviewed 2026-05-18 19:29 UTC · model grok-4.3

classification ⚛️ physics.data-an
keywords Poisson distributionconfidence intervalsGarwood intervalsphysics data analysisstatistical reportingp-valuesdata interpretation
0
0 comments X p. Extension

The pith

Garwood confidence intervals should be used to report results from Poisson sampling in physics.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines conventional and alternative methods for constructing confidence intervals for the Poisson distribution, which often arises in physical science measurements. It evaluates these methods based on several desirable properties including p-value behavior and consistency. The central recommendation is to use the intervals proposed by Garwood for summarizing results. This approach is favored because the resulting p-values are well-behaved and intuitive, allowing for consistent treatment of data. The paper also highlights that averaging intervals can be problematic unless the underlying Poisson distributions are properly accounted for.

Core claim

After considering performance with respect to several desirable properties we recommend summarizing the results of Poisson sampling with confidence intervals proposed by Garwood. We note that the p-values obtained from these intervals are well-behaved and intuitive, providing for consistent treatment. We also find that averaging intervals can be problematic if the underlying Poisson distributions are not used.

What carries the argument

Garwood's confidence intervals, selected after comparing their performance on p-value behavior, consistency, and other properties against conventional and alternative techniques.

If this is right

  • Poisson sampling results can be summarized with consistent and intuitive p-values using Garwood intervals.
  • Averaging intervals requires using the original Poisson distributions to avoid problems.
  • Physicists gain a uniform way to describe results across different count measurements.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Standard adoption could reduce confusion in how uncertainties are reported across experiments.
  • The approach may influence limit-setting procedures in low-count regimes.
  • Practical tests in existing analysis software would show whether the recommendation improves reporting workflows.

Load-bearing premise

The chosen desirable properties for evaluation, such as p-value behavior and consistency, are the most relevant criteria for practical use in physics data reporting rather than other statistical measures like exact coverage guarantees.

What would settle it

A specific physics analysis example where an alternative interval method shows superior p-value intuition or consistency compared to Garwood intervals would challenge the recommendation.

Figures

Figures reproduced from arXiv: 2509.02852 by Frank C. Porter.

Figure 1
Figure 1. Figure 1: Performance of 68% confidence standard upper limits. Left: coverage probability [PITH_FULL_IMAGE:figures/full_fig_p018_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance of Garwood intervals. Left: coverage probability as a function of [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Performance of Sterne and Crow&Gardner 68% confidence intervals. Left: cover [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance of Garwood and Crow&Gardner 68% confidence intervals. Left: cov [PITH_FULL_IMAGE:figures/full_fig_p023_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: The Crow&Gardner intervals overcover less than the Sterne intervals. [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of the symmetry of the Garwood and Crow&Gardner 68% confidence [PITH_FULL_IMAGE:figures/full_fig_p024_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance of Blaker and Sterne 68% confidence intervals. Left: coverage probabil [PITH_FULL_IMAGE:figures/full_fig_p025_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of the symmetry of the Garwood and Blaker 68% confidence intervals. [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Confidence intervals as a function of confidence level 1 [PITH_FULL_IMAGE:figures/full_fig_p027_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Performance of Kabaila&Byrne and Crow&Gardner 68% confidence intervals. Left: [PITH_FULL_IMAGE:figures/full_fig_p028_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Performance of LR and Crow&Gardner 68% confidence intervals. Left: coverage [PITH_FULL_IMAGE:figures/full_fig_p030_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Performance of LR and score 68% confidence intervals. Left: coverage probability [PITH_FULL_IMAGE:figures/full_fig_p031_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Coverage probability of √ n confidence intervals as a function of µ. The horizontal line shows 1 − α. We could generalize this approach to other confidence levels, still using the asymptotic normal approximation. The coverage probability of the √ n intervals is shown as a function of µ in [PITH_FULL_IMAGE:figures/full_fig_p032_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Bayes 68% confidence credible intervals for [PITH_FULL_IMAGE:figures/full_fig_p034_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Comparison of Crow&Gardner 68% confidence intervals and Bayes 68% confidence [PITH_FULL_IMAGE:figures/full_fig_p035_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Performance of 90% confidence upper limits, comparing [PITH_FULL_IMAGE:figures/full_fig_p038_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Performance of 68% confidence intervals for [PITH_FULL_IMAGE:figures/full_fig_p039_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Performance of 68% confidence intervals for [PITH_FULL_IMAGE:figures/full_fig_p040_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Coverage of averaging (symmetrized) Garwood intervals (blue dashed curve) ac [PITH_FULL_IMAGE:figures/full_fig_p042_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Left: Expected length of 68% CL intervals, as a function of [PITH_FULL_IMAGE:figures/full_fig_p045_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Left: Expected length of 68% CL intervals, as a function of [PITH_FULL_IMAGE:figures/full_fig_p046_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: The Garwood confidence interval as a function of confidence level for [PITH_FULL_IMAGE:figures/full_fig_p048_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: The p values as a function of null hypothesis θ0 for an observation n = 3 (and no background). Left: The continuous orange curve is according to the Garwood interval and the discontinuous purple curve is according to the likelihood ratio interval. Right: The p values according to the Kabailla&Byrne intervals, also for n = 3. large change in p, which is not something we would normally expect given the cont… view at source ↗
read the original abstract

The Poisson probability distribution is frequently encountered in physical science measurements. In spite of the simplicity and familiarity of this distribution, there is considerable confusion among physicists concerning the description of results obtained via Poisson sampling. The goal of this paper is to mitigate this confusion by examining and comparing the properties of both conventional and popular alternative techniques. We concern ourselves in particular with the description of results, as opposed to interpretation. After considering performance with respect to several desirable properties we recommend summarizing the results of Poisson sampling with confidence intervals proposed by Garwood. We note that the p-values obtained from these intervals are well-behaved and intuitive, providing for consistent treatment. We also find that averaging intervals can be problematic if the underlying Poisson distributions are not used.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The manuscript compares conventional and alternative techniques for constructing confidence intervals for the Poisson distribution in the context of physical science measurements. It evaluates these methods with respect to several desirable properties, including p-value behavior and consistency, and recommends the Garwood intervals for summarizing results from Poisson sampling. The paper notes that these intervals yield well-behaved and intuitive p-values and highlights problems with averaging intervals when the underlying Poisson distributions are not accounted for.

Significance. If the evaluation of properties is comprehensive and the recommendation is supported, this work could help standardize the reporting of Poisson-distributed measurements and reduce confusion among physicists. The emphasis on descriptive properties rather than interpretive ones aligns with practical needs in data analysis for low-count regimes.

major comments (1)
  1. [Evaluation of properties and recommendation] The recommendation for Garwood intervals rests on p-value behavior and consistency. However, the defining property of a confidence interval is its frequentist coverage probability (at least the nominal level for all true parameter values). The manuscript does not report explicit calculations or tabulations of the coverage function for Garwood intervals (or comparators) across a range of small Poisson means, where discreteness effects are strongest. This is a load-bearing omission for the central claim, as alternatives are frequently criticized for undercovering in exactly these regimes.
minor comments (1)
  1. [Abstract] The abstract refers to 'several desirable properties' without enumerating them; a brief list in the abstract or introduction would improve accessibility.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their thoughtful and constructive review of our manuscript. We address the major comment below.

read point-by-point responses

  1. Referee: The recommendation for Garwood intervals rests on p-value behavior and consistency. However, the defining property of a confidence interval is its frequentist coverage probability (at least the nominal level for all true parameter values). The manuscript does not report explicit calculations or tabulations of the coverage function for Garwood intervals (or comparators) across a range of small Poisson means, where discreteness effects are strongest. This is a load-bearing omission for the central claim, as alternatives are frequently criticized for undercovering in exactly these regimes.

    Authors: We agree that frequentist coverage is the defining property of a confidence interval and that explicit evaluation of the coverage function is important, particularly for small Poisson means where discreteness effects are pronounced. Our manuscript prioritizes properties directly relevant to the descriptive use of intervals in physical measurements—such as p-value behavior, intuitiveness, and consistency—because these address common sources of confusion among physicists when reporting results. Nevertheless, we acknowledge the value of also presenting coverage calculations to provide a fuller assessment. In the revised manuscript we will add explicit tabulations and figures showing the coverage probability of the Garwood intervals and the comparator methods over a range of small means (for example, λ = 0 to 10). This addition will allow readers to evaluate coverage alongside the other performance characteristics already discussed. revision: yes

Circularity Check

0 steps flagged

No circularity: recommendation derived from comparison of independent prior methods

full rationale

The paper evaluates established techniques for Poisson confidence intervals by comparing their performance on properties such as p-value behavior and consistency, drawing from prior statistical literature (e.g., Garwood intervals). No steps reduce claims to self-definitions, fitted inputs renamed as predictions, or load-bearing self-citations. The central recommendation rests on external benchmarks rather than internal construction, making the derivation self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard assumptions about the Poisson model and the appropriateness of the chosen evaluation criteria, with no free parameters, new entities, or ad-hoc inventions required.

axioms (1)
  • domain assumption The Poisson distribution is the correct model for the counting measurements under consideration.
    Invoked implicitly as the foundation for all interval constructions discussed in the abstract.

pith-pipeline@v0.9.0 · 5635 in / 1112 out tokens · 50239 ms · 2026-05-18T19:29:33.144147+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Poisson Empirical Bayes via Gamma-Smoothed Nonparametric Maximum Likelihood

    math.ST 2026-05 unverdicted novelty 7.0

    A Gamma-smoothed NPMLE for Poisson empirical Bayes achieves optimal nearly parametric rates for posterior means and enables asymptotically exact, shorter marginal coverage confidence sets under compact support.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    James and M

    F. James and M. Roos. Statistical notes on the problem of experimental observations near an unphysical region. Phys. Rev. D , 44:299–301, 1991

    work page 1991

  2. [2]

    F. T. Solmitz. Analysis of experiments in particle physics. Annual Review of Nuclear and Particle Science , 14:375–402, 1964

    work page 1964

  3. [3]

    J. Neyman. Outline of a theory of statistical estimation based on the classi- cal theory of probability. Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences , 236:333–380, 1937. 51

    work page 1937

  4. [4]

    Biller and Scott M

    Steven D. Biller and Scott M. Oser. Another look at confidence intervals: Proposal for a more relevant and transparent approach. Nuclear Instru- ments and Methods in Physics Research Section A , 774:103–119, 2015

    work page 2015

  5. [5]

    R.A. Fisher. Inverse probability. Proceedings of the Cambridge Philosoph- ical Society, 26:528–535

    work page

  6. [6]

    Pedersen

    J.G. Pedersen. Fiducial inference. International Statistical Review, 46:147– 170, 1978

    work page 1978

  7. [7]

    Y. H. Wang. Fiducial intervals: What are they? The American Statistician, 54(2):105–111, 2000

    work page 2000

  8. [8]

    Kendall and Alan Stuart

    Maurice G. Kendall and Alan Stuart. The Advanced Theory of Statistics, 3rd edition. Hafner Publishing Company, New York, 1973

    work page 1973

  9. [9]

    Narsky and F

    I. Narsky and F. C. Porter. Statistical analysis techniques in particle physics: Fits, density estimation and supervised learning . Wiley-VCH, Weinheim, Germany, 2014

    work page 2014

  10. [10]

    W. L. Stevens. Fiducial limits of the parameter of a discontinuous distri- bution. Biometrika, 37:117–129, 1950

    work page 1950

  11. [11]

    A method which eliminates the discreteness in Poisson confidence limits and lessens the effect of moving cuts specifically to elimi- nate candidate events

    Robert Cousins. A method which eliminates the discreteness in Poisson confidence limits and lessens the effect of moving cuts specifically to elimi- nate candidate events. Nucl. Instrum. Meth. A , 337:557–565, 1994

    work page 1994

  12. [12]

    Interval estimation using the likelihood function

    Frank Porter. Interval estimation using the likelihood function. Nucl. Instrum. Meth. A , 368:793–803, 1996

    work page 1996

  13. [13]

    Vos and Suzanne Hudson

    Paul W. Vos and Suzanne Hudson. Evaluation criteria for discrete confi- dence intervals: Beyond coverage and length. The American Statistician , 59:137–141, 2005

    work page 2005

  14. [14]

    Mathematical Statistics, 2nd edition

    Jun Shao. Mathematical Statistics, 2nd edition. Springer-Verlag, New York, 2003

    work page 2003

  15. [15]

    Brown, T

    Lawrence D. Brown, T. Tony Cai, and Anirban DasGupta. Interval esti- mation in exponential families. Statistica Sinica, 13:19–49, 2003

    work page 2003

  16. [16]

    Karim F. Hirji. Exact Analysis of Discrete Data . Chapman&Hall/CRC, 2006

    work page 2006

  17. [17]

    V. V. Patil and H. V. Kulkarni. Comparison of confidence intervals for the Poisson mean: Some new aspects. Revstat Statistical Journal, 10, 2012

    work page 2012

  18. [18]

    Exact confidence intervals and hypoth- esis tests for parameters of discrete distributions

    M˚ ans Thulin and Silvelyn Zwanzig. Exact confidence intervals and hypoth- esis tests for parameters of discrete distributions. Bernoulli, 28:459–502, 2017. 52

    work page 2017

  19. [19]

    F. Garwood. Fiducial limits for the Poisson distribution. Biometrika, 28:437–442, 1936

    work page 1936

  20. [20]

    Crow and Robert S

    Edwin L. Crow and Robert S. Gardner. Confidence intervals for the expec- tation of a Poisson variable. Biometrika, 46:441–453, 1959

    work page 1959

  21. [21]

    Refining poisson confidence intervals

    George Casella and Christian Robert. Refining poisson confidence intervals. The Canadian Journal of Statistics , 17:45–57, 1989

    work page 1989

  22. [22]

    Confidence curves and improved exact confidence intervals for discrete distributions

    Helge Blaker. Confidence curves and improved exact confidence intervals for discrete distributions. Canadian Journal of Statistics, 28:783–798, 2000

    work page 2000

  23. [23]

    Exact short poisson confidence intervals

    Paul Kabaila and John Byrne. Exact short poisson confidence intervals. The Canadian Journal of Statistics , 29:99–106, 2001

    work page 2001

  24. [24]

    Theodore E. Sterne. Remarks on confidence or fiducial limits. Biometrika, 41:275–278, 1954

    work page 1954

  25. [25]

    Somerville and Rebekkah S

    Matthew C. Somerville and Rebekkah S. Brown. Exact likelihood ratio and score confidence intervals for the binomial proportion. Pharmaceut. Statist., 12:120–128, 2013

    work page 2013

  26. [26]

    M. F. Schilling and B. Holladay. Length minimization for Poisson con- fidence procedures. Communications in Statistics—Theory and Methods , 46(2):861–873

    work page

  27. [27]

    Edwin L. Crow. Confidence intervals for a proportion. Biometrika, 43:423– 435, 1956

    work page 1956

  28. [28]

    Vos and Suzanne Hudson

    Paul W. Vos and Suzanne Hudson. Problems with binomial two-sided tests and the associated confidence intervals. Aust. N. Z. Stat. , 50:81–89, 2008

    work page 2008

  29. [29]

    Kullback, R

    Herman Chernoff. A property of some type A regions. Ann. Math. Stat., 22:472–474, 1951. https://projecteuclid.org/download/pdf 1/ euclid.aoms/1177729598

    work page arXiv 1951

  30. [30]

    K. Ulm. A simple method to calculate the confidence interval of a standardized mortality ratio (SMR). American Journal of Epidemiology , 131:373–375, 1990

    work page 1990

  31. [31]

    poisson.exact

    Michael Fay. poisson.exact. https://www.rdocumentation.org/packages/ exactci/versions/1.3-3/topics/poisson.exact, accessed May 1, 2020

    work page 2020

  32. [32]

    Navas and others (Particle Data Group)

    S. Navas and others (Particle Data Group). Review of particle properties. Phts. Rev. D , page 030001, 2024

    work page 2024

  33. [33]

    P. M. Kulkarni, R. C. Tripathi, and J. E. Michalek. Maximum (Max) and mid-P confidence intervals and p values for the standardized mortality and incidence ratios. American Journal of Epidemiology , 147(1):83–83, 1998. 53

    work page 1998

  34. [34]

    An invariant form for the prior probability in estimation problems

    Harold Jeffreys. An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. A , 186:453–461, 1946

    work page 1946

  35. [35]

    Alexander L. Read. Modified frequentist analysis of search results (The CL(s) method). In Workshop on Confidence Limits , pages 81–101, 8 2000

    work page 2000

  36. [36]

    A. L. Read. Presentation of search results: The CL(s) technique. Journal of Physics G: Nuclear and Particle Physics , 28(10):2693–2704, 2002

    work page 2002

  37. [37]

    G. Zech. Upper limits in experiments with background or measurement errors. Nucl. Instrum. Methods Phys. Res. A , 277:608–610, 1989

    work page 1989

  38. [38]

    X. Qian, A. Tan, J. J. Ling, Y. Nakajima, and C. Zhang. The Gaussian CLs method for searches of new physics. Nucl. Instrum. Meth. A , 827:63–78, 2016

    work page 2016

  39. [39]

    Feldman and Robert D

    Gary J. Feldman and Robert D. Cousins. A unified approach to the classical statistical analysis of small signals. Phys. Rev. D , 57:3873–3889, 1998

    work page 1998

  40. [40]

    Wendell and Josef Schmee

    John P. Wendell and Josef Schmee. Likelihood confidence intervals for proportions in finite populations. The American Statistician , 55:55–61, 2001. 54

    work page 2001