REVIEW 3 major objections 6 minor 17 references
Grouped data can be ungrouped: a mean-adjusted log-concave estimator recovers the underlying density from bin counts alone, with no bandwidth parameter to select.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
A bandwidth-free estimator for grouped data that recovers a log-concave density by first estimating the mean with a Gaussian EM, smoothing bin counts, jittering within bins, and fitting a log-concave MLE.
T0 review reviewed 2026-08-02 challenge →
load-bearing objection A solid, honest methods preprint: new construction and favorable simulations, but consistency only in the double limit and the fixed-grid gap to the true density is unquantified — worth a referee, needs revision. the 3 major comments →
Bandwidth-free nonparametric density estimation for grouped data
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The central claim is that the MALC estimator f̂_n (or its smoothed version f̂_smooth_n) converges almost surely to the true log-concave density f0 in L1 as n→∞ and bin width δ→0 (Corollary 10). For fixed δ, it converges to f̂0, the unique log-concave density closest in Kullback-Leibler divergence to Q_δ, the convolution of the discretely log-concave-smoothed bin probabilities and a beta perturbation whose mean is the Gaussian-EM estimate of the population mean. In addition, the paper shows that the Gaussian-EM mean estimate is unique, exists, and is within δ of the true mean on a uniform grid, so the mean recovery is accurate even when the Gaussian model is misspecified. The method requires
What carries the argument
The load-bearing construction is the mean-adjusted ungrouping step: a discrete log-concave smoothing of the bin proportions is placed at the bin midpoints, then convolved with a beta-distributed perturbation Z_n,δ on [0,δ) whose parameters are chosen so that the convolution's mean equals μ̂_n, the Gaussian-EM estimate of the true mean. Samples drawn from this convolution are fed into the log-concave maximum-likelihood estimator, whose KL-projection property (with a smoothing option) yields the final density. This machinery converts the inherently unidentifiable grouped-data problem into a well-posed log-concave approximation problem while preserving the most important first moment.
Load-bearing premise
The consistency proof establishes convergence to an internally defined target—the KL projection of a beta-smoothed version of the bin probabilities—not directly to the true density, so for a fixed, realistic grid the estimation error is governed by an unquantified double-limit gap.
What would settle it
Take a known log-concave distribution (e.g., Gamma(6,1)) on a fixed grid with δ/σ = 0.5, let n grow from 10^2 to 10^6, and record the L2 distance between the MALC estimate and the true density. If the error plateaus at a positive floor rather than decreasing to zero, then the double-limit consistency does not extend to fixed grids, confirming that the practical performance rests on the unquantified gap between f̂0 and f0.
If this is right
- Density estimation from binned data becomes fully automatic: no bandwidth, bin width, or parametric family needs to be chosen by the user.
- The estimator works well even when the true distribution is not log-concave (e.g., heavy-tailed, skew, Pareto, log-normal), especially for small to medium sample sizes.
- Because it is stable across a wide range of grid widths, MALC is usable when bins are coarse or when the grid was chosen externally without regard to the underlying density.
- The method yields smooth, non-decreasing hazard estimates, making it directly applicable in survival and mortality analysis where only age-group death counts are available.
- The error analysis shows that refining the bin width plus enlarging the sample drives the estimate to the true density, so the approach is principled rather than purely heuristic.
Where Pith is reading between the lines
- For a fixed real-world grid, the consistency result does not directly bound the error against the true density, since the limiting object f̂0 is the KL projection of a beta-smoothed version of the bin probabilities and generally does not reproduce the observed bin proportions; the practical gap is left unquantified and would be worth measuring empirically.
- The mean-recovery theorem suggests the Gaussian-EM step alone provides an accurate estimate of the population mean from grouped data, essentially within one bin width; this could be used as a standalone first-moment estimator or as a diagnostic for binning schemes.
- The beta perturbation is a designer's degree of freedom; an alternative perturbation whose shape depends on bin location might restore compatibility with the observed bin probabilities and is a natural direction to test, since the paper notes its current choice sacrifices compatibility for simplicity.
- Because the method currently assumes a uniform grid and univariate data, a concrete extension would be to non-uniform grids and multivariate grouped data, where the same mean-matching idea could generalize.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a mean-adjusted log-concave (MALC) density estimator for univariate grouped data. The algorithm first estimates the population mean by Gaussian EM on the grouped counts (Step 1), computes a discrete log-concave estimate of the bin probabilities (Step 2), and then constructs a continuous log-concave density estimate by sampling from the convolution of the discrete estimate with a beta perturbation on [0,δ) whose mean is forced to equal the EM mean (Step 3). The theoretical core consists of mean-recovery theorems (Theorems 4–5), convergence of the estimator to the KL projection f̂0 of the limiting convolution Qδ at fixed δ (Theorem 9), and a consistency claim in the double limit n→∞, δ→0 (Corollary 10). Simulation studies compare MALC with three kernel-based competitors across ten distributions, and the method is applied to human mortality data. The paper also describes an R package named MALC.
Significance. If the statistical claims held in the form stated, the paper would contribute a fully automatic, bandwidth-free nonparametric estimator for grouped data, a practically important problem. The simulation study is broad and includes non-log-concave targets, and the R package and explicit comparison implementations are valuable for reproducibility. The paper also deserves credit for Remark 11, which candidly states that the fixed-δ limit f̂0 is generally not compatible with the observed bin probabilities and that no bound on the difference between f̂0 and f0 is provided. That concession, however, points to the main weakness: the headline consistency result is only a double-limit statement, and the practical fixed-grid claim is left unquantified. Because of this gap, and because several implementation choices (σ, α, B) are not fully automatic despite the paper's language, the contribution is not yet at the level claimed. The method may well be useful, but the central consistency claim needs either strengthening with finite-δ bounds or careful reframing.
major comments (3)
- [§3.2, Corollary 10 and Remark 11] The only asymptotic guarantee for a fixed grid is Theorem 9, which sends n→∞ and states that f̂n converges to the log-concave KL projection f̂0 of Qδ. Remark 11 concedes that this target is generally not compatible with the observed bin probabilities: ∫_{a_j}^{a_{j+1}} f̂0 dx need not equal p0,j. Corollary 10 then sends δ→0 and asserts limsup_δ limsup_n ∫|f*_n − f0| = 0. For any real grouped-data problem, δ is fixed and generally not shrinking; Lemma 2 already establishes non-identifiability at fixed δ, so some bias is unavoidable. But the paper gives no bound on ‖f̂0 − f0‖ or on the compatibility gap as a function of δ, and the simulations do not probe the double limit: the grid-width design fixes n=10^3, while the large-n design fixes δ/σ=0.5. Thus the abstract and Section 4.1.1 claim that MALC estimates the underlying density from grouped data rests on an unquantified double-limit gap
- [§3, Steps 1 and 3] The paper repeatedly describes MALC as fully automatic and requiring no user-specified inputs, including in Section 4.1.1. However, the algorithm requires choices of σ in Step 1 (only constrained by δ<2σ), α in Step 3 (the shape of the beta perturbation), and B in Step 3(a) (the number of pseudo-samples). Figure 1 shows the effect of α in one example, but there is no sensitivity analysis, selection rule, or asymptotic guidance for these choices. In particular, unless B→∞, the estimator is random given the grouped counts, and the paper does not specify how large B must be for the stated mean-preservation properties to hold. This does not invalidate the method, but it qualifies the 'bandwidth-free/no-tuning parameter' claim.
- [§3.2, after Eq. (5)] The formula for the beta perturbation appears inconsistent with the stated mean-calibration. If Z is beta(α−β, α+β), then E[δZ] = δ(α−β)/(2α). Equating this to M = μ̂0 − Σ_j a_j p0,j gives β = α(1 − 2M/δ), not β = α(δ/M − 1). As written, the displayed expression does not yield the claimed mean, so condition (5) and the existence of a valid perturbation are not established. If this is a typographical error, it should be corrected; otherwise Theorem 9's construction of Qδ is unsupported.
minor comments (6)
- [§2.1, Lemma 8] The display after Lemma 8 reads 'maximizes Σ_j φ_{0,j} p0,j − Σ_j e^{φ_{0,j}} p0,j+1'. The second term should presumably be Σ_j e^{φ_{0,j}} (or the analogue appropriate for the discrete log-concave likelihood); as written, the objective depends on p0,j+1 in a way not consistent with the cited discrete log-concave MLE and would not imply the stated mean-preservation property.
- [§2.2, Theorem 5] The claim that, for a fixed uniform partition, |μ0 − μ̂n| ≤ δ for all n ≥ N(ω) and also |μ0 − μ̂0| ≤ δ is strong. The proof is deferred to the supplementary material, which was not part of the review package. Since this theorem underpins Step 1 of the algorithm, please ensure the proof is fully available and check whether the statement should be |μ0 − μ̂0| ≤ δ or |μ0 − μ̂0| ≤ Cδ with a stated constant.
- [§3, Step 3 and §3.2] The paper notes that the log-concave MLE preserves the empirical mean and that Step 3 is designed so that the convolution has mean μ̂n. This means part of the mean-recovery behavior is baked into the construction. That is not a logical flaw, but it should be stated explicitly so that the reader does not interpret Theorem 5 as independently validating the final estimator's location.
- [§4.1, Figures 2–6] The grid-width figures plot the x-axis from 2.5 down to 0.5, while the text describes increasing δ/σ. This makes the figures harder to read; consider reversing the axis. Also, Figure 4's legend uses 'Dens−OLog' rather than 'MALC', which is inconsistent with the other figures.
- [§2, Step 1] The Gaussian variance σ² is introduced as known, but Step 1 only requires choosing σ such that δ<2σ. If σ is actually known and small relative to δ, the condition may fail; if σ is a free parameter, the 'known variance' language should be revised.
- [§5] There is a typo: 'an log-concave density estimator' should be 'a log-concave density estimator'. Similar minor wording issues appear elsewhere (e.g., 'grid with is large').
Circularity Check
No significant circularity: the mean-matching step is an explicit construction, not a disguised prediction, and the consistency proof's internal target is disclosed as such.
full rationale
The paper's derivation chain is not circular. The mean adjustment is openly constructed: Step 3 defines Z_{n,δ} so that X_{n,δ} has mean ̂μ_n, and the log-concave MLE then inherits that mean. This is not a fitted input being relabeled as a prediction, because the consistency of ̂μ_n for μ0 is established independently in Theorems 4–5 through the Gaussian EM score equation, without using the MALC construction. The paper never claims that MALC 'predicts' the mean; it uses the EM mean as an input. The limiting target ƒ̂0 is indeed an internal construction (the log-concave KL projection of Qδ), and Remark 11 openly concedes that it need not reproduce the true bin probabilities for fixed δ and that no δ-quantified bound on ‖ƒ̂0−f0‖ is provided. That is a substantive limitation about the double-limit gap and practical fixed-grid relevance, but it is not circularity: the proof of consistency as n→∞ and δ→0 is a continuous-projection argument, not a restatement of an assumption. The self-citations to Balabdaoui et al. (2013) supply external, previously published theorems on discrete log-concave MLE existence, uniqueness, mean preservation, and consistency; they are load-bearing but independent mathematical results, not unverified restatements of this paper's conclusions. Accordingly, no circular step can be exhibited, and the appropriate score is 0.
Axiom & Free-Parameter Ledger
free parameters (3)
- σ, working variance in the Gaussian EM mean-recovery step
- α, shape parameter of the beta perturbation Z_n,δ
- B, number of pseudo-samples generated in Step 3(b)
axioms (7)
- domain assumption The true density f0 is log-concave.
- domain assumption The bin grid is uniform: a_{j+1} − a_j = δ for all j.
- standard math Binning a log-concave density on a uniform grid yields a discrete log-concave pmf (Lemma 1).
- standard math The discrete log-concave MLE exists, is unique, and preserves the empirical mean.
- standard math The continuous log-concave MLE exists, is unique, and is consistent; log-concave projections are continuous in Mallows distance.
- ad hoc to paper The perturbation variables Z_n,δ converge in Mallows distance to Zδ satisfying condition (5).
- ad hoc to paper For the beta perturbation, β = α(δ/(μ̂0 − ∑ a_j p0,j) − 1) yields a valid distribution with |β| < α.
invented entities (1)
-
Z_n,δ (beta-distributed within-bin perturbation)
no independent evidence
Cite this review
Pith. "Pith review of Bandwidth-free nonparametric density estimation for grouped data." pith.science (2026). https://pith.science/paper/LERYDEFF
@misc{pith2026260713182,
author = {Pith},
title = {Pith review of: Bandwidth-free nonparametric density estimation for grouped data},
year = {2026},
howpublished = {\url{https://pith.science/paper/LERYDEFF}},
note = {Machine review of arXiv:2607.13182}
}
read the original abstract
In some situations, data is collected under systematical and technical constraints due to uncertainty in experimental reports, intermittent measurements, confidentiality, and non-detects. For this reason, it might not be possible to retrieve or receive the data in a conventional format but rather in a grouped form where only the number of occurrences is known within intervals. The challenge is to estimate the density of the underlying ungrouped data based on the observed grouped data with no information regarding the underlying distribution. To overcome this problem, this study introduces a mean-adjusted log-concave (MALC) density estimation method for univariate grouped data, aiming to provide a bandwidth-free non-parametric approach that does not rely on specific distributional assumptions. The performance of the MALC method is evaluated through simulations across various distributions with different sample sizes and grid widths. The results demonstrate the robustness and effectiveness of the MALC approach in grouped data analysis, offering a broader range of applications over traditional methods.
Figures
Reference graph
Works this paper leans on
-
[10]
Revista de la Facultad de Ciencias 4, 74–82
Bootstrap-based inference for grouped data. Revista de la Facultad de Ciencias 4, 74–82. doi:10.15446/rev.fac. cienc.v4n2.54254. Victoria-Feser, M., Ronchetti, E.,
-
[12]
Inference and modeling with log-concave distributions. Statistical Science , 319–327. Wand,M.,Moler,C., Ripley,B.D.,2025. KernSmooth:Functions forKernelSmoothingSupportingWand&Jones (1995). doi:10.32614/CRAN. package.KernSmooth. r package version 2.23-26. Wand, M.P., Jones, M.C.,
doi:10.32614/cran 2025
-
[17]
The purple curve represents females and the red curve represents males. Danisman, Jankowski, and de Souza:PreprintPage 19 of 11 Bandwidth-free density estimation for grouped data 0 20 40 60 80 100 0.00 0.01 0.02 0.03 0.04 Age Norway (79) Japan (80) Australia (77) Canada (76) USA (74) Republic of Korea (73) Figure 9:Estimated mortality densities for the to...
2010
-
[1956]
The Annals of Mathematical Statistics 27, 832–837
Remarks on some nonparametric estimates of a density function. The Annals of Mathematical Statistics 27, 832–837. Rubio,F.J.,Steel,M.F.,2011. Inferenceforgroupeddatawithatruncatedskew-Laplacedistribution. ComputationalStatistics&DataAnalysis55, 3218–3231. Samworth, R.J.,
2011
-
[1967]
Approximate maximum likelihood estimates from grouped data. Technometrics 9, 599–606. doi:10.2307/1266198. Teimouri, M.,
-
[1995]
Kernel Smoothing. Chapman & Hall/CRC. doi:10.1201/b14876. Wieringa, J., et al.,
-
[1997]
Journal of the American Statistical Association 92, 333–340
Robust estimation for grouped data. Journal of the American Statistical Association 92, 333–340. doi:10.1080/01621459.1997.10473631. Walther, G.,
arXiv 1997
-
[2002]
Non-destructive testing of aerospace structures: granularity and data mining approach, in: Proceedings of FUZZ-IEEE 2002, IEEE, Honolulu, Hawaii. pp. 685–689. Parzen, E.,
2002
-
[2007]
University of Pretoria
Statistical Analysis of Grouped Data. University of Pretoria. Cule,M.L.,Samworth,R.J.,Stewart,M.I.,2008. Maximumlikelihoodestimationofamulti-dimensionallog-concavedensity. JournaloftheRoyal Statistical Society: Series B (Statistical Methodology)
2008
-
[2009]
The Annals of Statistics 37, 1299–1331
Limit distribution theory for maximum likelihood estimation of a log-concave density. The Annals of Statistics 37, 1299–1331. BarreiroUres,D.,Fraguela,B.B.,DoalloBiempica,R.,Cao,R.,Francisco-Fernández,M.,Reyes,M.,2019a. binnednp:NonparametricEstimation for Interval-Grouped Data. CRAN. R package version 0.4.0. Barreiro Ures, D., Francisco-Fernández, M., Ca...
-
[2010]
Statistical Methods in Medical Research 19, 53–70
Interval censoring. Statistical Methods in Medical Research 19, 53–70. doi:10.1177/0962280209105023. Supplementary Material.The alternative methods, proofs of all theoretical results, and a review of competing nonparametric methods are provided in the Supplementary Material. Danisman, Jankowski, and de Souza:PreprintPage 11 of 11 Bandwidth-free density es...
-
[2011]
Active set and EM algorithms for log-concave densities based on complete and censored data. arXiv:0707.4643. Dümbgen, L., Rufibach, K.,
-
[2013]
Journal of the Royal Statistical Society Series B: Statistical Methodology 75, 769–790
Asymptotics of the Discrete Log-Concave Maximum Likelihood Estimator and Related Applications. Journal of the Royal Statistical Society Series B: Statistical Methodology 75, 769–790. doi:10.1111/rssb.12011, arXiv:https://academic.oup.com/jrsssb/article-pdf/75/4/769/49507094/jrsssb_75_4_769.pdf. Balabdaoui, F., Rufibach, K., Wellner, J.,
-
[2015]
American Journal of Epidemiology 182, 138–147
Efficient estimation of smooth distributions from coarsely grouped data. American Journal of Epidemiology 182, 138–147. doi:10.1093/aje/kwv020. Rosenblatt, M.,
-
[2020]
The purple curve represents females and the red curve represents males. Danisman, Jankowski, and de Souza:PreprintPage 18 of 11 Bandwidth-free density estimation for grouped data 0 20 40 60 80 100 0.0 0.5 1.0 1.5 2.0 2.5 (a)Canada 0 20 40 60 80 100 0.0 0.5 1.0 1.5 (b)USA 0 20 40 60 80 100 0 1 2 3 4 (c)Japan 0 20 40 60 80 100 0 1 2 3 4 5 (d)Republic of Kor...
2010
-
[2021]
Journal of Business Research 122, 915–925
Data analytics in a privacy-concerned world. Journal of Business Research 122, 915–925. doi:10.1016/j.jbusres. 2019.05.005. Xiao,X.,Mukherjee,A.,Xie,M.,2016. Estimationproceduresforgroupeddata–acomparativestudy. JournalofAppliedStatistics43,2110–2130. doi:10.1080/02664763.2015.1130801. Zhang, Z., Sun, J.,
arXiv 2019
-
[2024]
Journal of Econometrics 238, 105568
Tuning parameter-free nonparametric density estimation from tabulated summary data. Journal of Econometrics 238, 105568. Danisman, Jankowski, and de Souza:PreprintPage 10 of 11 Bandwidth-free density estimation for grouped data Lindsey,J.,Ryan,L.,1998. Methodsforinterval-censoreddata. StatisticsinMedicine17,219–238. doi:10.1002/(SICI)1097-0258(19980130) 1...
This paper was first reviewed by deepseek-v4-flash on August 2, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.