How to detect a biased sample using the Renyi divergence measure?

Rajesh G; Vaishnavi Pavithradas

REVIEW 4 major objections 5 minor 14 references

A Rényi-divergence statistic γ detects size-biased samples in complete and type-I censored data, with simulated Weibull critical values and asymptotic normality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 14:19 UTC pith:XSLA657W

load-bearing objection Rényi generalization of the KL bias test is worth taking seriously, but the real-data examples are internally inconsistent and the asymptotic variance is not derived correctly. the 4 major comments →

arxiv 2607.18790 v1 pith:XSLA657W submitted 2026-07-21 stat.OT

How to detect a biased sample using the Renyi divergence measure?

Vaishnavi Pavithradas , Rajesh G This is my paper

classification stat.OT MSC 94A1762G07

keywords Rényi divergencesize-biased samplinglength biasweighted distributionstype-I censoringWeibull distributionhypothesis testingasymptotic normality

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a statistical test for detecting size-biased (especially length-biased) sampling, a distortion that arises when observations are selected with probability proportional to their size. The test statistic γ is the ratio of a power mean of order 1−α to the arithmetic mean of the weight function, derived from the empirical Rényi divergence. The authors show γ is asymptotically normal, scale-invariant, and that its null distribution under the Weibull model depends only on the ratio η/r and the censoring rate q, enabling tabulated critical values. They demonstrate the test on two real datasets: it rejects the null for a known length-biased aircraft failure-time sample and does not reject for an electronic-component sample, supporting its use as a diagnostic before reliability modeling.

Core claim

The central claim is that γ = [ (1/n Σ x*^{r(1−α)}) ]^{1/(1−α)} / ( (1/n) Σ x*^r ) — the ratio of a power mean of order 1−α to the arithmetic mean of the size-bias weight x*^r — provides a valid test statistic for H0: r = 0 versus H1: r = r0, where r0 is typically 1 (length bias) or 2 (area bias). Under the null, γ converges to a normal distribution (Theorem 1, via the delta method and Slutsky's theorem). For Weibull-distributed data, the test's null distribution depends only on the ratio η/r and the censoring proportion q (Proposition 3), so critical values can be pre-computed. In simulated power comparisons, γ performs competitively with the Kullback–Leibler and likelihood-ratio tests, and

What carries the argument

The statistic γ is the empirical Rényi divergence between the underlying density f and its size-biased version f_r, expressed as a ratio of a power mean of order (1−α) of the weight w(x)=x^r to the arithmetic mean of w(x). When α→1 it reduces to the geometric-mean/arithmetic-mean ratio used in the Kullback–Leibler test. The key structural property is Proposition 3: under a scale-family Weibull distribution, both moments E[X*^{r(1−α)}] and E[X*^r] depend only on the ratio η/r and the censoring rate q, making the null distribution independent of the scale parameter and free of the unknown mean. This allows Monte Carlo critical values to be tabulated once for a range of n, η/r, and q.

Load-bearing premise

The critical values are tabulated for a Weibull distribution with a known shape parameter η; in the real-data examples η is estimated from the same sample and then treated as known, so if η is misspecified the test's true rejection rate under the null can drift away from the nominal 5%.

What would settle it

Simulate samples from a Weibull distribution with a known shape η (e.g., η=1.5) under the null hypothesis (r=0), then apply the test using critical values from Tables 1–4 for a different η (e.g., η=1.0). If the empirical rejection rate over many replications departs substantially from 0.05, the claim that the test controls the level when η is unknown or estimated fails. Equivalently, one could compute the power-mean/arithmetic-mean ratio γ under the null for a broad class of distributions and check whether its distribution depends only on η/r as Proposition 3 asserts.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Practitioners can use the tabulated critical values to test for length or size bias in complete and type-I censored reliability samples without further simulation, provided a Weibull model is plausible.
The test remains valid when data are censored at a fixed time x0, a common situation in survival and reliability analysis, and its power is comparable to or better than existing divergence-based tests under censoring.
The asymptotic normality of γ means that for large samples, approximate p-values can be computed analytically rather than relying solely on tables.
Because γ is scale-invariant, the test does not require knowing the scale parameter of the underlying distribution, simplifying its use in practice.
The test is computationally simple, requiring only two sample moments of the weight function, so it can be applied to very large datasets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The tabulated critical values presuppose a known Weibull shape parameter η. In the two real-data applications, η is estimated from the same sample (η̂=1.118 and η̂=0.8) and then the critical values are read as if η were known; this ignores estimation uncertainty, so the actual size of the test may differ from the nominal 5% unless the estimator is very precise or the test is calibrated for estimat
The fact that γ is a ratio of a power mean to an arithmetic mean suggests that the test is essentially checking a moment-ratio condition; for a given dataset, the choice of α (the paper selects α=0.5 after a power experiment) could be tuned to the specific alternative, and a bootstrap calibration might yield a more robust version that avoids the η-dependence altogether.
The methodology should extend naturally to other parametric families that satisfy a property analogous to Proposition 3 (where the divergence depends only on a shape ratio and censoring rate), or to semiparametric settings by replacing the Weibull assumption with bootstrap critical values.
A natural next step is to develop an analogous test using the empirical Rényi divergence for progressively censored or truncated samples, where the censoring mechanism is more complex than the single-point type-I censoring treated here.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Rényi generalization of the KL bias test is worth taking seriously, but the real-data examples are internally inconsistent and the asymptotic variance is not derived correctly.

read the letter

Two things first. The test itself is a legitimate extension: replacing the KL ratio in Economou and Tzavelas with the power-mean/arithmetic-mean ratio gamma for Rényi order alpha, with simulated Weibull critical values for complete and type-I censored data, is new and potentially useful. The KL test is indeed the alpha-to-1 limit (eq. 12), and the censored tables are a real resource. But the real-data section is not trustworthy. For Dataset 2, the reported acceptance region (0.223, 0.532) is the eta=0.4 row from Tables 1-2, not the eta=0.8 row implied by the fitted shape; the reported gamma=0.424 is numerically consistent with the KL statistic for Weibull(0.8), not with the alpha=0.5 Rényi statistic (which would be roughly 0.71). Using the paper's own eta=0.8 critical values (about 0.60, 0.79), the test rejects H0, reversing the stated conclusion. Dataset 1 also appears to use the uncensored table despite 18/69 (about 26%) censoring. So the practical-utility claim is unsupported as written.

What the paper does well: the divergence algebra in (5)-(7) checks out; Proposition 3's dependence on eta/r and censoring rate is correct and useful; the alpha=0.5 choice is at least fixed before the applications; and the power tables suggest the Rényi version can outperform KL under censoring, which is the most interesting empirical claim. That claim is weakened by the fact that lambda1 and LR powers are taken from the earlier paper rather than from matched simulations, but the direction is plausible.

Soft spots: Theorem 1's variance is not proved. The denominator is a random sample mean, and the delta method for the ratio requires its variance and the covariance with the numerator; the formula given only accounts for the numerator. Since the critical values are simulated, this doesn't invalidate the tables, but the asymptotic result needs repair. More serious for practice: the critical values depend on eta, and in both applications eta is estimated from the same sample and then treated as known, with no adjustment or sensitivity analysis. With a misspecified eta the actual level can differ from 5%. The authors also select alpha=0.5 after looking at power simulations; that's acceptable if stated as calibration, but it should be presented as such.

Who is this for? People doing quick pre-modeling bias checks in reliability and survival analysis. The core idea deserves a referee; the tables and the censoring extension could be cited once cleaned up. I'd send it to review, with a clear request to redo the data analyses, fix Theorem 1, and address the eta-estimation issue.

Referee Report

4 major / 5 minor

Summary. This paper proposes a test for size bias (H0: r=0 vs H1: r=r0) based on an empirical Rényi divergence. For a possibly type-I censored sample and weight w0(x)=x^{r0}, the test statistic is γ = [n^{-1}Σ x_i^{* r0(1-α)}]^{1/(1-α)} / [n^{-1}Σ x_i^{* r0}], with rejection when γ falls outside simulated Weibull critical values. The paper derives properties (scale invariance, monotonicity, dependence of the null distribution on η/r and censoring rate q), claims asymptotic normality, provides extensive Monte Carlo critical value tables for uncensored and censored cases, compares power with KL-divergence and likelihood-ratio tests, and applies the test to two real datasets.

Significance. If the statistic performed as claimed, the paper would offer a useful generalization of the KL-divergence test, particularly for type-I censored reliability data. The main strengths are the simple closed-form statistic, the clear tabulation of critical values, and Proposition 3's reduction of the null distribution to η/r and q. The reported censored-case power values are favorable. However, the asymptotic variance derivation is not valid as written, and the two real-data examples are internally inconsistent with the paper's own tables, so the central claims are not yet established.

major comments (4)

[Section 3.1, Theorem 1, Eqs. (15)–(18)] The derivation of the asymptotic variance in Theorem 1 is not valid. In Eq. (17) the limiting variance still contains the random denominator (1/n)Σ w(x_i^*), and Eq. (18) replaces it by E[w(X^*)] without justification. Since γ is a ratio of two correlated sample means, the correct bivariate delta method also contributes the fluctuation of the denominator and the covariance between the two means. The stated variance therefore omits a term of the same n^{-1/2} order. The asymptotic normality claim may still be true, but the variance formula as stated is unsupported.
[Section 5, Dataset 2] The example is internally inconsistent. With η̂=0.8, n≈50, r0=1, q=0 and α=0.5, Tables 1–2 give an approximate 5% acceptance region of (0.60,0.79) (interpolating between n=40 and n=60), not (0.223,0.532). The printed region is close to the η=0.4 row. In addition, γ=0.424 is close to the KL geometric/arithmetic mean ratio for a Weibull(0.8) sample (≈0.43), whereas the α=0.5 Rényi statistic has probability limit Γ(1.625)^2/Γ(2.25)≈0.71. Thus the example uses either the wrong critical row or the wrong statistic. Under the paper's own rule with η=0.8, γ=0.424 lies below the lower critical value, so H0 is rejected, reversing the reported conclusion.
[Section 5, Dataset 1] With 18/69≈26% censored observations, the stated acceptance region (0.738,0.868) does not match Tables 3–4 for q=0.2 or q=0.4 at η≈1.1 (e.g., q=0.2, n=60 gives roughly (0.803,0.899)); it resembles the uncensored tables. The authors should state which censoring proportion and table row are used and recompute the interval. As it stands, the claimed correct rejection of H0 for a known length-biased sample is not demonstrated.
[Section 5 and Proposition 3] Critical values depend on the Weibull shape η (through η/r), but in both applications η is estimated from the same data (η̂=1.118 and η̂=0.8) and the tables are used as if η were known. The effect on the test's size is not assessed. A sensitivity analysis over plausible η values, or an explicit statement of the limitation, is needed before the real-data decisions can be taken at the stated 5% level.

minor comments (5)

[Section 3.1, Eq. (12)] Since D^E_{α,c} = -ln γ, the sentence 'equivalently, the test rejects H0 when γ > u(αc,q,n)' uses the same symbol u for the critical value of D and for the critical value of γ. Use different notation, e.g., v = exp(-u).
[Section 4.1] The choice α=0.5 is justified by Figure 1, but the figure or the supporting power values are not described in the text; report the numerical comparison or include the figure with enough detail.
[Section 4.2, Eq. (20)] The LRT formula is incomplete: Λ is written with E[w(X)] but the parameters θ̂_w and θ̂ mentioned in the text do not appear. Clarify how the expectation is estimated under H0 and H1.
[Tables 5–7] The column headers 'H0:r=1' and 'H0:r=2' should read 'H1:r=1' and 'H1:r=2', since powers are computed under the alternative.
[Throughout] Typographical errors: 'annd' (p.2), 'provideed' (p.2), 'lower and critical values' (p.9, missing 'upper').

Circularity Check

0 steps flagged

No circularity found: the Rényi statistic, its asymptotic normality, and the simulated critical values are derived from stated definitions and independent simulations; the real-data inconsistencies are correctness issues, not circular steps.

full rationale

The paper defines D_α(f||f_w) from Rényi (1961), derives the empirical form as -ln(power mean / arithmetic mean), and proposes γ as the ratio; Theorem 1 follows from the CLT, delta method and Slutsky, without assuming the conclusion. Proposition 3 shows dependence of the divergence on η/r and q, which is then used only to justify the simulation design. Critical values are Monte Carlo quantiles under the Weibull null, and power comparisons are against the external KL and LR tests of Economou and Tzavelas (2014), who are not the present authors; there is no load-bearing self-citation and no imported uniqueness theorem. The choice α=0.5 is a simulation-based tuning choice, and η̂ is estimated from the real data before applying the test; these affect the nominal level and the strength of the real-data demonstration, but they do not make the claimed detection result equal to a fitted value by construction. The reported acceptance regions for the two datasets do not match the stated η and censoring (e.g., Dataset 2's (0.223,0.532) corresponds to η=0.4 rather than the fitted η=0.8), but that is an internal consistency/correctness problem, not circularity. No step in the derivation chain reduces a first-principles result to an input definition.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The core Rényi-divergence formula is derived in closed form with no fitted constants; but the practical test relies on a hand-tuned order α=0.5, a significance split p=0.90 taken from prior work, and Weibull shape parameters estimated from the data used to evaluate the test. No invented entities are introduced.

free parameters (3)

Rényi order α = 0.5
Selected because the power curve in Figure 1 peaks at α=0.5; power tables are then reported at this value without accounting for selection.
Significance split p = 0.90
p=0.90 sets α1=0.9αc and α2=0.1αc; adopted from Economou and Tzavelas (2014) rather than derived in this paper.
Weibull shape η in applications = 1.118 (Dataset 1), 0.8 (Dataset 2)
Critical values depend on η; in Section 5 η is estimated from the same samples and used without adjusting the critical values for estimation error.

axioms (4)

domain assumption Weighted distribution model f_w = w f / E[w] with positive weight function w
Standard Rao-Fisher model used throughout; it defines what 'bias' means in the test.
domain assumption Underlying population is Weibull(β,η) for critical values
Critical values and power simulations are generated only under Weibull; Proposition 3 requires the location-scale form F((x/β)^η,1,1).
domain assumption Type-I censoring with fixed threshold x0, X*=min(X,x0)
Censored critical values use this model; other censoring schemes are not covered.
standard math CLT, delta method, and Slutsky's theorem
Invoked in Theorem 1; note the theorem's variance omits the covariance between the two sample moments, so the theorem is not actually established as stated.

pith-pipeline@v1.3.0-alltime-deepseek · 14192 in / 23595 out tokens · 235479 ms · 2026-08-01T14:19:29.604013+00:00 · methodology

0 comments

read the original abstract

Weighted distributions arise in situations where observations are selected with unequal probabilities or because of the non-observability of some events. Detecting such sampling bias is essential for ensuring valid statistical inference. In this paper, we propose a statistical test for detecting bias in a sample using the Renyi divergence measure. The proposed test statistic is formulated for both uncensored and type-I censored data and possesses several theoretical properties including asymptotic normality. Critical values are obtained through simulations under the Weibull distribution for a range of sample sizes and censoring proportions. A comprehensive power study compares the proposed test with the existing Kullback-Leibler divergence-based test and the likelihood ratio test. Two real datasets are analyzed to demonstrate the practical utility of the proposed test in detecting length bias.

discussion (0)

Reference graph

Works this paper leans on

14 extracted references

[1]

Annals of Eugenics , volume=

The effect of methods of ascertainment upon the estimation of frequencies , author=. Annals of Eugenics , volume=. 1934 , publisher=

1934
[2]

On discrete distributions arising out of methods of ascertainment , author=. Sankhy. 1965 , publisher=

1965
[3]

On size-biased sampling and related form-invariant weighted distributions , author=. Sankhy. 1976 , publisher=

1976
[4]

Statistics: Applications and New Directions , pages=

Studies in statistical ecology involving weighted distributions , author=. Statistics: Applications and New Directions , pages=. 1984 , publisher=

1984
[5]

Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics , volume=

On measures of entropy and information , author=. Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics , volume=. 1961 , organization=

1961
[6]

The annals of mathematical statistics , volume=

On information and sufficiency , author=. The annals of mathematical statistics , volume=. 1951 , publisher=

1951
[7]

The Bell system technical journal , volume=

A mathematical theory of communication , author=. The Bell system technical journal , volume=. 1948 , publisher=

1948
[8]

Communications in Statistics-Simulation and Computation , volume=

Exponentiality test based on Renyi distance between equilibrium distributions , author=. Communications in Statistics-Simulation and Computation , volume=. 2018 , publisher=

2018
[9]

Statistical Methodology , volume=

Kullback--Leibler divergence measure based tests concerning the biasness in a sample , author=. Statistical Methodology , volume=. 2014 , publisher=

2014
[10]

2004 , publisher=

Weibull models , author=. 2004 , publisher=

2004
[11]

Journal of Applied Statistics , volume=

Fitting parametric frailty and mixture models under biased sampling , author=. Journal of Applied Statistics , volume=. 2009 , publisher=

2009
[12]

4 , author=

Weighted distributions,Encyclopedia of Environmetrics, Vol. 4 , author=. 2002 , publisher=

2002
[13]

Biometrical Journal: Journal of Mathematical Methods in Biosciences , volume=

How to detect biased samples? , author=. Biometrical Journal: Journal of Mathematical Methods in Biosciences , volume=. 2003 , publisher=

2003
[14]

JP Journal of Biostatistics , volume=

A simple test for detection of length-biased sampling , author=. JP Journal of Biostatistics , volume=

[1] [1]

Annals of Eugenics , volume=

The effect of methods of ascertainment upon the estimation of frequencies , author=. Annals of Eugenics , volume=. 1934 , publisher=

1934

[2] [2]

On discrete distributions arising out of methods of ascertainment , author=. Sankhy. 1965 , publisher=

1965

[3] [3]

On size-biased sampling and related form-invariant weighted distributions , author=. Sankhy. 1976 , publisher=

1976

[4] [4]

Statistics: Applications and New Directions , pages=

Studies in statistical ecology involving weighted distributions , author=. Statistics: Applications and New Directions , pages=. 1984 , publisher=

1984

[5] [5]

Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics , volume=

On measures of entropy and information , author=. Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics , volume=. 1961 , organization=

1961

[6] [6]

The annals of mathematical statistics , volume=

On information and sufficiency , author=. The annals of mathematical statistics , volume=. 1951 , publisher=

1951

[7] [7]

The Bell system technical journal , volume=

A mathematical theory of communication , author=. The Bell system technical journal , volume=. 1948 , publisher=

1948

[8] [8]

Communications in Statistics-Simulation and Computation , volume=

Exponentiality test based on Renyi distance between equilibrium distributions , author=. Communications in Statistics-Simulation and Computation , volume=. 2018 , publisher=

2018

[9] [9]

Statistical Methodology , volume=

Kullback--Leibler divergence measure based tests concerning the biasness in a sample , author=. Statistical Methodology , volume=. 2014 , publisher=

2014

[10] [10]

2004 , publisher=

Weibull models , author=. 2004 , publisher=

2004

[11] [11]

Journal of Applied Statistics , volume=

Fitting parametric frailty and mixture models under biased sampling , author=. Journal of Applied Statistics , volume=. 2009 , publisher=

2009

[12] [12]

4 , author=

Weighted distributions,Encyclopedia of Environmetrics, Vol. 4 , author=. 2002 , publisher=

2002

[13] [13]

Biometrical Journal: Journal of Mathematical Methods in Biosciences , volume=

How to detect biased samples? , author=. Biometrical Journal: Journal of Mathematical Methods in Biosciences , volume=. 2003 , publisher=

2003

[14] [14]

JP Journal of Biostatistics , volume=

A simple test for detection of length-biased sampling , author=. JP Journal of Biostatistics , volume=