Testing equivalence to binary generalized linear models with application to logistic regression

Vladimir Ostrovski

REVIEW 4 major objections 5 minor 14 references

This paper proposes an equivalence test for binary generalized linear models with categorical covariates: it rejects the null that the model is at least ε away from the data when a minimum-distance statistic falls below a normal quantile.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A minimum-distance equivalence test for binary generalized linear models with categorical covariates, with asymptotic and bootstrap critical values, is developed and evaluated on real and simulated data.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection A genuinely new equivalence test for binary GLMs with categorical covariates, but Proposition 5 rests on an unproved, self-cited differentiability condition that must be resolved before the method can be trusted. the 4 major comments →

arxiv 2607.14724 v1 pith:MJIZ7T2H submitted 2026-07-16 stat.ME

Testing equivalence to binary generalized linear models with application to logistic regression

Vladimir Ostrovski This is my paper

classification stat.ME MSC 62F0362F0562J12

keywords equivalence testingbinary GLMlogistic regressionminimum distance estimationcategorical covariatesbootstrap testasymptotic normalitygoodness of fit

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces an equivalence test for binary generalized linear models, aimed at showing that observed data agree sufficiently well with the model rather than merely failing to show lack of fit. With categorical covariates, the data are summarized by cell frequencies, and the test statistic is the scaled difference between the squared minimum Euclidean distance from the probability vector to the model family and the tolerance ε². The central result is that this statistic is asymptotically normal with mean zero and an explicit variance, so critical values can be obtained either from the normal approximation or from the bootstrap. If correct, practitioners gain a way to affirmatively conclude that a logistic or other binary GLM is close enough to the truth, and the simulations point to the bootstrap-variance version as the one that controls the error rate on the boundary.

Core claim

The load-bearing result is Proposition 5: for a fixed probability vector p₀ lying on the boundary d(p₀, M) = ε, and assuming a continuous local minimizer h(p) exists, the statistic T(p_n) = √n(d²(p_n, M) − ε²) converges weakly to a mean-zero Gaussian with variance σ²(p₀) = 4 Σ_{x∈C} (1/w_x) p_{0,x}(1 − p_{0,x})(p_{0,x} − q_x(β₀))². This justifies rejecting H₀ = {d(p, M) ≥ ε} when T(p_n) ≤ c_α σ(p_n). The paper also derives a bootstrap-t version, but its simulations show that version over-rejects on the boundary, while the asymptotic test with bootstrap-estimated variance performs well on the two real data sets considered.

What carries the argument

The central object is the minimum Euclidean distance d(p, M) = inf_β ||p − q(β)|| between the cell probability vector p and the model surface q(β). The argument rests on differentiability of p ↦ d²(p, M) at the boundary point, with gradient 2(p_{0,x} − q_x(β₀)); the delta method then transfers √n(p_n − p₀) convergence to the distance statistic. The explicit variance formula makes the test computable, and the bootstrap version estimates that variance from resampled cell counts.

Load-bearing premise

The load-bearing premise is that the squared minimum distance p ↦ d²(p, M) is differentiable at the boundary point, which requires a continuous minimizer h(p) to exist nearby; the paper assumes this essential condition and checks it only by trying multiple optimization starting points.

What would settle it

Construct a boundary point p₀ with d(p₀, M) = ε where the design matrix is collinear, a cell has zero observations, or the argmin β is not unique; simulate T(p_n) and compare its distribution to the claimed Gaussian with variance (1). A visible departure—or a failure of the gradient to exist—would falsify the central claim and the validity of both critical-value procedures.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

A statistically valid equivalence test now exists for logistic and probit regression with categorical covariates, complementing goodness-of-fit tests that only detect lack of fit.
The minimum tolerance parameter ε_min gives a data-scale measure of how close the model is to the data, useful for reporting model quality.
On the two real data sets, the test fails to reject equivalence, showing that the procedure can yield meaningful negative conclusions.
The simulation study provides a practical recommendation: use the asymptotic test with bootstrap-estimated variance rather than the bootstrap-t version.
The multinomial extension sketched in the paper suggests the approach generalizes beyond binary outcomes to multicategory GLMs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Extension the paper leaves implicit: the same test could be used for ongoing model monitoring, such as checking whether a deployed credit-risk or insurance model has drifted beyond tolerance relative to current data.
The differentiability assumption could be checked systematically by computing subgradients at boundary points; if non-unique minimizers appear, a smoothed or projected version of d² might be needed to retain the Gaussian limit.
The boundary-point sampling algorithm in the simulations could be repurposed as a calibration tool: practitioners can generate local boundary points to see whether reported p-values are honest in their own sparse-cell settings.
The variance formula shows that rare covariate cells, through the 1/w_x factor, dominate the uncertainty; the paper does not emphasize this practical warning.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A genuinely new equivalence test for binary GLMs with categorical covariates, but Proposition 5 rests on an unproved, self-cited differentiability condition that must be resolved before the method can be trusted.

read the letter

This paper is a good candidate for referee time, but the referee should be told up front where the load-bearing weight sits. The new thing is real: an equivalence test for binary GLMs with categorical covariates, a gap the authors correctly identify. The test statistic is the minimum Euclidean distance between observed cell frequencies and the model family, and the asymptotic and bootstrap critical values are worked out. The simulation study is honest — it reports that the bootstrap-t version over-rejects (max rejection 0.115 at nominal 0.05, Table 3) and recommends the asymptotic BV test. The R code and both datasets are public, which is reproducible evidence.

Now the soft spots. The central Gaussian limit in Proposition 5 hangs entirely on the differentiability of p ↦ d²(p,M) at boundary points. The paper calls this 'essential' (Remark 6) and takes it from Ostrovski [12, Theorem 1], a self-cited paper, without stating the theorem or verifying its hypotheses for binary GLMs. The stress-test note is right: if the argmin β(p) is not locally unique, or the design has empty or collinear cells, that differentiability can fail. The numerical check with multiple starting points on 100 boundary points near two datasets is not a proof; the authors themselves hedge with 'shed some light.' This is not a demonstrated error, but it is a condition that must be stated and either proved or formally verified before the method can be a reference.

Second, the size control evidence is thin: only 100 boundary points constructed near two real datasets. That is a start, not a guarantee. Third, Table 2 calibrates ε at 0.9 power on the very model fitted to the tested data; that is a mild double-use and should be stated as conditional. Fourth, the conclusion that the Fiji/Titanic logistic models are 'not equivalent' overstates what a failure to reject H0 establishes — it only means the test did not find evidence of closeness.

None of these are fatal; the internal math is consistent, the variance formula is dimensionally correct, and the evidence that the asymptotic BV test controls size in the studied neighborhoods is decent. But the differentiability premise is the load-bearing wall, and it comes from a self-citation without proof. That deserves a serious referee, not a desk reject.

Referee Report

4 major / 5 minor

Summary. The paper develops an equivalence test for binary generalized linear models (GLMs) with categorical covariates. The test statistic is T(p_n)=√n(d^2(p_n,M)−ε^2), where d(p,M)=inf_β ||p−q(β)|| is the minimum Euclidean distance from the vector of conditional response probabilities to the model family M, and H0: d(p,M)≥ε is rejected for sufficiently small values of T(p_n). Critical values are obtained from an asymptotic normal approximation (Proposition 5) or from bootstrap procedures (an 'asymptotic BV' variance estimator and a bootstrap-t method). The paper applies the test to two real datasets (Fiji fertility survey and Titanic survival), compares minimum-distance estimates with MLEs, calibrates ε through power simulations, and studies type I errors at randomly generated boundary points. The asymptotic BV test is recommended on the basis of the simulations.

Significance. If the main assumptions hold, the paper fills a genuine gap: there is no standard equivalence test for binary GLMs, and the proposed minimum-distance construction gives a clearly interpretable test with a simple asymptotic form. The variance formula (1) is dimensionally correct, the test is easy to implement, and the authors provide publicly available R code and reproducible simulations based on real data—these are strengths. However, the central theoretical step, the differentiability of p↦d^2(p,M) and the existence of a continuous minimizer, is imported from a theorem in the author's earlier paper [12] without statement or verification in the GLM context. Because both the asymptotic and bootstrap critical-value procedures rest on this step, the theoretical foundation of the proposed tests is not yet fully established.

major comments (4)

[§2, Proposition 5 and Remark 6] The proof of Proposition 5 relies on the assertion that p↦d^2(p,M) is differentiable at p0 with gradient 2(p0−q(β0)), attributed to Ostrovski [12, Theorem 1]. That theorem is not stated, and its hypotheses are not verified for the binary GLM family considered here. Remark 6 calls the existence of a continuous minimizer h 'essential' and suggests numerical validation via multiple optimization starting points, but this does not prove the condition. Without a self-contained proof—or at least a restatement of the theorem and a verification of its hypotheses—the delta-method step and the variance formula (1) are not justified. This gap also undermines the bootstrap-t procedure, which cites consistency theorems requiring differentiability of the statistic. Please add a lemma giving sufficient conditions (e.g., identifiability of the design matrix, positive cell weights, local uniqueness of the
[§2, asymptotic test after Corollary 7] The paper proposes a level-α test for the composite null H0={d(p,M)≥ε}, but Proposition 5 is pointwise at a fixed boundary point p0 with d(p0,M)=ε. To justify rejecting H0 for all p in H0, one needs uniform control of the rejection probability over the null, or an argument that the supremum is attained at the boundary and that the convergence in Proposition 5 is locally uniform. Without this, sequences of null points approaching the boundary at rate n^{−1/2} could produce rejection probabilities not controlled by the boundary calculation. The simulation study in §3.3 checks only 100 boundary points near two datasets and cannot substitute for a uniformity proof. Please either provide a uniform asymptotic size argument or state explicitly that the test is justified pointwise at the boundary only.
[§2, bootstrap-t paragraph] The bootstrap-t test is declared consistent by citing van der Vaart [14, Theorems 23.4 and 23.5], but those theorems require conditions—including differentiability of the statistic and consistency of the variance estimator—that are not established in the present setting. The required differentiability is exactly the unproved assumption in Proposition 5. Moreover, Table 3 shows that the bootstrap-t test over-rejects at boundary points (mean rejection probabilities 0.081 and 0.061 at nominal 0.05, maxima above 0.10). Unless the bootstrap validity is proved under the stated assumptions, the bootstrap-t procedure should be presented as an exploratory variant, with the asymptotic BV test as the recommended procedure, rather than as a fully validated test.
[§1, definition of the test statistic] The test statistic T(p_n) requires computing d(p_n,M)=inf_β ||p_n−q(β)||. The paper says 'If there exists β_n such that d(p_n,M)=l_2(p_n,q(β_n))', but the existence of a global minimizer is not established for binary GLMs. For logistic regression, M is not closed in [0,1]^C (probabilities approach 0 or 1 as ||β||→∞ but do not attain them), so for some observed vectors p_n—particularly those with cells at 0 or 1—the infimum may not be attained. This affects the definition of T(p_n) and the numerical optimization. Please state conditions ensuring that a global minimizer exists for the sample p_n, or define the analysis in terms of the infimum with an appropriate treatment of the non-attainment case.

minor comments (5)

[§1, hypothesis statement] The hypothesis is written as H0={d(p_n,M)≥ε}, but hypotheses should be about the true p, not the estimator p_n. It should read H0={d(p,M)≥ε}.
[§2, proof of Proposition 5] The name 'Slutzki's lemma' should be 'Slutsky's lemma'.
[§3.1, boundary-point algorithm] Step 4 says 'for a∈[0,1]' and then 'The value of a_n'; the variable should be w throughout, i.e., 'for w∈[0,1]' and 'the value of w_n'.
[§2, Remark 8] Remark 8 claims the asymptotic test is locally asymptotically most powerful, citing [11, Proposition 3] without proof or conditions. If kept, this claim should be stated precisely and proved, or removed, as it is not needed for the validity of the test.
[§3.3, Table 3] The table reports means and maxima of rejection probabilities over 100 boundary points, but does not report the standard error of the mean or the number of boundary points with rejection probabilities above the nominal level. Since 1000 simulations per boundary point give a binomial standard error of about 0.007 at rejection probability 0.05, the reported differences are meaningful; please add the Monte Carlo uncertainty for completeness.

Circularity Check

0 steps flagged

No circularity: the asymptotic test is a derived statistic rather than a fitted prediction; the self-cited differentiability theorem is a supporting external result, not a definitional equivalence.

full rationale

I walked the derivation chain. Proposition 5 combines the classical CLT for binomial cell frequencies, Slutsky's lemma, and the delta method applied to p -> d^2(p,M). The only non-classical input is the differentiability of the minimum-distance functional at p_0, which the paper imports as 'by Ostrovski [12, Theorem 1]' under the explicit continuous-minimizer assumption stated in Proposition 5 and Remark 6. Nothing in the paper shows that this theorem is identical to the conclusion being derived: the conclusion is the Gaussian limit of sqrt(n)(d^2(p_n,M) - eps^2), while the cited theorem supplies the gradient (2(p0,x - q_x(beta0))) of the minimum-distance function. The variance formula (1) is obtained by combining that gradient with the multinomial covariance, not by fitting a parameter and renaming it a prediction. The eps_min formula (2) is an algebraic restatement of the rejection rule T <= c_alpha sigma(p_n), not a separate predicted quantity. The bootstrap-t consistency is cited to van der Vaart's standard theorems. The tolerance eps calibration in Section 3.2 is a data-analytic choice, not a derivation from the test statistic itself. The genuine caveat is rigor, not circularity: the essential continuous-minimizer/differentiability condition is inherited from the author's earlier papers and only validated numerically in Remark 6, so the theoretical foundation is not fully self-contained here. But an unproved or self-cited supporting theorem is an omitted-proof/verification concern, not an equivalence of inputs and outputs. No circular step can be exhibited from the text.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The paper introduces no new entities. The load-bearing unproven input is a differentiability/continuity result on the minimum-distance function, cited from the author's prior paper [12]; the tolerance ε is a user-chosen calibration rather than a mathematical free parameter. The bootstrap-t validity is invoked from a textbook but is contradicted by the paper's own boundary simulations.

free parameters (1)

Tolerance parameter ε = 0.32–0.37 (Fiji); 0.13–0.16 (Titanic) across the three tests at 0.9 power
User-chosen threshold, calibrated in Table 2 so that simulated power at the MDE-fitted model equals 0.9. It is a modeling choice rather than an estimated parameter, but the calibration uses the same data that are later tested, a mild double-use.

axioms (5)

domain assumption Existence of a continuous minimizer h(p) on an open neighborhood of each boundary point p_0, and differentiability of p↦d²(p,M) with gradient 2(p_{0,x}−q_x(β_0))
Remark 6 states this is assumed for the rest of the paper and is 'essential' for Proposition 5 and for computing d(p_0,M). The gradient formula is taken from the self-cited Ostrovski [12, Theorem 1], not proved here. Boundary points with a discontinuous or non-unique argmin would break the Gaussian limit.
domain assumption n_x > 0 for every cell x and n_x/n → w_x ∈ (0,1)
Stated before Proposition 5; guarantees √n(p̂_n − p_0) has a finite Gaussian limit with variance p(1−p)/w_x. Fails for sparse contingency tables, a common real-world case for categorical GLMs.
domain assumption All covariates are categorical; observations form a multi-way contingency table
Explicit scope restriction in the Introduction. The test does not apply to continuous covariates, which limits the method's reach.
standard math CLT, Slutsky's lemma, delta method, continuous mapping theorem
Used in the proof of Proposition 5 and Corollary 7; standard background.
standard math Bootstrap consistency (van der Vaart Theorems 23.4/23.5)
Cited to justify the bootstrap-t test. The paper's own boundary-point simulations (Table 3) show the bootstrap-t over-rejects (max 0.115 at nominal 0.05), so the applicability of this axiom at the complex H_0 boundary is in question.

reviewed 2026-08-02 · how reviews work

0 comments

Cite this review

Pith. "Pith review of Testing equivalence to binary generalized linear models with application to logistic regression." pith.science (2026). https://pith.science/paper/MJIZ7T2H

@misc{pith2026260714724,
  author       = {Pith},
  title        = {Pith review of: Testing equivalence to binary generalized linear models with application to logistic regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MJIZ7T2H}},
  note         = {Machine review of arXiv:2607.14724}
}

Share X Bluesky LinkedIn Reddit HN

read the original abstract

We introduce a new equivalence test to show sufficiently good agreement of observed data with a binary generalized linear model (GLM). The test statistic is constructed via the minimum distance method. The test is developed for the important special case where all covariates are categorical. The critical values can be calculated using an asymptotic approximation or by means of bootstrapping. The application of the test to logistic regression is illustrated on two real data sets. The finite sample performance of the proposed test is studied by simulations which are based on these two data sets.

discussion (0)

Reference graph

Works this paper leans on

14 extracted references

[1]

A. Basu, I. R. Harris, N. L. Hjort, and M. C. Jones. Robust and efficient estimation by minimising a density power divergence.Biometrika, 85(3): 549–559, 1998

1998
[2]

H. D. Bondell. Minimum distance estimation for the logistic regression model.Biometrika, 92(3):724–731, 2005. 11

2005
[3]

E. C. Chi and D. W. Scott. Robust parametric classification and variable selection by a minimum distance criterion.Journal of Computational and Graphical Statistics, 23(1):111–128, 2014

2014
[4]

The”unusualepisode” datarevisited.Journal of Statistics Education, 3(3), 1995

R.J.M.Dawson. The”unusualepisode” datarevisited.Journal of Statistics Education, 3(3), 1995

1995
[5]

Automatic

D. L. Donoho and R. C. Liu. The "Automatic" Robustness of Minimum Distance Functionals.The Annals of Statistics, 16(2):552 – 586, 1988

1988
[6]

Efron and R

B. Efron and R. J. Tibshirani.An introduction to the bootstrap. Chapman & Hall, 1993

1993
[7]

D. W. Hosmer and N. L. Hjort. Goodness-of-fit processes for logistic regres- sion: simulation results.Statistics in Medicine, 21(18):2723–2738, 2002

2002
[8]

D. W. Hosmer, S. Taber, and S. Lemeshow. The importance of assessing the fit of logistic regression models: a case study.American Journal of Public Health, 81(12):1630–1635, 1991

1991
[9]

D. W. Hosmer, T. Hosmer, S. Le Cessie, and S. Lemeshow. A comparison of goodness-of-fittestsforthelogisticregressionmodel.Statistics in Medicine, 16(9):965–980, 1997

1997
[10]

Little.Generalized Linear Models for Cross-classified Data from the World Fertility Survey.International Statistical Institute, 1978

R. Little.Generalized Linear Models for Cross-classified Data from the World Fertility Survey.International Statistical Institute, 1978

1978
[11]

Ostrovski

V. Ostrovski. Testing equivalence of multinomial distributions.Statistics and Probability Letters, 124:77–82, 2017

2017
[12]

Ostrovski

V. Ostrovski. Testing equivalence to families of multinomial distributions with application to the independence model.Statistics and Probability Letters, 139:61–66, 2018. 12

2018
[13]

Pregibon

D. Pregibon. Logistic Regression Diagnostics.The Annals of Statistics, 9 (4):705 – 724, 1981

1981
[14]

van der Vaart.Asymptotic Statistics

A. van der Vaart.Asymptotic Statistics. Cambridge University, 1998. 13

1998

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.

[1] [1]

A. Basu, I. R. Harris, N. L. Hjort, and M. C. Jones. Robust and efficient estimation by minimising a density power divergence.Biometrika, 85(3): 549–559, 1998

1998

[2] [2]

H. D. Bondell. Minimum distance estimation for the logistic regression model.Biometrika, 92(3):724–731, 2005. 11

2005

[3] [3]

E. C. Chi and D. W. Scott. Robust parametric classification and variable selection by a minimum distance criterion.Journal of Computational and Graphical Statistics, 23(1):111–128, 2014

2014

[4] [4]

The”unusualepisode” datarevisited.Journal of Statistics Education, 3(3), 1995

R.J.M.Dawson. The”unusualepisode” datarevisited.Journal of Statistics Education, 3(3), 1995

1995

[5] [5]

Automatic

D. L. Donoho and R. C. Liu. The "Automatic" Robustness of Minimum Distance Functionals.The Annals of Statistics, 16(2):552 – 586, 1988

1988

[6] [6]

Efron and R

B. Efron and R. J. Tibshirani.An introduction to the bootstrap. Chapman & Hall, 1993

1993

[7] [7]

D. W. Hosmer and N. L. Hjort. Goodness-of-fit processes for logistic regres- sion: simulation results.Statistics in Medicine, 21(18):2723–2738, 2002

2002

[8] [8]

D. W. Hosmer, S. Taber, and S. Lemeshow. The importance of assessing the fit of logistic regression models: a case study.American Journal of Public Health, 81(12):1630–1635, 1991

1991

[9] [9]

D. W. Hosmer, T. Hosmer, S. Le Cessie, and S. Lemeshow. A comparison of goodness-of-fittestsforthelogisticregressionmodel.Statistics in Medicine, 16(9):965–980, 1997

1997

[10] [10]

Little.Generalized Linear Models for Cross-classified Data from the World Fertility Survey.International Statistical Institute, 1978

R. Little.Generalized Linear Models for Cross-classified Data from the World Fertility Survey.International Statistical Institute, 1978

1978

[11] [11]

Ostrovski

V. Ostrovski. Testing equivalence of multinomial distributions.Statistics and Probability Letters, 124:77–82, 2017

2017

[12] [12]

Ostrovski

V. Ostrovski. Testing equivalence to families of multinomial distributions with application to the independence model.Statistics and Probability Letters, 139:61–66, 2018. 12

2018

[13] [13]

Pregibon

D. Pregibon. Logistic Regression Diagnostics.The Annals of Statistics, 9 (4):705 – 724, 1981

1981

[14] [14]

van der Vaart.Asymptotic Statistics

A. van der Vaart.Asymptotic Statistics. Cambridge University, 1998. 13

1998

Testing equivalence to binary generalized linear models with application to logistic regression

The reading

Core claim §

What carries the argument

Load-bearing premise §

What would settle it §

If this is right

Where Pith is reading between the lines

discussion (0)

Core claim

Load-bearing premise

What would settle it