REVIEW 64 references
Beyond Modern Asymptotics for Log-Likelihood Ratios in Logistic Regression
T0 review · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read In binary logistic regression, the worst-case (1−δ) quantile of the log-likelihood ratio is d log(en/d) + log(1/δ) for n ≥ d ≥ 3, up to universal constants.
desk verdict The d≥3 worst-case LLR characterization is a genuine finite-sample Wilks analogue and deserves a serious referee; the d=2 log log log n upper bound is not yet proven — it sits in an appendix as an AI-generated proof sketch with load-bearing gaps. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument rests on three mechanisms. First, the log-likelihood ratio depends on the design only through the subspace W = ran(X) and on the target only through v = Xθ*, reducing the statistic to Γ_W(ε;v) = sup_{w∈W} log(p_w(ε)/p_v(ε)). Second, the exponential moment of Γ_W evaluated at λ=1 is exactly the Shtarkov sum S_log(W) = Σ_ε sup_{w∈W} p_w(ε), which is independent of v and bounded by the number of regions in an arrangement of at most n hyperplanes, at most Σ_{ℓ=0}^k C(n,ℓ) ≤ (en/k)^k; Markov's inequality then yields the upper tail. Third, matching lower bounds come from Vandermonde subspaces—ranges of polynomial-evaluation maps—whose sign patterns realize all patterns with few negati
What would settle it
For n = exp(exp(exp(20))), simulate the two-dimensional block design from Proposition 4.1 with k=20 and estimate the 90% quantile of the log-likelihood ratio; the paper's d=2 claim predicts it to be about 20 + log(1/δ) up to a universal constant, so a value outside a constant factor of that would refute the d=2 scale.
Extended reading notes
Core claim
The paper's central discovery is a nonasymptotic Wilks-type law for logistic regression that is uniform in the worst possible way. For n≥d≥3, no matter which fixed design vectors x_1,...,x_n in R^d and no matter which target parameter θ* in R^d, the (1−δ) quantile of the log-likelihood ratio Λ satisfies Q_{1−δ} ≈ d log(en/d) + log(1/δ). The same scale holds for random design when the supremum is taken over all distributions. In dimension 2 the worst case drops to log log log n + log(1/δ), and in dimension 1 to log(1/δ). Under i.i.d. Gaussian design the logarithmic factor disappears: with n ≳ d + log(1/δ), the bound is d + log(1/δ), uniformly over θ*. The upper bounds require no MLE existence
Load-bearing premise
Everything in the d≥3 theorem rests on standard inequalities, but the d=2 upper bound rests on the appendix's proof sketch—after fixing the common offset and conditioning on the number of mislabeled points, the remaining uncertainty is handled by a one-dimensional bound—and if that reduction or the planar Helly step has a gap, the log log log n scale for d=2 is not established.
Editorial extensions
If this is right
- The confidence set {θ : Λ(θ) ≤ d log(en/d) + log(1/δ)} has coverage at least 1−δ for every fixed design and every θ*, with no regularity and no need for the MLE to exist.
- The logarithmic factor log(en/d) is an unavoidable price of uniformity: some fixed designs and target parameters (Vandermonde designs with θ* along a coordinate axis) have likelihood-ratio quantiles of this order, so the classical chi-square scale cannot hold in worst-case fixed design.
- Under i.i.d. Gaussian design, the classical chi-square-type scale d + log(1/δ) is restored uniformly over θ* as soon as n ≳ d + log(1/δ).
- The validity of the classical Wilks approximation at the origin is governed by d^{3/2}/n → 0, not by the aspect ratio d/n; the distribution can be far from chi-square even when d/n is small.
- The worst-case scale is sharply dimension-sensitive: d=1 costs only log(1/δ), d=2 costs log log log n, and d≥3 costs d log(en/d).
Reading between the lines
- The same subspace-and-Shtarkov machinery likely gives sharp finite-sample regret bounds for transductive sequential prediction of logistic labels; the paper's upper bound is effectively a worst-case regret of order d log(en/d) for the logistic class, and the lower bounds indicate this is minimax in the worst case.
- The d=2 anomaly suggests that the plane has special geometry for this problem; one can ask whether other low-dimensional substructures inside higher-dimensional designs could create intermediate scales, a question the paper leaves open.
- The Gaussian-design guarantee is proved only for Gaussian covariates, but the route through Hessian concentration suggests a testable extension to other random designs with well-conditioned covariance and sub-Gaussian tails; this is an editorial conjecture, not a paper claim.
- The d=2 upper bound rests on an appendix proof sketch whose key reductions are asserted rather than fully fleshed out; until that sketch is completed, the log log log n scale should be treated as conditional, while the d≥3 statement does not inherit this caveat.
Editorial analysis
A structured set of objections, weighed in public.
Circularity Check
No significant circularity: the main d>=3 worst-case derivation is self-contained; the d=2 upper bound rests on an unverified AI-assisted proof sketch, which is a correctness risk, not a circularity.
full rationale
The central result, Theorem 2.1 for d>=3, is derived without fitting or renaming inputs as outputs. The upper bound comes from the Shtarkov-sum/hyperplane-arrangement count of Lemma 2.2 / Theorem 4.1(i) plus Proposition 2.1, and the lower bound comes from an explicit Vandermonde subspace construction in Proposition 2.4. These arguments are self-contained and do not assume the target quantile scale. No parameter is fitted to a subset of data and then called a prediction; the worst-case quantile is derived uniformly over all designs and all target parameters. The paper does contain self-citations: [14] is used for the Gaussian-design analysis, with one overlapping author, but [14] is an independent prior result with its own stated assumptions and proofs and is not used for the d>=3 worst-case theorem. The transductive regret paper [45] is mentioned but explicitly not relied upon. The main caveat is Appendix A: Theorem A.1, the d=2 upper bound, is credited to GPT-5.6-Sol, and the proof is only a sketch. The critical scalar reduction (58) and the pathwise planar quantitative Helly aggregation are asserted rather than fully proved. This is a genuine verification gap for the advertised log log log n scale, but it is not circularity: the asserted steps reduce the problem to univariate bounds and geometric covering arguments, not to the theorem being proved. Thus the paper does not exhibit definitional circularity, fitted-prediction circularity, or a load-bearing self-citation loop; the low score reflects only minor self-citations and the verification risk in the appendix.
Assumptions & free parameters
assumptions (8)
- standard math An arrangement of at most n affine hyperplanes in R^d has at most sum_{ℓ=0}^d C(n,ℓ) cells.
- standard math Hoeffding's extremal theorem [27, Theorem 4]: among Poisson-binomial laws with mean k, Bin(n,k/n) maximizes upper-tail probabilities.
- standard math Cover's formula: for n points in general position in R^d, the number of dichotomies induced by homogeneous halfspaces is 2 sum_{ℓ=0}^{d−1} C(n−1,ℓ).
- standard math Conic kinematic estimate of Amelunxen–Lotz–McCoy–Tropp [4, Thm 7.1] for random subspaces and cones, and the statistical-dimension bound for the cones C_r.
- standard math Planar quantitative Helly theorem [8]: for convex sets in R^2, a small subfamily controls the intersection area, used to reduce the number of dyadic blocks to at most 4.
- domain assumption Nonasymptotic bounds on the logistic MLE and Hessian under Gaussian design from Chardon–Lerasle–Mourtada [14, Prop 5, Thm 6, Lemma 3, Sec 8.2].
- domain assumption The observational model is well-specified logistic regression with deterministic covariates and labels in {−1,1}.
- standard math Okamoto's lower-tail bound for binomial tails [41, Theorem 2].
Cite this review
Pith. "Pith review of Beyond Modern Asymptotics for Log-Likelihood Ratios in Logistic Regression." pith.science (2026). https://pith.science/paper/7OCFL2UO
@misc{pith2026260802507,
author = {Pith},
title = {Pith review of: Beyond Modern Asymptotics for Log-Likelihood Ratios in Logistic Regression},
year = {2026},
howpublished = {\url{https://pith.science/paper/7OCFL2UO}},
note = {Machine review of arXiv:2608.02507}
}
abstract
We characterize the finite sample behavior of the log-likelihood ratio statistic in binary logistic regression, uniformly over both the design and the target parameter. For $n\geq d\geq 3$, we determine, up to universal constants, its worst case $(1-\delta)$ quantile over all fixed collections of design vectors and all target parameters: \[ d\log\left(\frac{e n}{d}\right)+\log\left(\frac{1}{\delta}\right). \] This is a nonasymptotic analogue of the Wilks $\chi^2_d$ phenomenon and requires no regularity assumptions on the design. The low dimensional cases exhibit unusual behavior. The worst case quantile in dimension $d=2$ is sharply of order \[ \log\log\log n+\log\left(\frac{1}{\delta}\right). \] The worst case quantile in dimension $d=1$ is of order $\log(1/\delta)$, with no dependence on $n$. Finally, i.i.d. Gaussian design vectors recover the classical Wilks scale. In the regime $n\gtrsim d+\log(1/\delta)$, we prove the sharp bound \[ d+\log\left(\frac{1}{\delta}\right). \] Unlike existing asymptotic results, our bounds are uniform over the target parameter, which may depend on $n$, $d$, and $\delta$.
Reference graph
Works this paper leans on
-
[14]
H. Chardon, M. Lerasle, and J. Mourtada. Finite-sample performance of the maximum likelihood estimator in logistic regression.arXiv preprint arXiv:2411.02137v3, 2024
arXiv 2024
-
[45]
J. Qian, A. Rakhlin, and N. Zhivotovskiy. Refined risk bounds for unbounded losses via transductive priors. Journal of Machine Learning Research, 27(26):1–64, 2026
2026
-
[1]
Abbasi-Yadkori, D
Y. Abbasi-Yadkori, D. P´ al, and C. Szepesv´ ari. Online-to-confidence-set conversions and application to sparse stochastic bandits. InProceedings of the Fifteenth International Conference on Artificial Intelligence and Statis- tics, volume 22, pages 1–9. PMLR, 2012
2012
-
[2]
R. Agrawal. Finite-sample concentration of the multinomial in relative entropy.IEEE Transactions on Infor- mation Theory, 66(10):6297–6302, 2020
2020
-
[3]
Albert and J
A. Albert and J. A. Anderson. On the existence of maximum likelihood estimates in logistic regression models. Biometrika, 71(1):1–10, 1984
1984
-
[4]
Amelunxen, M
D. Amelunxen, M. Lotz, M. B. McCoy, and J. A. Tropp. Living on the edge: Phase transitions in convex programs with random data.Information and Inference: A Journal of the IMA, 3(3):224–294, 2014
2014
-
[5]
Andresen and V
A. Andresen and V. Spokoiny. Critical dimension in profile semiparametric estimation.Electronic Journal of Statistics, 8(2):3077–3125, 2014
2014
-
[6]
F. Bach. Self-concordant analysis for logistic regression.Electronic Journal of Statistics, 4:384–414, 2010
2010
Show all 64 references
-
[7]
Baillon, R
J.-B. Baillon, R. Cominetti, and J. Vaisman. A sharp uniform bound for the distribution of sums of Bernoulli trials.Combinatorics, Probability and Computing, 25(3):352–361, 2016
2016
-
[8]
B´ ar´ any, M
I. B´ ar´ any, M. Katchalski, and J. Pach. Quantitative Helly-type theorems.Proceedings of the American Mathe- matical Society, 86(1):109–114, 1982
1982
-
[9]
A. R. Barron. Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems. In J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, editors, Bayesian Statistics 6: Proceedings of the Sixth Valencia Inte...
1999
-
[10]
P. J. Bickel and K. A. Doksum.Mathematical Statistics: Basic Ideas and Selected Topics, volume 1. Pearson Prentice Hall, Upper Saddle River, NJ, 2nd edition, 2007
2007
-
[11]
Boucheron, G
S. Boucheron, G. Lugosi, and P. Massart.Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, Oxford, 2013
2013
-
[12]
E. J. Cand` es and P. Sur. The phase transition for the existence of the maximum likelihood estimate in high- dimensional logistic regression.The Annals of Statistics, 48(1):27–42, 2020
2020
-
[13]
Cesa-Bianchi and G
N. Cesa-Bianchi and G. Lugosi.Prediction, Learning, and Games. Cambridge University Press, Cambridge, 2006
2006
-
[15]
Chernoff
H. Chernoff. On the distribution of the likelihood ratio.Ann. Math. Statistics, 25(3):573–578, 1954
1954
-
[16]
Clerico, H
E. Clerico, H. Flynn, W. Kot lowski, and G. Neu. Confidence sequences for generalized linear models via regret analysis.arXiv preprint arXiv:2504.16555, 2025. 51
2025 arXiv
-
[17]
T. M. Cover. Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition.IEEE Transactions on Electronic Computers, EC-14(3):326–334, 1965
1965
-
[18]
J. N. Darroch. On the distribution of the number of successes in independent trials.The Annals of Mathematical Statistics, 35(3):1317–1321, 1964
1964
-
[19]
Dembo and O
A. Dembo and O. Zeitouni.Large deviations techniques and applications, volume 38 ofStochastic Modelling and Applied Probability. Springer-Verlag, Berlin, 2010. Corrected reprint of the second (1998) edition
2010
-
[20]
J. G. Dowty. Volumes of logistic regression models with applications to model selection.arXiv preprint arXiv:1408.0881, 2014
2014 arXiv
-
[21]
Drmota, P
M. Drmota, P. Jacquet, C. Wu, and W. Szpankowski. Minimax regret with unbounded weights. In2024 IEEE International Symposium on Information Theory (ISIT), pages 2305–2310. IEEE, 2024
2024
-
[22]
Drmota, P
M. Drmota, P. Jacquet, C. Wu, and W. Szpankowski. Precise regularized minimax regret with unbounded weights.IEEE Transactions on Information Theory, 71(11):8753–8776, 2025
2025
-
[23]
Drmota, P
M. Drmota, P. Jacquet, C. Wu, and W. Szpankowski. Phase transition of regret for logistic regression with large weights. In M. Telgarsky and J. Ullman, editors,Proceedings of the 37th International Conference on Algorithmic Learning Theory, volume 313 ofProceedings of Machine ...
2026
-
[24]
J. Fan, C. Zhang, and J. Zhang. Generalized likelihood ratio statistics and Wilks phenomenon.The Annals of Statistics, 29(1):153–193, 2001
2001
-
[25]
D. J. Foster, S. Kale, H. Luo, M. Mohri, and K. Sridharan. Logistic regression: The importance of being improper. InProceedings of the 31st Conference On Learning Theory, volume 75, pages 167–208, 2018
2018
-
[26]
F. R. Guo and T. S. Richardson. Chernoff-type concentration of empirical probabilities in relative entropy.IEEE Transactions on Information Theory, 67(1):549–558, 2021
2021
-
[27]
Hoeffding
W. Hoeffding. On the distribution of the number of successes in independent trials.The Annals of Mathematical Statistics, 27(3):713–721, 1956
1956
-
[28]
Hsu and A
D. Hsu and A. Mazumdar. On the sample complexity of parameter estimation in logistic regression with normal design. InProceedings of Thirty Seventh Conference on Learning Theory, volume 247 ofProceedings of Machine Learning Research, pages 2418–2437. PMLR, 2024
2024
-
[29]
I. A. Ibragimov and R. Z. Khas’minskii.Statistical Estimation: Asymptotic Theory, volume 16 ofApplications of Mathematics. Springer-Verlag, New York, 1981
1981
-
[30]
Jacquet, G
P. Jacquet, G. Shamir, and W. Szpankowski. Precise minimax regret for logistic regression with categorical feature values. In V. Feldman, K. Ligett, and S. Sabato, editors,Proceedings of the 32nd International Conference on Algorithmic Learning Theory, volume 132 ofProceedings...
2021
-
[31]
Jacquet, G
P. Jacquet, G. I. Shamir, and W. Szpankowski. Precise minimax regret for logistic regression. In2022 IEEE International Symposium on Information Theory (ISIT), pages 444–449, 2022
2022
-
[32]
S. M. Kakade and A. Y. Ng. Online bounds for Bayesian algorithms. In L. K. Saul, Y. Weiss, and L. Bottou, editors,Advances in Neural Information Processing Systems 17, pages 641–648. MIT Press, 2005
2005
-
[33]
R. W. Keener.Theoretical Statistics: Topics for a Core Course. Springer Science & Business Media, 2010
2010
-
[34]
Kirschner, A
J. Kirschner, A. Krause, M. Meziu, and M. Mutn´ y. Confidence estimation via sequential likelihood mixing. arXiv preprint arXiv:2502.14689, 2025
2025 arXiv
-
[35]
Kuchelmeister and S
F. Kuchelmeister and S. van de Geer. Finite sample rates for logistic regression with small noise or few samples. Sankhya A, 2024
2024
-
[36]
Laurent and P
B. Laurent and P. Massart. Adaptive estimation of a quadratic functional by model selection.Ann. Statist., 28(5):1302–1338, 2000
2000
-
[37]
L. Le Cam. Locally asymptotically normal families of distributions.University of California Publications in Statistics, 3(2):37–98, 1960
1960
-
[38]
Le Cam and G
L. Le Cam and G. L. Yang.Asymptotics in Statistics: Some Basic Concepts. Springer Series in Statistics. Springer, New York, 2nd edition, 2000
2000
-
[39]
Lee, S.-Y
J. Lee, S.-Y. Yun, and K.-S. Jun. Improved regret bounds of (multinomial) logistic bandits via regret-to- confidence-set conversion. InProceedings of the 27th International Conference on Artificial Intelligence and Statistics, volume 238 ofProceedings of Machine Learning Resea...
2024
-
[40]
A. Maurer. A note on the PAC Bayesian theorem.arXiv preprint cs/0411099, 2004
2004 arXiv
-
[41]
M. Okamoto. Some inequalities relating to the partial sum of binomial probabilities.Annals of the Institute of Statistical Mathematics, 10(1):29–35, 1959
1959
-
[42]
Orlitsky and N
A. Orlitsky and N. P. Santhanam. Speaking of infinity [iid strings].IEEE Transactions on Information Theory, 50(10):2215–2230, 2004
2004
-
[43]
D. M. Ostrovskii and F. Bach. Finite-sample analysis of M-estimators using self-concordance.Electronic Journal of Statistics, 15(1):326–391, 2021
2021
-
[44]
S. Portnoy. Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity.The Annals of Statistics, 16(1):356–366, 1988
1988
-
[46]
J. J. Rissanen. Fisher information and stochastic complexity.IEEE Transactions on Information Theory, 42(1):40–47, 1996
1996
-
[47]
R. T. Rockafellar.Convex analysis. Princeton Mathematical Series, No. 28. Princeton University Press, Prince- ton, NJ, 1970
1970
-
[48]
G. I. Shamir and W. Szpankowski. A lower bound for regret in logistic regression. In2021 IEEE International Symposium on Information Theory (ISIT), pages 2507–2512, 2021
2021
-
[49]
Y. M. Shtarkov. Universal sequential coding of single messages.Problems of Information Transmission, 23(3):175–186, 1987
1987
-
[50]
Spokoiny
V. Spokoiny. Parametric estimation. Finite sample theory.The Annals of Statistics, 40(6):2877–2909, 2012
2012
-
[51]
Spokoiny
V. Spokoiny. Finite sample expansions and risk bounds in high-dimensional SLS models.arXiv preprint arXiv:2404.14227v3, 2025
2025 arXiv
-
[52]
Spokoiny and M
V. Spokoiny and M. Zhilova. Bootstrap confidence sets under model misspecification.The Annals of Statistics, 43(6):2653–2675, 2015
2015
-
[53]
Sur and E
P. Sur and E. J. Cand` es. A modern maximum-likelihood theory for high-dimensional logistic regression.Pro- ceedings of the National Academy of Sciences, 116(29):14516–14525, 2019
2019
-
[54]
P. Sur, Y. Chen, and E. J. Cand` es. The likelihood ratio test in high-dimensional logistic regression is asymp- totically a rescaled chi-square.Probability Theory and Related Fields, 175(1–2):487–558, 2019
2019
-
[55]
A. W. van der Vaart.Asymptotic statistics, volume 3 ofCambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998
1998
-
[56]
J. G. Wendel. A problem in geometric probability.Mathematica Scandinavica, 11:109–112, 1962
1962
-
[57]
S. S. Wilks. The large-sample distribution of the likelihood ratio for testing composite hypotheses.The Annals of Mathematical Statistics, 9(1):60–62, 1938
1938
-
[58]
0 is trivial. Assume thatHě1, fixRą0, and defineB R“ ␣ uPR 2 :}u} 2ďR ( . ForPĂt1, . . . , Hu, setL∅
Q. Zhao, P. Sur, and E. J. Cand` es. The asymptotic distribution of the MLE in high-dimensional logistic models: Arbitrary covariance.Bernoulli, 28(3):1835–1861, 2022. A AI disclosure and the proof of thed“2upper bound The results in the main body of the paper were developed b...
2022
-
[59]
Extreme block: modal-recode and set \(a_i=|v_i|\). For \(E=\{i:a_i>2\log n\}\) and the true product law \(P_E\) of their modal-error vector \(B_E\), prove \[ \Gamma_E(B_E)\le-\log P_E(B_E),\qquad \mathbb E e^{\eta[-\log P_E(B_E)]}\le C \] for universal \(\eta,C>0\)
-
[60]
Treat all-equal fibers and endpoint suprema directly
Offset profile: for arbitrary fixed logistic offsets and a common intercept, profile the intercept, condition on total error count, regularize the scalar slope, and prove a universal fractional exponential moment for the extended gain. Treat all-equal fibers and endpoint supre...
-
[61]
Factor-two block: normalize by its true margins; use concavity to compare the rescaled likelihoods; then reduce its gain, including fractional rescaling through Bernoulli thinning, to a bounded number of arbitrary-offset common-intercept profiles
-
[62]
\] On a shell \(s\le\Gamma<2s\), the ridge/KL identity gives \(Q(h_\varepsilon)\le Cs\)
Central geometry: use uniform local logistic curvature and \[ Q(h)=\sum_i\min\{z_i(h)^2,1\}. \] On a shell \(s\le\Gamma<2s\), the ridge/KL identity gives \(Q(h_\varepsilon)\le Cs\). For \(u\ge1\), prove by planar rank-two packing \[ \#\{i:\exists h,\ Q(h)\le u,\ |z_i(h)|\ge1\}...
-
[63]
Central gain: control the exceptional rows by a finite planar ray profile and the rest by a rank-two quadratic profile, obtaining a universal exponential tail and fractional exponential moment for the central-block gain
-
[64]
With \(\Delta=\sum_{s=0}^H\Gamma_s-\Gamma_\varepsilon\), prove that planar quantitative Helly selects \(|P|\le4\) with \[ \frac{Z_P}{Z}\le C(H+1)^2(\Delta+1)^2
Aggregation: use convex deficits and the sublevel-set/partition-function comparison. With \(\Delta=\sum_{s=0}^H\Gamma_s-\Gamma_\varepsilon\), prove that planar quantitative Helly selects \(|P|\le4\) with \[ \frac{Z_P}{Z}\le C(H+1)^2(\Delta+1)^2. \] 62
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.