REVIEW 2 major objections 6 minor 38 references
Asymptotic Normality of Infinite Centered Random Forests -Application to Imbalanced Classification
T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper proves a central limit theorem for infinite centered random forests with explicit rate and constant, then shows that after rebalancing and debiasing the estimator has smaller asymptotic variance as the minority class probability…
desk verdict The ICRF CLT with explicit constant is a real contribution, but the paper's headline variance-reduction claim for imbalanced classification is unproven because Theorem 3.5 leaves the i.i.d. framework without a valid substitute. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the U-statistic representation of the infinite forest, which reduces the CLT to verifying that $nV^s_1 \to \infty$ and that the bias vanishes. Under the uniform covariate assumption the leaf probability is deterministic, $p_{k,U}(x)=2^{-k}$, which makes $V^s_1$ computable: it is asymptotically $C(d)2^k/(s^2 k^{(d-1)/2})\mu(x)(1-\mu(x))$, where $C(d)$ comes from a kernel equivalent $E[P(X_1 \in L_U(x)|X_1)^2] \sim C(d)/(2^k k^{(d-1)/2})$. The same variance analysis, extended to the rebalanced distribution through a density formula and the odds-ratio identity linking $\mu$ and $\mu'$, yields the CLT for the rebalanced forest and the variance comparison.
What would settle it
Simulate under (H0), (H1), (G1) with $d=2$ and hyperparameters satisfying (9), and estimate the empirical variance of $\sqrt{n k^{1/2}/2^k}(\widehat{\mu}^{\mathrm{ICRF}}_s(x)-\mu(x))$; if it does not converge to $C(2)\mu(x)(1-\mu(x))$, the CLT constant is wrong. Similarly, reweight one original sample to build IS-ICRF and check whether its variance ratio to ICRF actually decays like $p$; if not, the extension behind Theorem 3.5 fails.
Extended reading notes
Core claim
The central discovery is that the subsampling infinite centered random forest estimator $\widehat{\mu}^{\mathrm{ICRF}}_s(x)$ satisfies $\sqrt{n k^{(d-1)/2}/2^k}(\widehat{\mu}^{\mathrm{ICRF}}_s(x)-\mu(x)) \to N(0, C(d)\mu(x)(1-\mu(x)))$ under the paper's conditions (H0), (H1), (G1) and the rate condition (9), with the constant $C(d)$ given explicitly. Training on rebalanced data moves the centering to $\mu'(x)$, but the importance-sampling corrected IS-ICRF restores centering at $\mu(x)$. For small $p$, the asymptotic variance of IS-ICRF is only $O(p)$ times that of the original ICRF, so the debiased rebalanced forest is provably more stable in exactly the regime where the minority class is rare.
Load-bearing premise
The variance-reduction conclusion rests on treating the rebalanced training set as fresh draws from a reweighted distribution, while the practical construction reweights one original sample and creates repetitions, an extension the paper explicitly leaves unproved.
Editorial extensions
If this is right
- Asymptotic confidence intervals for $\mu(x)$ can be built at the explicit rate $\sqrt{2^k/(n k^{(d-1)/2})}$.
- The hyperparameter conditions in (9) give a concrete guide for choosing subsample size and tree depth in practice.
- Training on a rebalanced dataset without correction is asymptotically biased; the debiasing step is necessary for valid inference about $\mu(x)$.
- In high-imbalance settings, the debiased rebalanced forest reduces asymptotic variance by a factor $O(p)$ compared to the original-data forest.
- Experiments suggest the same variance rates and variance reduction also hold for Breiman's random forests, not only centered forests.
Reading between the lines
- The explicit constant $C(d)$ invites a finite-sample variance estimator for Breiman forests, since the experiments show matching slopes though not matching intercepts.
- The i.i.d. assumption on rebalanced samples suggests the results cover undersampling without replacement but not oversampling or SMOTE-type duplicates; a CLT for resampling with replacement would be a natural extension.
- One could optimize the choice of the rebalanced minority proportion $p'$ to minimize the variance ratio for a fixed $p$, connecting this analysis to cost-sensitive learning.
- The variance-reduction mechanism may be generic: any base estimator whose CLT variance is of the form $\mathrm{const}\cdot \mu(x)(1-\mu(x))$ would inherit the same $O(p)$ reduction after this debiasing.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies the asymptotic normality of subsampling infinite centered random forests (ICRF), a purely random forest model in which splits occur at cell midpoints along uniformly chosen coordinates. The main technical results are a CLT for the ICRF around its own expectation (Theorem 2.4) and around the regression function μ(x) (Theorem 2.5), with an explicit limiting variance constant C(d) and explicit conditions linking the subsample size s, tree depth k, and sample size n. The paper then turns to imbalanced classification: it defines a rebalanced ICRF trained on an i.i.d. sample from a rebalanced distribution (X',Y'), introduces a debiased importance-sampling estimator IS-ICRF, proves CLTs for both (Theorem 3.2 and Corollary 3.4), and claims in Theorem 3.5 that for small minority probability p the IS-ICRF has asymptotic variance ratio O(p) relative to the ICRF, i.e., a variance reduction under high imbalance. Numerical experiments with Breiman's random forests illustrate the predicted variance decay rates and the variance reduction.
Significance. The ICRF portion of the paper is a solid contribution: Theorems 2.4–2.5 provide a U-statistic-style CLT with an explicit constant C(d), obtained through a detailed Hajek-projection variance computation (Proposition A.8) and a delicate kernel estimate (Proposition A.9). The paper supplies full proofs of the auxiliary lemmas and gives testable, explicit hyperparameter conditions, so the variance-rate predictions of Section 4 are falsifiable and indeed appear to agree qualitatively with simulations on Breiman's forests. The debiasing construction via the odds-ratio function g (Lemma B.2) and the Delta-method transfer (Corollary 3.4) are clean and potentially useful beyond this model. However, the headline application claim — the O(p) variance reduction of Theorem 3.5 — is not supported by the proofs in the setting in which it is stated, because the comparison relies on a reweighted sample that Section 3.4 itself concedes is outside the i.i.d. framework used to prove Corollary 3.4 and Proposition B.7. The ICRF CLTs are largely independent of this gap and should be assessed separately.
major comments (2)
- [Theorem 3.5 and Section 3.4] The stated comparison in Theorem 3.5 is not a consequence of the paper's theorems. Corollary 3.4 and Proposition B.7 are proved for an i.i.d. sample Z'1,...,Z'n' drawn from the rebalanced distribution (X',Y'), with CLT normalization sqrt(n'/(s^2 V'_{1,s})) in (21) and (30). Theorem 3.5 instead claims (36)–(37) with the same n in both normalizations, justified in Section 3.4 by assuming "both estimators are built using n observations" via reweighting; the same paragraph concedes that "this scenario departs from our theoretical analysis, as we do not allow repetitions in the observations." No theorem in the paper covers a reweighted (or otherwise dependent) sample, so (37) and (38) are unproven in the setting in which they are stated. This is not a harmless technicality: if one keeps the no-repetition i.i.d. construction, the rebalanced sample has size n' ≤ O(p n) (at most 2 p n when p' = 1/2), so the normalization in (21) carries a factor n'/n = Θ(p), and the variance ratio becomes (V'_{1,s} g'(μ')^2 / n') / (V_{1,s} / n) = O(p) · Θ(1/p) = Θ(1); the advertised O(p) reduction cancels. The variance-reduction claim therefore needs either a genuine CLT and variance analysis for the reweighted estimator, or a reformulated statement that does not assert the comparison in the reweighted setting.
- [Abstract, Section 1.2, and Theorem 2.4] The paper advertises "explicit convergence rates" and "exact rate of convergence" for the CLT. What Theorems 2.4–2.5 actually deliver is the magnitude of the sampling fluctuations, namely sqrt(2^k/(n k^{(d-1)/2})), or equivalently the reciprocal n^{1/2} k^{(d-1)/4}/2^{k/2}; this is the variance scale of the Hajek projection. The theorems do not provide any bound on the distance between the distribution of the normalized estimator and the Gaussian limit: no Berry–Esseen-type inequality is proved anywhere in the paper. The repeated phrasing "explicit convergence rate of the CLT" is therefore misleading and should be replaced by wording such as "explicit variance rate" unless a genuine distributional approximation bound is added.
minor comments (6)
- [Theorem 2.4 and Appendix A.4] The symbol N1 is overloaded: it denotes both the first component of the Gaussian vector and, in equation (83), the all-ones vector times the average N̄. The definition of Θ in (83) should use an explicitly defined mean vector to avoid confusion.
- [Lemma B.5, proof] The displayed Bayes computation for P(X ∈ L_U(x) | U, Y = 1) is garbled; a division by P(Y = 1 | U) appears to be missing. The final formula is correct, but the intermediate display should be corrected.
- [Assumption (H3)] Assumption (H3) postulates the existence of p'' such that the mixture p'' f_{X|Y=1} + (1-p'') f_{X|Y=0} is exactly uniform on [0,1]^d. This is a strong structural condition; no example or discussion of its plausibility is given, and its relationship to (H0) is not explained. Since Theorem 3.5 depends on it, a comment on when it holds (or a relaxation to bounded densities) would help the reader.
- [Section 3.1] The suggested construction "draw n' ≤ n1 samples from the original samples without replacement" produces an exchangeable sample, not an exactly i.i.d. sample from (X',Y'); a remark clarifying that the i.i.d. assumption in the theory is an idealization that is approximately valid when n' is small relative to n0 and n1 would be useful.
- [Definition 3.3 and Corollary 3.4] The IS-ICRF estimator (28) plugs in n0 and n1 for the unknown (1-p) and p, while the Delta-method proof of Corollary 3.4 treats p as known. A sentence noting that consistency of n1/n makes this substitution asymptotically negligible would remove a small gap.
- [Section 4, Figure 1] The claim that curves with the same β have the same slope rests on visual inspection; since the theory predicts slopes −(1−β) independent of α, reporting estimated slopes with standard errors would make the comparison quantitative.
Circularity Check
No circular reduction: the ICRF CLT rests on a self-cited general theorem whose conditions are verified independently in this paper, and Theorem 3.5's gap is an unproven extension, not a circularity.
full rationale
The central ICRF CLT (Theorems 2.4-2.5) is obtained by applying Theorem A.1, restated from Mayala et al. (2024) with substantial author overlap, and then verifying its conditions. This is a load-bearing self-citation, but the verification is performed in the present paper: Proposition A.8 computes V1^s ~ C(d)2^k/(s^2 k^{(d-1)/2}) mu(x)(1-mu(x)), Proposition A.9 (proved in the appendix, refining Arnould et al. 2023) supplies the kernel constant, and Proposition A.7 controls the bias. The CLT is therefore not assumed as its own conclusion. The debiased IS-ICRF CLT (Corollary 3.4) follows from the Delta method applied to the odds-ratio identity (Lemma B.2), and Theorem 3.5's O(p) variance reduction is derived from the explicit variance bounds of Proposition B.7 together with the explicit factor g'(mu'(x))^2; no fitted parameter is renamed as a prediction. The only load-bearing assumption not proved is in Theorem 3.5, where the comparison uses both estimators built from n reweighted observations, a scenario the paper itself flags as departing from the i.i.d. rebalanced-sample theory ('this scenario departs from our theoretical analysis, as we do not allow repetitions in the observations'). That is a correctness/extension gap rather than a circular reduction: the variance ratio is computed inside the i.i.d. rebalanced model, not derived from the reweighted estimator's own variance. No exhibited equation reduces to its own input, so the paper sits in the 0-2 no-significant-circularity band.
Assumptions & free parameters
assumptions (6)
- domain assumption (H0) X is uniformly distributed on [0,1]^d
- domain assumption (H1) The regression function mu is L-Lipschitz with respect to the max norm
- domain assumption (G1) s/(k 2^k) -> infinity
- standard math Theorem A.1, the U-statistic CLT from Mayala et al. (2024), is assumed without proof
- domain assumption Rebalanced samples Z'_i are i.i.d. from the distribution defined in (14)
- ad hoc to paper (H3) Existence of p'' making f_{X''} uniform, plus bounded Lipschitz conditional densities
Cite this review
Pith. "Pith review of Asymptotic Normality of Infinite Centered Random Forests -Application to Imbalanced Classification." pith.science (2026). https://pith.science/paper/VW2BENBM
@misc{pith2026250608548,
author = {Pith},
title = {Pith review of: Asymptotic Normality of Infinite Centered Random Forests -Application to Imbalanced Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/VW2BENBM}},
note = {Machine review of arXiv:2506.08548}
}
read the original abstract
Many classification tasks involve imbalanced data, in which a class is largely underrepresented. Several techniques consists in creating a rebalanced dataset on which a classifier is trained. In this paper, we study theoretically such a procedure, when the classifier is a Centered Random Forests (CRF). We establish a Central Limit Theorem (CLT) on the infinite CRF with explicit rates and exact constant. We then prove that the CRF trained on the rebalanced dataset exhibits a bias, which can be removed with appropriate techniques. Based on an importance sampling (IS) approach, the resulting debiased estimator, called IS-ICRF, satisfies a CLT centered at the prediction function value. For high imbalance settings, we prove that the IS-ICRF estimator enjoys a variance reduction compared to the ICRF trained on the original data. Therefore, our theoretical analysis highlights the benefits of training random forests on a rebalanced dataset (followed by a debiasing procedure) compared to using the original data. Our theoretical results, especially the variance rates and the variance reduction, appear to be valid for Breiman's random forests in our experiments.
Figures
Reference graph
Works this paper leans on
-
[1]
Ludovic Arnould, Claire Boyer, and Erwan Scornet. Is interpolation benign for random forest regression? In International Conference on Artificial Intelligence and Statistics, pages 5493--5548. PMLR, 2023
work page 2023
-
[2]
Analysis of a random forests model
G \'e rard Biau. Analysis of a random forests model. The Journal of Machine Learning Research, 13: 0 1063--1095, 2012
work page 2012
-
[3]
G \'e rard Biau and Luc Devroye. On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification. Journal of Multivariate Analysis, 101 0 (10): 0 2499--2518, 2010
work page 2010
-
[4]
G \'e rard Biau and Erwan Scornet. A random forest guided tour. Test, 25: 0 197--227, 2016
work page 2016
-
[5]
L. Breiman. Consistency for a simple model of random forests. 2004. URL https://api.semanticscholar.org/CorpusID:123042984
work page 2004
-
[6]
Random forests
Leo Breiman. Random forests. Machine learning, 45 0 (1): 0 5--32, 2001
2001
-
[7]
Smote: synthetic minority over-sampling technique
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16: 0 321--357, 2002
2002
-
[8]
Using random forest to learn imbalanced data
Chao Chen, Andy Liaw, Leo Breiman, et al. Using random forest to learn imbalanced data. University of California, Berkeley, 110 0 (1-12): 0 24, 2004
work page 2004
Show all 38 references
-
[9]
A note on inverse moments of binomial variates
Francisco Cribari-Neto, Nancy Lopes Garcia, and Klaus LP Vasconcellos. A note on inverse moments of binomial variates. Brazilian Review of Econometrics, 20 0 (2): 0 269--277, 2000
2000
-
[10]
Supervised machine learning algorithms for credit card fraudulent transaction detection: a comparative study
Sahil Dhankhad, Emad Mohammed, and Behrouz Far. Supervised machine learning algorithms for credit card fraudulent transaction detection: a comparative study. In 2018 IEEE international conference on information reuse and integration (IRI), pages 122--125. IEEE, 2018
2018
-
[11]
Synthetic tabular data generation for imbalanced classification: The surprising effectiveness of an overlap class, 2025
Annie D'souza, Swetha M, and Sunita Sarawagi. Synthetic tabular data generation for imbalanced classification: The surprising effectiveness of an overlap class, 2025
2025
-
[12]
Do we need hundreds of classifiers to solve real world classification problems? The journal of machine learning research, 15 0 (1): 0 3133--3181, 2014
Manuel Fern \'a ndez-Delgado, Eva Cernadas, Sen \'e n Barro, and Dinani Amorim. Do we need hundreds of classifiers to solve real world classification problems? The journal of machine learning research, 15 0 (1): 0 3133--3181, 2014
2014
-
[13]
Variance reduction in purely random forests
Robin Genuer. Variance reduction in purely random forests. Journal of Nonparametric Statistics, 24 0 (3): 0 543--562, 2012
2012
-
[14]
Tabllm: Few-shot classification of tabular data with large language models
Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, and David Sontag. Tabllm: Few-shot classification of tabular data with large language models. In International Conference on Artificial Intelligence and Statistics, pages 5549--5581. PMLR, 2023
2023
-
[15]
A class of statistics with asymptotically normal distribution
Wassily Hoeffding. A class of statistics with asymptotically normal distribution. Ann. Math. Statist., 19 0 (4): 0 293--325, 1948
1948
-
[16]
Sharp analysis of a simple model for random forests
Jason Klusowski. Sharp analysis of a simple model for random forests. In International Conference on Artificial Intelligence and Statistics, pages 757--765. PMLR, 2021
2021
-
[17]
Tabddpm: Modelling tabular data with diffusion models
Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. Tabddpm: Modelling tabular data with diffusion models. In International Conference on Machine Learning, pages 17564--17579. PMLR, 2023
2023
-
[18]
Learning from imbalanced data: open challenges and future directions
Bartosz Krawczyk. Learning from imbalanced data: open challenges and future directions. Progress in artificial intelligence, 5 0 (4): 0 221--232, 2016
2016
-
[19]
U-statistics: Theory and Practice
A J Lee. U-statistics: Theory and Practice. Routledge, 2019
2019
-
[20]
Early failure detection of paper manufacturing machinery using nearest neighbor-based feature extraction
Wonjae Lee and Kangwon Seo. Early failure detection of paper manufacturing machinery using nearest neighbor-based feature extraction. Engineering Reports, 3 0 (2): 0 e12291, 2021
2021
-
[21]
Downsampling for binary classification with a highly imbalanced dataset using active learning
Wonjae Lee and Kangwon Seo. Downsampling for binary classification with a highly imbalanced dataset using active learning. Big Data Research, 28: 0 100314, 2022
2022
-
[22]
Addressing the class imbalance problem in twitter spam detection using ensemble learning
Shigang Liu, Yu Wang, Jun Zhang, Chao Chen, and Yang Xiang. Addressing the class imbalance problem in twitter spam detection using ensemble learning. Computers & Security, 69: 0 35--49, 2017
2017
-
[23]
Infinite random forests for imbalanced classification tasks
Moria Mayala, Olivier Wintenberger, Charles Tillier, and Cl \'e ment Dombry. Infinite random forests for imbalanced classification tasks. arXiv preprint arXiv:2408.01777, 2024
2024 arXiv
-
[24]
Machine learning for imbalanced datasets: Application in medical diagnostic
Luis J Mena and Jesus A Gonzalez. Machine learning for imbalanced datasets: Application in medical diagnostic. In FLAIRS, pages 574--579, 2006
2006
-
[25]
Quantifying uncertainty in random forests via confidence intervals and hypothesis tests
Lucas Mentch and Giles Hooker. Quantifying uncertainty in random forests via confidence intervals and hypothesis tests. The Journal of Machine Learning Research, 17 0 (1): 0 841--881, 2016
2016
-
[26]
Forecasting cyber attacks with imbalanced data sets and different time granularities
Ahmet Okutan, Shanchieh Jay Yang, and Katie McConky. Forecasting cyber attacks with imbalanced data sets and different time granularities. arXiv preprint arXiv:1803.09560, 2018
2018 arXiv
-
[27]
A precise local limit theorem for the multinomial distribution and some applications
Fr \'e d \'e ric Ouimet. A precise local limit theorem for the multinomial distribution and some applications. Journal of Statistical Planning and Inference, 215: 0 218--233, 2021
2021
-
[28]
A random forests quantile classifier for class imbalanced data
Robert O’Brien and Hemant Ishwaran. A random forests quantile classifier for class imbalanced data. Pattern recognition, 90: 0 232--249, 2019
2019
-
[29]
Rates of convergence for random forests via generalized u-statistics
Wei Peng, Tim Coleman, and Lucas Mentch. Rates of convergence for random forests via generalized u-statistics. Electronic Journal of Statistics, 16 0 (1): 0 232--292, 2022
2022
-
[30]
Imbalanced dataset classification and solutions: a review
Duraisamy Ramyachitra and Parasuraman Manikandan. Imbalanced dataset classification and solutions: a review. International Journal of Computing and Business Research (IJCBR), 5 0 (4): 0 1--29, 2014
2014
-
[31]
On the asymptotics of random forests
Erwan Scornet. On the asymptotics of random forests. Journal of Multivariate Analysis, 146: 0 72--83, 2016 a
2016
-
[32]
Random forests and kernel methods
Erwan Scornet. Random forests and kernel methods. IEEE Transactions on Information Theory, 62 0 (3): 0 1485--1500, 2016 b
2016
-
[33]
Theory of random forests: A review
Erwan Scornet and Giles Hooker. Theory of random forests: A review. 2025. URL https://hal.science/hal-05006431/
2025
-
[34]
Asymptotic approximations for the distributions of multinomial goodness-of-fit statistics
Minoru Siotani and Yasunori Fujikoshi. Asymptotic approximations for the distributions of multinomial goodness-of-fit statistics. Hiroshima mathematical journal, 14 0 (1): 0 115--124, 1984
1984
-
[35]
Estimation and inference of heterogeneous treatment effects using random forests
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113 0 (523): 0 1228--1242, 2018
2018
-
[36]
ranger: A fast implementation of random forests for high dimensional data in c++ and r
Marvin N Wright and Andreas Ziegler. ranger: A fast implementation of random forests for high dimensional data in c++ and r. Journal of statistical software, 77: 0 1--17, 2017
2017
-
[37]
Modeling tabular data using conditional gan
Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Modeling tabular data using conditional gan. Advances in neural information processing systems, 32, 2019
2019
-
[38]
V-statistics and variance estimation
Zhengze Zhou, Lucas Mentch, and Giles Hooker. V-statistics and variance estimation. Journal of Machine Learning Research, 22 0 (287): 0 1--48, 2021
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.