Pith. sign in

REVIEW 3 major objections 5 minor 114 references

Feature bagging stabilizes predictions against removing features, and a modest number of rounds is enough.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 15:36 UTC pith:HRNEEYOP

load-bearing objection Clean feature-axis dual of bagging stability with solid linear asymptotics; the RF story is real but only proved for shallow dyadic/orthogonal special cases. the 3 major comments →

arxiv 2607.26964 v1 pith:HRNEEYOP submitted 2026-07-29 stat.ML cs.LGmath.STstat.TH

Feature Bagging Provides Stability

Yuheng Ma , Qiang Sun This is my paper
classification stat.ML cs.LGmath.STstat.TH
keywords stabilitybaggingfeature instabilityrandom forestsrandom forward selectionalgorithmic stabilityfeature subsampling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that randomly training base learners on feature subsets and averaging them—feature bagging—makes algorithms less sensitive to which features are present. The authors introduce feature instability (FI), which measures how much predictions change when one feature is dropped, as the natural counterpart to classical instance instability. They show empirically that FI carries generalization-relevant information that instance instability does not capture alone. In linear regression they give exact asymptotic formulas for how bagging drives both instabilities down, and in a model-free setting covering random forward selection and simplified random forests they prove that feature bagging improves FI relative to the non-bagged baseline, with stronger gains under more aggressive feature subsampling. They also show that a number of bagging rounds on the order of the feature dimension already approaches the infinite-ensemble stability level. A sympathetic reader cares because feature availability is often messy in practice, and this gives a stability reason—beyond accuracy folklore—why random forests and related ensembles work.

Core claim

Feature bagging improves the relevant stability relative to its non-bagged counterpart: in linear models it replaces large same-submodel instability with small cross-submodel terms of order 1/n, and in recursive feature-subsampling algorithms it yields FI bounds of order q/((d−1)(1−q)) times a recursion factor that beat the non-bagged 2T/d baseline in shallow regimes, with finite B ≳ d(1−q)/q already approaching the infinite-bagging level.

What carries the argument

Feature instability (FI): the average squared change in prediction when one feature is removed (the feature-axis analogue of leave-one-instance-out instability). In the model-free theory, a recursive peeling argument under a one-step inflation bound turns one-shot feature-bagging stability into multi-step guarantees for random forward selection and dyadic random forests.

Load-bearing premise

The recursive guarantees need each update step not to amplify path differences too much, and the forest analysis is proved for simplified midpoint dyadic splits rather than full data-dependent tree splits.

What would settle it

Train shallow random forward selection or max-edge dyadic forests with and without feature bagging on the paper’s orthogonal-design or MARSadd setups; if measured FI does not fall below the non-bagged 2T/d baseline for small q and T ≲ 1/q, or does not approach the infinite-B curve once B is a few times d(1−q)/q, the central stability claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • More aggressive feature subsampling (smaller q) should measurably lower FI, not only diversify trees.
  • For tabular problems with d ≪ n, far fewer bagging rounds are needed for feature stability than classical instance bagging would require for instance stability.
  • FI and II together explain more of the train–test gap than either alone, so monitoring FI is a useful diagnostic when features may be missing or corrupted.
  • In linear sketched ensembles, bagging flattens the double-descent instability peak near the effective interpolation threshold γq = p.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the one-step inflation condition holds for full CART with greedy thresholds, the same FI story would justify feature randomness in production random forests, not only dyadic proxies.
  • Practitioners tuning max_features could treat FI (or a cheap proxy) as a regularizer signal complementary to validation error when features are unstable or costly to acquire.
  • The duality with instance bagging suggests hybrid designs that allocate ensemble budget between the n-axis and d-axis according to which perturbation axis dominates risk.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies feature bagging through algorithmic stability. It introduces feature instability (FI), the leave-one-feature-out analogue of classical instance instability (II), and argues empirically that FI carries generalization-relevant information complementary to II. Theoretically, it gives two complementary analyses: (i) exact proportional-asymptotic characterizations of II and FI for bagged sketched least squares under an affected-axis resampling coupling (Theorem 3.1, Table 1), showing that bagging replaces same-submodel O(1) instability peaks by O(1/n) cross-submodel limits; and (ii) model-free FI bounds for infinite and finite feature bagging in Hilbert space (Propositions 4.1–4.3, 4.8), extended via a peel/pith recursion to random forward selection under orthogonal design (Theorem 4.5) and to max-edge dyadic random forests (Theorem 4.7). The main message is that feature bagging improves the relevant stability relative to non-bagged baselines, more aggressively under smaller feature-subsampling ratios q, and that B on the order of d(1−q)/q rounds approaches the infinite-bagging level.

Significance. If the results hold as stated, the paper makes a genuine contribution: FI is a clean, independently defined object that dualizes the recent instance-bagging stability program (Soloff et al.), and the linear-model asymptotics are sharp, simulation-matched, and informative about how p and q enter asymmetrically. The one-shot model-free bound and the finite-B concentration argument are useful and practically relevant when d ≪ n. Strengths include closed-form limits (Table 1), explicit non-bagged baselines 2T/d, noise-controlled PMI experiments supporting complementarity of FI and II, and a careful separation of shared-seed versus affected-axis couplings. The recursive RF theory is more limited in scope than the RF-motivated framing suggests, but within its stated model it is a legitimate first step. Overall this is a solid stability-theory paper with a clear conceptual advance.

major comments (3)
  1. [§4.2.2, Theorem 4.7] §4.2.2, Theorem 4.7 and Appendix S.3.4: the bagged FI bound is q A_T²/((d−1)(1−q)) with A_T = Σ_t (1+√2)^{T−t} = Θ((1+√2)^T), while the non-bagged dyadic baseline is 2T/d. The claimed strict improvement over the non-bagged counterpart therefore holds only for shallow T and sufficiently small q; for moderate depths the exponential factor dominates. The paper notes a “shallow-tree” caveat, but the abstract and introduction still present recursive RF-inspired bagging as improving stability in parallel with the linear case. Please state explicitly (e.g., a short corollary or plot of the (T,q,d) region) when the bagged bound is smaller than 2T/d, and soften abstract/intro language so that the proved RF regime is not read as covering practical CART depths or data-dependent thresholds.
  2. [§4.2.1, Theorem 4.5] §4.2.1, Theorem 4.5: the RFS contraction (δ_t = (1/4)max{q(1+q), q²t}) and the reduction β^(b)=E_w β rely on the orthogonal design X⊤X = n I_d. Under non-orthogonal designs the residual-gain identity (S.8) fails and the pathwise difference structure used in the auxiliary fact need not hold, so the comparison to non-bagged 2T/d is not justified off the orthogonal case. Either extend the argument under a restricted eigenvalue / incoherence condition, or clearly label Theorem 4.5 as an orthogonal-design specialization and avoid implying a general RFS guarantee in the main narrative.
  3. [§2.1, §3, §4] §3 vs §4 coupling conventions: Section 3 uses affected-axis resampling (ξ_j ≠ ξ) and reports distributional E[∥β−β_ℓ∥²], while Section 4 uses shared-seed conditional stability (Definitions 2.1–2.2). Both are defensible, but FI numbers are not comparable across sections, and “feature bagging improves FI” means different things in the two settings. A short remark in §2.1 or the start of §4 quantifying how the two couplings relate (e.g., triangle inequality / extra resampling variance) would prevent misreading Theorem 3.1’s FI limits as the same object bounded in Propositions 4.3–4.4.
minor comments (5)
  1. [Figures 2–3, Theorem 3.1] Figure 2–3 captions and Table 1: the FI panels are labeled on an n⁻¹ scale in some places and raw scale in others; unify the scaling notation so that the O(1/n) vs O(1/d) discussion after Theorem 3.1 is easy to match to the plots.
  2. [Proposition 4.8] Proposition 4.8: the displayed inequality mixes a sum of norms with an average; the √3 factor and the passage from Hilbert Hoeffding to the averaged FI bound would be clearer if the left-hand side were written consistently as (1/d)Σ_j ∥·∥² or as an ℓ2-of-averages form throughout.
  3. [Front matter, §4.2] Typos / duplicates: author line “Y uheng Ma”; Keywords block repeated twice; “Date: July 29, 2026” looks like a placeholder; “one-step1” footnote marker in §4.2; “γ↓0” asymptotic remark after Theorem 3.1 is easy to misread.
  4. [§2.2, Appendix S.1] §2.2 / Appendix S.1: the PMI analysis is a useful robustness check, but the main text could cite the noise-control tables (S.6–S.9) in one sentence so readers see that the complementary-FI claim is not an artifact of an extra correlated covariate.
  5. [§1.1] Related work: briefly situate against Mentch & Zhou (2020) and Curth et al. (2024) on RF as regularization, beyond the citations already present, to clarify what stability adds over risk/df explanations.

Circularity Check

0 steps flagged

No load-bearing circularity: FI/II are defined independently and stability bounds are derived from sampling geometry and stated contraction assumptions, not fitted to the target.

full rationale

The paper’s central claims are asymptotic limits (Thm. 3.1) and upper bounds (Props. 4.1–4.4, Thms. 4.5/4.7, Prop. 4.8) for instance and feature instability under bagging. FI and II are introduced as leave-one-out sensitivity measures (Defs. 2.1–2.3) without reference to the bagging conclusions. Linear-model limits follow from explicit variance/bias decompositions of sketched min-norm OLS under proportional asymptotics, with closed forms in Table 1 and full matrix-expectation lemmas in App. S.2—not by calibrating free constants to observed instability. Model-free bounds dualize bounded-output averaging (cf. Soloff et al.) to the feature axis and then peel recursive updates under an explicit one-step inflation condition (4.3) and Hilbert radius (Assump. 1); non-bagged baselines 2T/d are derived separately (App. S.3.4) and compared, not assumed equal to the bagged object. Empirical PMI analyses regress held-out generalization gaps on measured FI/II with noise controls; they do not define FI to match the gap. Self-citations (e.g. Wu & Sun 2025 for sketched bagged LS setup; Ma et al. for application context) supply background tools or motivation and are not used as uniqueness theorems that force the FI improvement. Minor author-overlap citation is normal background, not a self-citation chain that collapses the derivation. Score 1 only for that non-load-bearing overlap.

Axiom & Free-Parameter Ledger

3 free parameters · 7 axioms · 3 invented entities

Load-bearing content is standard probability/linear algebra plus domain modeling choices (Gaussian linear model, proportional asymptotics, bounded Hilbert outputs, one-step path Lipschitzness, orthogonal design, dyadic forests). No fitted physical constants drive the theorems; free choices are algorithmic hyperparameters (p,q,B,T) appearing in the bounds.

free parameters (3)
  • feature subsampling ratio q and instance ratio p = varied, e.g. q∈{0.05..0.5}, p∈{0.4..1}
    Algorithmic knobs that enter all leading constants and the regime split γq ≶ p; not fitted to prove the theorem but chosen in experiments.
  • bagging rounds B and recursion depth T = B=256 in main sims; T grid
    Finite-B theory needs B ≳ d(1-q)/q; recursive bounds grow with T. Experimental defaults B=256, T≤20/7 are design choices.
  • MARSadd mixing ν and RF hyperparameter grid = ν∈{0..1}; max_features/samples/depth grids
    Control synthetic signal composition and model complexity for PMI/generalization experiments only; not used in formal stability theorems.
axioms (7)
  • domain assumption Gaussian isotropic linear model yi=x_i^T β*+ε with β*~N(0,I/d), d/n→γ proportional asymptotics (universality claimed under moments).
    Section 3.1; enables exact II/FI limits in Theorem 3.1 and Table 1.
  • ad hoc to paper Affected-axis resampling coupling: after removing an instance/feature, redraw sketching on the perturbed axis only.
    Section 3.1–3.2 Definitions; FI/II expectations are under this coupling, not pure shared-seed removal.
  • domain assumption Outputs lie in a closed convex Hilbert set W with finite radius rad(W) (bounded predictions in scalar case).
    Assumption 1 / Prop 4.1–4.3; standard for bagging stability à la Soloff et al.
  • ad hoc to paper One-step expected transition inflation ||pith_t|| ≤ (1+δ_t)||E Δ_{t-1}|| for recursive maps.
    Eq. (4.3); load-bearing for Prop 4.4 and all recursive theorems.
  • domain assumption Orthogonal design X^T X = n I_d for random forward selection specialization.
    Section 4.2.1; reduces estimator stability to selection-frequency vector w.
  • domain assumption Max-edge dyadic midpoint splitting and local path encoding for random forests; additive uniform design for affine prediction map K(w).
    Section 4.2.2, Prop 4.6, Thm 4.7; simplifies full CART.
  • standard math Uniform fixed-size sampling without replacement for sub-bagging; Moore–Penrose ridgeless base learners in the linear section.
    Section 2.3 and 3.1; standard ensemble constructions.
invented entities (3)
  • Feature instability (FI) / φ-feature stability independent evidence
    purpose: Feature-axis analogue of instance instability to measure leave-one-feature-out prediction sensitivity.
    Definition 2.2–2.3; central conceptual contribution. Operational and falsifiable via retraining, not a hidden physical entity.
  • Coupled / affected-axis resampling feature stability no independent evidence
    purpose: Handle seed spaces that change when a feature is removed under sketching.
    Definition 2.3 and Section 3 coupling; technical device for linear analysis.
  • Local feature-weight matrix encoding w for dyadic forest paths no independent evidence
    purpose: Reduce local RF prediction stability to Frobenius stability of split-count encodings.
    Section 4.2.2 and Prop 4.6 affine map K; analysis device for simplified forests.

pith-pipeline@v1.2.0-daily-grok45 · 65748 in / 3714 out tokens · 73920 ms · 2026-07-30T15:36:09.504498+00:00 · methodology

0 comments
read the original abstract

We study feature bagging through the lens of algorithmic stability. Feature bagging is an ensemble strategy that aggregates base learners trained on randomly subsampled feature subsets, possibly in a data-dependent manner. We introduce feature instability (FI), the feature-axis analogue of instance instability (II), which measures sensitivity to removing a single feature. Smaller values of II or FI correspond to stronger stability, and our experiments show that FI captures generalization-relevant information complementary to II. Within this framework, we analyze feature bagging in both a parametric linear model and a model-free setting inspired by recursive feature subsampling in random forests. In both settings, we establish formal guarantees showing that feature bagging improves the relevant stability relative to its non-bagged counterpart, with larger improvements under more aggressive subsampling. We further show that a modest number of bagging rounds is sufficient to approach the infinite-bagging stability level.

Figures

Figures reproduced from arXiv: 2607.26964 by Qiang Sun, Yuheng Ma.

Figure 1
Figure 1. Figure 1: Illustration of subsampling and bagging schemes. In the random forest example (d), all [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Instability of the single subsampled minimum-norm OLS estimator as a function of the aspect [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Instability E[∥β − β ℓ ∥ 2 2 ] of the bagged least-squares estimator. Panels (a)–(d) use B = 200 bagging rounds and vary the aspect ratio γ; solid lines denote theoretical predictions, crosses mark empirical averages, and dashed curves show the infinite-bagging limit. Panels (e)–(f) compare different numbers of bagging rounds on an n −1 scale; colors indicate B, solid lines denote theoretical predictions, … view at source ↗
Figure 4
Figure 4. Figure 4: Left: Feature instability of feature-bagged random forward selection and non-bagged baseline [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Two possibilities of dyadic tree partition. The black dot represents the target test sample [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of feature instability across di [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

114 extracted references · 1 canonical work pages

  1. [1]

    arXiv preprint arXiv:2410.18268 , year=

    Stabilizing black-box model selection with the inflated argmax , author=. arXiv preprint arXiv:2410.18268 , year=

    arXiv

  2. [2]

    2013 , Owner =

    Lq Averaging for Symmetric Positive-Definite Matrices , Author =. 2013 , Owner =

    2013

  3. [3]

    The 11th International Conference on Learning Representations , year=

    Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning , author=. The 11th International Conference on Learning Representations , year=

  4. [4]

    Transactions on Machine Learning Research , issn=

    Sparse MoEs meet Efficient Ensembles , author=. Transactions on Machine Learning Research , issn=. 2022 , note=

    2022

  5. [5]

    Proceedings of the 26th International Conference on Artificial Intelligence and Statistics , year=

    Is interpolation benign for random forest regression? , author=. Proceedings of the 26th International Conference on Artificial Intelligence and Statistics , year=

  6. [6]

    SIAM Journal of Matrix Analysis and Applications , Year =

    Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , Author =. SIAM Journal of Matrix Analysis and Applications , Year =

  7. [7]

    The Annals of Statistics , volume=

    Predictive inference with the jackknife+ , author=. The Annals of Statistics , volume=. 2021 , publisher=

    2021

  8. [8]

    Large sample theory of intrinsic and extrinsic sample means on manifolds

    Bhattacharya, Rabi and Patrangenaru, Vic , Journal =. Large sample theory of intrinsic and extrinsic sample means on manifolds. 2003 , Number =. arXiv , Arxivid =:0507423 , File =

    2003

  9. [9]

    Large sample theory of intrinsic and extrinsic sample means on manifolds

    Bhattacharya, Rabi and Patrangenaru, Vic , Journal =. Large sample theory of intrinsic and extrinsic sample means on manifolds. 2005 , Number =. arXiv , Arxivid =:0507423 , File =

    2005

  10. [10]

    Rabi Bhattacharya and Lizhen Lin , Journal =. Omnibus. 2017 , Pages =

    2017

  11. [11]

    , author=

    On the Rate of Convergence of the Bagged Nearest Neighbor Estimate. , author=. Journal of Machine Learning Research , volume=

  12. [12]

    Machine Learning , volume=

    Optimal dyadic decision trees , author=. Machine Learning , volume=. 2007 , publisher=

    2007

  13. [13]

    2013 , publisher=

    Boucheron, St. 2013 , publisher=. doi:10.1093/acprof:oso/9780199535255.001.0001 , isbn=

    arXiv 2013

  14. [14]

    Journal of Machine Learning Research , volume=

    Stability and generalization , author=. Journal of Machine Learning Research , volume=

  15. [15]

    Proceedings of the 33rd Conference on Learning Theory , year=

    Sharper bounds for uniformly stable algorithms , author=. Proceedings of the 33rd Conference on Learning Theory , year=

  16. [16]

    Machine Learning , volume=

    Bagging predictors , author=. Machine Learning , volume=. 1996 , publisher=

    1996

  17. [17]

    Machine Learning , volume=

    Random forests , author=. Machine Learning , volume=. 2001 , publisher=

    2001

  18. [18]

    and Olshen, Richard A

    Breiman, Leo and Friedman, Jerome H. and Olshen, Richard A. and Stone, Charles J. , series=. 1984 , publisher=

    1984

  19. [19]

    Proceedings of the 40th International Conference on Machine Learning , year=

    Extrapolated Random Tree for Regression , author=. Proceedings of the 40th International Conference on Machine Learning , year=

  20. [20]

    The Annals of Statistics , volume=

    Transfer learning for contextual multi-armed bandits , author=. The Annals of Statistics , volume=. 2024 , publisher=

    2024

  21. [21]

    Bagged Regularized

    Cai, Yuchao and Yang, Hanfang and Ma, Yuheng and Hang, Hanyuan , journal=. Bagged Regularized

  22. [22]

    Journal of Applied Statistics , volume=

    Handling high-dimensional data with missing values by modern machine learning techniques , author=. Journal of Applied Statistics , volume=. 2023 , publisher=

    2023

  23. [23]

    Proceedings of the 40th International Conference on Machine Learning , year=

    Sketched ridgeless linear regression: The role of downsampling , author=. Proceedings of the 40th International Conference on Machine Learning , year=

  24. [24]

    2025 , eprint=

    Revisiting Randomization in Greedy Model Search , author=. 2025 , eprint=

    2025

  25. [25]

    , booktitle=

    Cheng, Guang and Ho, Jeffrey and Salehian, Hesamoddin and Vemuri, Baba C. , booktitle=. Recursive computation of the. 2016 , publisher=. doi:10.1007/978-3-319-22957-7_2 , isbn=

    work page doi:10.1007/978-3-319-22957-7_2 2016

  26. [26]

    2003 , publisher=

    Chikuse, Yasuko , series=. 2003 , publisher=

    2003

  27. [27]

    Journal of Machine Learning Research , volume=

    Dropout Regularization Versus l2-Penalization in the Linear Model , author=. Journal of Machine Learning Research , volume=

  28. [28]

    Journal of the Royal Statistical Society: Series B (Statistical Methodology) , Year =

    Regression models on Riemannian symmetric spaces , Author =. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , Year =

  29. [29]

    Advances in Neural Information Processing Systems , year=

    A u-turn on double descent: Rethinking parameter counting in statistical learning , author=. Advances in Neural Information Processing Systems , year=

  30. [30]

    Why do random forests work?

    Curth, Alicia and Jeffares, Alan and van der Schaar, Mihaela , journal=. Why do random forests work?

  31. [31]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Moving beyond population variable importance: concept, theory and applications of individual variable importance , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

  32. [32]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    A Fourier approach to nonparametric deconvolution of a density estimate , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 1993 , publisher=

    1993

  33. [33]

    ESAIM: Probability and Statistics , volume=

    Impact of subsampling and tree depth on random forests , author=. ESAIM: Probability and Statistics , volume=. 2018 , publisher=

    2018

  34. [34]

    Journal of Machine Learning Research , volume=

    Stability of randomized learning algorithms , author=. Journal of Machine Learning Research , volume=

  35. [35]

    Journal of the American Statistical Association , volume=

    Variable selection via nonconcave penalized likelihood and its oracle properties , author=. Journal of the American Statistical Association , volume=. 2001 , publisher=

    2001

  36. [36]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Sure independence screening for ultrahigh dimensional feature space , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2008 , publisher=

    2008

  37. [37]

    Advances in Neural Information Processing Systems , year=

    Generalization bounds for uniformly stable algorithms , author=. Advances in Neural Information Processing Systems , year=

  38. [38]

    Proceedings of the 32nd Conference on Learning Theory , year=

    High probability generalization bounds for uniformly stable algorithms with nearly optimal rate , author=. Proceedings of the 32nd Conference on Learning Theory , year=

  39. [39]

    Thomas Fletcher , Journal =

    P. Thomas Fletcher , Journal =. Geodesic Regression and the Theory of Least Squares on. 2013 , Number =

    2013

  40. [40]

    The Annals of Statistics , volume=

    Multivariate adaptive regression splines , author=. The Annals of Statistics , volume=. 1991 , publisher=

    1991

  41. [41]

    2016 , Owner =

    Notes on Differential Geometry and Lie Groups , Author =. 2016 , Owner =

    2016

  42. [42]

    Artificial Intelligence , volume=

    Towards convergence rate analysis of random forests for classification , author=. Artificial Intelligence , volume=

  43. [43]

    Machine Learning , volume=

    Extremely randomized trees , author=. Machine Learning , volume=. 2006 , publisher=

    2006

  44. [44]

    2014 , publisher=

    Gradshteyn, Izrail Solomonovich and Ryzhik, Iosif Moiseevich , edition=. 2014 , publisher=

    2014

  45. [45]

    Journal of Multivariate Analysis , volume=

    An identity for the Wishart distribution with applications , author=. Journal of Multivariate Analysis , volume=. 1979 , publisher=

    1979

  46. [46]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Neural network ensembles , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 1990 , publisher=

    1990

  47. [47]

    Proceedings of the 33rd International Conference on Machine Learning , year=

    Train faster, generalize better: Stability of stochastic gradient descent , author=. Proceedings of the 33rd International Conference on Machine Learning , year=

  48. [48]

    Proceedings of the 20th International Conference on Machine Learning , year=

    Online ranking/collaborative filtering using the perceptron algorithm , author=. Proceedings of the 20th International Conference on Machine Learning , year=

  49. [49]

    The Annals of Statistics , volume=

    Surprises in high-dimensional ridgeless least squares interpolation , author=. The Annals of Statistics , volume=

  50. [50]

    Journal of Business & Economic Statistics , volume =

    Haoze Hou and Wei Huang and Zheng Zhang , title =. Journal of Business & Economic Statistics , volume =

  51. [51]

    The Annals of Applied Statistics , volume=

    Weak signal inclusion under dependence and applications in genome-wide association study , author=. The Annals of Applied Statistics , volume=. 2024 , publisher=

    2024

  52. [52]

    Statistics and Computing , volume=

    Forward stability and model path selection , author=. Statistics and Computing , volume=. 2024 , publisher=

    2024

  53. [53]

    Proceedings of the 24th International Conference on Artificial Intelligence and Statistics , year=

    Sharp analysis of a simple model for random forests , author=. Proceedings of the 24th International Conference on Artificial Intelligence and Statistics , year=

  54. [54]

    Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence , year=

    Almost-everywhere algorithmic stability and generalization error , author=. Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence , year=

  55. [55]

    Advances in Neural Information Processing Systems , year=

    Simple and scalable predictive uncertainty estimation using deep ensembles , author=. Advances in Neural Information Processing Systems , year=

  56. [56]

    1995 , publisher=

    Lang, Serge , edition=. 1995 , publisher=

    1995

  57. [57]

    1999 , publisher=

    Lang, Serge , series=. 1999 , publisher=

    1999

  58. [58]

    , series=

    Lee, John M. , series=. 1997 , publisher=

    1997

  59. [59]

    , series=

    Lee, John M. , series=. 2003 , publisher=

    2003

  60. [60]

    Proceedings of the 37th International Conference on Machine Learning , year=

    Fine-grained analysis of stability and generalization for stochastic gradient descent , author=. Proceedings of the 37th International Conference on Machine Learning , year=

  61. [61]

    Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics , year=

    The implicit regularization of ordinary least squares ensembles , author=. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics , year=

  62. [62]

    arXiv preprint arXiv:2506.02257 , year=

    Assumption-free stability for ranking problems , author=. arXiv preprint arXiv:2506.02257 , year=

    Pith/arXiv arXiv

  63. [63]

    Little, Roderick J. A. and Rubin, Donald B. , edition=. 2019 , publisher=

    2019

  64. [64]

    Proceedings of the 34th International Conference on Machine Learning , year=

    Algorithmic stability and hypothesis complexity , author=. Proceedings of the 34th International Conference on Machine Learning , year=

  65. [65]

    Less is More: Exploiting Feature Density for Enhanced Membership Inference Attacks , author=

  66. [66]

    Journal of Machine Learning Research , volume=

    Randomization can reduce both bias and variance: A case study in random forests , author=. Journal of Machine Learning Research , volume=

  67. [67]

    Advances in Neural Information Processing Systems , year=

    Decision Tree for Locally Private Estimation with Public Data , author=. Advances in Neural Information Processing Systems , year=

  68. [68]

    Journal of Machine Learning Research , volume=

    Optimal Locally Private Nonparametric Classification with Public Data , author=. Journal of Machine Learning Research , volume=

  69. [69]

    Proceedings of the 28th International Conference on Artificial Intelligence and Statistics , year=

    Locally Private Estimation with Public Features , author=. Proceedings of the 28th International Conference on Artificial Intelligence and Statistics , year=

  70. [70]

    Advances in Neural Information Processing Systems , year=

    Fast and robust least squares estimation in corrupted linear models , author=. Advances in Neural Information Processing Systems , year=

  71. [71]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Stability selection , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2010 , publisher=

    2010

  72. [72]

    Journal of Machine Learning Research , volume=

    Randomization as regularization: A degrees of freedom explanation for random forest success , author=. Journal of Machine Learning Research , volume=

  73. [73]

    Journal of Machine Learning Research , volume=

    Getting better from worse: Augmented bagging and a cautionary tale of variable importance , author=. Journal of Machine Learning Research , volume=

  74. [74]

    Linear Algebra and its Applications , Year =

    Infinite-dimensional Log-Determinant divergences between positive definite trace class operators , Author =. Linear Algebra and its Applications , Year =

  75. [75]

    Advances in Computational Mathematics , volume=

    Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization , author=. Advances in Computational Mathematics , volume=. 2006 , publisher=

    2006

  76. [76]

    James Murdoch and Chandan Singh and Karl Kumbier and Reza Abbasi-Asl and Bin Yu , title =

    W. James Murdoch and Chandan Singh and Karl Kumbier and Reza Abbasi-Asl and Bin Yu , title =. Proceedings of the National Academy of Sciences , volume =

  77. [77]

    Journal of Machine Learning Research , volume=

    On the stability of feature selection algorithms , author=. Journal of Machine Learning Research , volume=

  78. [78]

    Electronic Journal of Statistics , volume=

    The wishart and inverse wishart distributions , author=. Electronic Journal of Statistics , volume=

  79. [79]

    Proceedings of the 8th International Workshop on Artificial Intelligence and Statistics , year=

    Online bagging and boosting , author=. Proceedings of the 8th International Workshop on Artificial Intelligence and Statistics , year=

  80. [80]

    and Varoquaux, G

    Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V. and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P. and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E. , journal=. Scikit-learn: Machine Learning in

Showing first 80 references.