K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)

Jean-Francois Bonbhel

REVIEW 3 major objections 8 minor 12 references

Skipping easy samples during backprop without biasing the gradient

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-08 21:00 UTC pith:7CSJQHHE

load-bearing objection Selective backprop with Horvitz-Thompson reweighting: clean theory, CPU-scale experiments, variance scaling is the real question the 3 major comments →

arxiv 2607.05903 v1 pith:7CSJQHHE submitted 2026-07-07 cs.LG cs.AIcs.CL

K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)

Jean-Francois Bonbhel This is my paper

classification cs.LG cs.AIcs.CL

keywords selective backpropagationunbiased gradient estimationHorvitz-Thompson estimatorinverse-probability weightingdefensive mixture samplingnon-convex convergenceonline hard example miningclass imbalance

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that selective backpropagation — skipping the backward pass for low-loss (already-learned) samples — can be made statistically unbiased by combining a two-stratum sampling design with inverse-probability (Horvitz-Thompson) reweighting. The central object is the K-ABENA v3 estimator: high-loss samples are always retained; a fraction of low-loss samples is drawn under a defensive-mixture sampling design with strictly positive inclusion probabilities; and the retained subset is reweighted to recover an unbiased estimate of the full-batch gradient. The paper proves that this estimator achieves the standard O(1/sqrt(T)) non-convex SGD convergence rate, with a small additive bias floor in the practical self-normalized variant that shrinks as O(1/m). A companion result (Proposition 2) proves that uncompensated selective methods — those that skip easy samples without reweighting — cannot converge to the true minimizer when selection bias is non-negligible relative to the gradient signal, a failure the paper quantifies empirically at extreme class imbalance (AUC 0.53 vs. 0.9998 for full-batch). On real datasets, the compensated estimator is statistically indistinguishable from full-batch SGD while saving 28–54% of per-epoch gradient computation.

Core claim

The core discovery is that threshold-based sample exclusion and design-unbiased gradient estimation are compatible. By partitioning samples into a certainty stratum (high-loss majors, always retained) and a sampled stratum (low-loss minors, drawn under a defensive-mixture design with floor probability alpha/k), then applying Horvitz-Thompson inverse-probability weighting, one recovers an exactly unbiased gradient estimator while skipping a known, architecturally-independent fraction of backward passes. The paper further proves that the absence of such compensation is not merely a minor inaccuracy but a structural defect: under persistent selection bias bounded away from zero, the true minimz

What carries the argument

K-ABENA v3 estimator: a two-stratum sampling design combining (1) a certainty stratum of high-loss samples with inclusion probability 1, and (2) a defensive-mixture without-replacement sample of low-loss samples with inclusion probabilities bounded below by alpha*m/k, yielding a Horvitz-Thompson gradient estimator that is design-unbiased. The practical variant uses the self-normalized (Hajek) form with bias O(1/m).

Load-bearing premise

The Poisson/rejective approximation used to compute inclusion probabilities when the defensive-mixture coefficient alpha is less than 1 is acknowledged as approximate; the paper measures its residual bias at 0.004 for n=20,000 samples, but the exact behavior of this approximation under varying batch sizes, architectures, and loss landscapes at scale remains unverified.

What would settle it

On a task with extreme class imbalance (0.17% positives), run the compensated estimator and the uncompensated variants side by side: if the compensated estimator also fails to converge (AUC near 0.53), the claim that inverse-probability weighting resolves the selection-bias problem is false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Training pipelines for large-scale models could skip 28-54% of backward passes per epoch with no measurable accuracy loss, directly reducing compute cost and energy consumption.
The impossibility result (Proposition 2) provides a diagnostic: practitioners can estimate the ratio of selection bias to gradient norm at the minimizer to predict whether uncompensated selective methods like OHEM or SBP will fail on their specific task.
The two-stratum certainty-plus-defensive-mixture design principle may extend to other contexts where compute savings require subsampling correlated with the quantity being estimated, such as active learning or experience replay in reinforcement learning.
The convergence guarantee currently covers only SGD; extending to Adam-family optimizers faces a structural barrier (nonlinear preconditioning breaks the unbiasedness-to-update transfer), suggesting that new estimator designs or modified optimizers may be needed for compensated selection under adaptive methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the Poisson/rejective approximation for inclusion probabilities (Assumption 1(b)) degrades for specific loss landscapes or architectures not tested here, the design-unbiasedness guarantee becomes approximate in ways not fully bounded, particularly for deep networks with non-standard loss functions where the gradient factorization assumption may fail.
The variance increase (roughly 10x) from compensation suggests an optimal alpha exists that trades bias reduction against variance inflation; the paper fixes alpha=0.3 but an adaptive schedule could potentially recover some of the lost efficiency.
The structural incompatibility with Adam's nonlinear preconditioning implies that the practical deployment of unbiased selective backpropagation in modern deep learning pipelines may require either reverting to SGD (with momentum) or developing new preconditioning schemes that preserve linearity in the stochastic gradient.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Selective backprop with Horvitz-Thompson reweighting: clean theory, CPU-scale experiments, variance scaling is the real question

read the letter

Here's the short version: this paper applies survey-sampling estimators (Horvitz-Thompson and its self-normalized Hajek variant) to threshold-based selective backpropagation, proves the resulting gradient estimator is design-unbiased, and proves that uncompensated loss-based selection (OHEM, SBP, the author's own earlier variants) cannot converge to the minimizer when selection bias dominates the signal. The theory is mostly correct and the integration is genuinely new — nobody else has combined inverse-probability weighting with compute-saving sample exclusion in this way. Proposition 2 (the impossibility result) is simple and clean: if the bias field is bounded away from zero near a minimizer, the minimizer isn't a fixed point of the biased dynamics. The empirical instantiation on the 0.17% imbalance task — uncompensated variants get AUC 0.53, compensated gets 0.9991 — is a convincing demonstration. Code and seeds are released. The paper is unusually honest about its limitations, listing eight with quantified evidence for each. That honesty is real, not window dressing. The soft spots are exactly where you'd expect. First, all experiments are CPU-scale: logistic regression, linear SVM, softmax, one-hidden-layer MLP on bundled scikit-learn datasets. The central practical claim — compute reduction without convergence degradation — is untested at the scale where it would matter (deep CNNs, transformers, GPU training). The author says so explicitly as Limitation L4. Second, the stress-test concern about variance scaling is the more fundamental issue and it lands. Lemma 4 bounds the estimator variance as G_M^2 k^2 / (alpha m n^2), and the paper's own Table 4 shows 10x variance inflation over the uncompensated variant at one configuration. The bound grows quadratically in k (the minor fraction) and inversely in alpha and m. In deep learning regimes where 80-90% of samples are well-classified, k/n is large and m is constrained by compute budget — the variance could make the O(1/sqrt(T)) guarantee technically valid but practically slow. The paper doesn't analyze this scaling across k/n ratios. The AMSGrad degradation (AUC 0.806 vs 0.9991 for SGD at extreme imbalance) is a direct symptom, and the author correctly limits the theory to SGD (Limitation L1) but doesn't bound the regime where even SGD becomes impractically slow. The Poisson approximation (Assumption 1b) that the reader flags is real but secondary — the variance issue persists even at alpha=1. This paper is for researchers working on training efficiency and selective backpropagation. The theoretical framework — combining defensive-mixture sampling with HT reweighting, plus the impossibility result — is a legitimate contribution that someone could build on. It deserves a serious referee who can check the proofs carefully and push the author on the variance scaling question. I'd send it out for review.

Referee Report

3 major / 8 minor

Summary. The paper presents K-ABENA v3, a selective backpropagation framework that combines threshold-based sample exclusion with Horvitz-Thompson inverse-probability reweighting to achieve design-unbiased gradient estimation. The core theoretical contributions include: (1) a defensive-mixture sampling design (Definition 1) with bounded weights (Lemma 1), (2) design-unbiasedness of the HT estimator (Lemma 2) and an O(1/m) bias bound for the self-normalized Hajek variant (Lemma 3), (3) an O(1/sqrt(T)) non-convex SGD convergence guarantee (Theorem 1), and (4) an impossibility result showing that uncompensated loss-based selection cannot converge to minimizers where selection bias is bounded away from zero (Proposition 2). Empirical validation is on CPU-scale experiments: bundled scikit-learn datasets (Breast Cancer, Digits, Wine, Diabetes) and a synthetic fraud-replica at 0.17% class imbalance. The compensated estimator matches full-batch SGD at 28-54% compute savings on real datasets and resolves the AUC collapse (0.53 to 0.9991) at extreme imbalance. Code and seeds are released.

Significance. The paper addresses a well-defined problem: the incompatibility between compute-saving selective backpropagation and unbiased gradient estimation. The integration of survey-sampling estimators (Horvitz-Thompson, defensive mixture designs) into threshold-based sample exclusion is a clean and well-motivated contribution. The theoretical results (Lemmas 2-4, Theorem 1, Proposition 2) are largely self-contained, building on standard results from Ghadimi and Lan and from the survey-sampling literature. The impossibility result (Proposition 2) is a useful formalization of why uncompensated methods fail in adverse regimes. The paper ships reproducible code with released seeds, quantifies every claimed advantage and limitation, and provides falsifiable predictions (e.g., the signal-to-bias ratio of Remark 5). The honest scope statement (Limitation L4) and the gated contraindications for the biased v2 mode are commendable. However, the empirical validation is limited to shallow models on small datasets, which constrains the practical significance of the convergence guarantee.

major comments (3)

§4.2, Lemma 4 and Remark 2: The variance bound sigma_hat^2 <= G_M^2 k^2 / (alpha m n^2) scales quadratically in k and inversely in both alpha and m. The paper's own data (Table 4, Remark 2) documents a 10x variance inflation (1.5e-4 vs 1.5e-5) at a single configuration (n=20,000, k~8,000, m~2,400, alpha=0.3). No scaling analysis of sigma_hat^2 across different k/n ratios or problem dimensions is provided. For regimes where k/n is large (e.g., 80-90% of samples are well-classified) and m is constrained by compute budget, the convergence constant L*c*(sigma_hat^2 + delta^2)/sqrt(T) in Theorem 1 could become large enough that the O(1/sqrt(T)) guarantee is practically vacuous within realistic training horizons. The paper should either (a) provide a scaling analysis showing how sigma_hat^2 behaves across k/n ratios, or (b) explicitly discuss the regime where the bound becomes loose and bound,
§3, Assumption 1(b): The Poisson/rejective approximation pi_i = m * p_i for inclusion probabilities when alpha < 1 is stated as 'standard for high-entropy without-replacement designs.' The residual bias is measured at 0.004 for n=20,000 (Table 4), but the relationship between this approximation error and the convergence guarantee (Theorem 1) is not formally bounded. Lemma 2's unbiasedness proof relies on E[Z_i] = pi_i, but if pi_i is only approximate, the unbiasedness becomes approximate in ways not fully controlled by Lemma 3's bound. The paper should clarify whether the 0.004 measured bias subsumes the approximation error or is separate from it, and ideally provide a bound on the approximation error as a function of n and the design parameters.
§7, Table 3: The matched-budget comparison with rival methods (OHEM, Focal Loss, Global IS) uses the authors' own CPU re-implementations rather than official codebases. While this is acknowledged as extending Limitation L4, it raises a correctness-risk concern: the OHEM and Global IS baselines may not reflect the performance of properly tuned implementations. The Global IS result (accuracy 0.9573 vs 0.9720, p=0.002 on Breast Cancer) is a key load-bearing claim for the paper's argument that the two-stratum design is necessary. A reader cannot verify whether the Global IS baseline was fairly tuned without access to the implementation details. The paper should provide sufficient implementation details for these baselines or, at minimum, acknowledge this as a threat to the validity of the comparison.

minor comments (8)

§1, Contribution 3: The convergence rate is described as 'O(1/sqrt(T)) decay' but Theorem 1 includes an additive bias floor 2*delta*G_infty + delta^2 that does not vanish with T. The abstract should clarify that the rate is O(1/sqrt(T)) only for the HT form (delta=0); the Hajek form has a non-vanishing floor.
§7.1, Table 1: The Linear SVM / Breast Cancer row reports a 54.0% saving, which is inconsistent with the stated N=0.3, K at 40th percentile (which gives ~28% per Proposition 1). The paper should clarify what configuration produced 54%.
§5, Remark 5: The signal-to-bias ratio is stated as '~0.5%' (gradient norm ~7e-4, bias ~0.15), but 7e-4/0.15 = 0.0047 = 0.47%, which rounds to ~0.5%. This is correct but the arithmetic should be shown explicitly for the reader.
§2, Related Work: The distinction between K-ABENA and importance sampling methods (Katharopoulos and Fleuret [7]) is well-argued, but the claim that IS methods require a 'full-width scoring pass' could benefit from a citation to specific implementations that confirm this cost characterization.
§8, Limitation L1: The argument that Adam/AdamW break the linearity required for unbiasedness transfer is sound, but the claim about Lion (E[sign(g_hat)] != sign(E[g_hat])) is stated without formal justification. A one-line proof or citation would strengthen this.
Appendix A: The ratio-estimator expansion cites Särndal et al. [11, Ch. 5] but does not specify which theorem or equation number. A more precise reference would aid verification.
§9: The companion volume [2] is cited as 'available upon request from the author.' This is unusual for a journal submission and may not be accessible to reviewers or readers. If the companion volume contains material relevant to the paper's claims, it should be made available or the relevant content incorporated.
§7.5: The section title references 'Gradient moments, noise robustness, and optimizer interaction' but the section body appears to be empty or merged into Table 4 and Limitation L1. If content is missing, it should be added; if the section is intentionally a pointer to Table 4, this should be stated.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive report. The three major comments are addressed below: Comment 1 (variance scaling analysis) will be met with new experiments and an explicit discussion of the loose-bound regime; Comment 2 (approximation error of Assumption 1(b)) will be met with a formal bound and clarification of the 0.004 measurement; Comment 3 (baseline implementation details) will be met with expanded documentation and an explicit validity threat acknowledgment. All three require revision.

read point-by-point responses

Referee: §4.2, Lemma 4 and Remark 2: The variance bound sigma_hat^2 scales quadratically in k and inversely in alpha and m. No scaling analysis across k/n ratios is provided. The convergence constant could become practically vacuous. The paper should either (a) provide a scaling analysis, or (b) explicitly discuss the regime where the bound becomes loose.

Authors: The referee is correct that the paper presents only a single-configuration variance measurement and does not analyze how sigma_hat^2 scales across k/n ratios. We will address this through both requested remedies. (a) We will add a scaling analysis: a sweep over k/n in {0.2, 0.4, 0.6, 0.8, 0.9} at fixed n=20,000, alpha=0.3, and N=0.3, measuring gradient variance at a pre-trained iterate on the fraud replica. This will show whether the quadratic-in-k bound of Lemma 4 is tight or conservative in practice. (b) We will add an explicit paragraph in Section 4.2 discussing the regime where the Theorem 1 convergence constant L*c*(sigma_hat^2 + delta^2)/sqrt(T) becomes large — specifically when k/n is high (80-90% well-classified) and m is constrained — and state plainly that the O(1/sqrt(T)) guarantee may require impractical T in that regime. This is an honest limitation that should be visible to the reader, not buried. revision: yes
Referee: §3, Assumption 1(b): The Poisson/rejective approximation pi_i = m * p_i has residual bias measured at 0.004, but the relationship between this approximation error and the convergence guarantee is not formally bounded. The paper should clarify whether the 0.004 measured bias subsumes the approximation error or is separate from it, and ideally provide a bound on the approximation error as a function of n and the design parameters.

Authors: We will revise to clarify this point, which the current manuscript does indeed leave ambiguous. To answer the referee's specific question: the 0.004 measured value (Table 4, Remark 1) is the total empirical bias of the self-normalized estimator, which subsumes both the self-normalization bias of Lemma 3 and the approximation error from Assumption 1(b). These are not separate quantities in the measurement. However, the referee is correct that they are not formally separated in the theory. We will add a remark bounding the approximation error. Under rejective sampling, the standard result (Särndal et al. 1992, Ch. 6) gives |pi_i - m*p_i| = O(m*p_i*(1-m*p_i)/(n-k)), so the total variation distance between the true and approximate designs is O(k/(n*alpha*m)). This enters Lemma 2 as an additive perturbation of order O(G_M * k / (alpha * m * n)) on top of the self-normalization bias delta_m. We will state this bound explicitly and note that at the reported configuration (n=20,000, k~8,000, m~2,400, alpha=0.3) it evaluates to approximately 0.002, consistent with the measured 0.004 total. We will also note that setting alpha=1 eliminates the approximation entirely (exact SRSWOR), at the cost of losing the defensive floor's variance protection. revision: yes
Referee: §7, Table 3: The matched-budget comparison uses the authors' own CPU re-implementations rather than official codebases. The Global IS result is a key load-bearing claim. A reader cannot verify whether the Global IS baseline was fairly tuned. The paper should provide sufficient implementation details or acknowledge this as a threat to validity.

Authors: The referee's concern is legitimate. The current manuscript acknowledges this as extending Limitation L4 but does not provide sufficient implementation detail for a reader to assess fairness, and does not explicitly flag it as a validity threat in the body of Section 7. We will make two changes. First, we will add an implementation appendix (or expanded footnote) detailing: (i) the Global IS proposal distribution (q_i proportional to ell_i over all n samples, with self-normalized HT correction 1/(m*q_i)), (ii) the hyperparameter search performed for each baseline (learning rate grid {0.05, 0.1, 0.2, 0.5} for all methods; Focal Loss gamma in {1, 2, 3, 5}), (iii) the matched-budget protocol (identical retained-sample count per epoch). Second, we will add an explicit sentence in Section 7.3 stating that the CPU re-implementations are a threat to the validity of the rival comparison, and that the Global IS result in particular — while mechanistically explained by the weight-explosion argument — should be confirmed against an official implementation before being treated as definitive. We agree this caveat belongs in the body text, not only in the limitations list. revision: yes

Circularity Check

0 steps flagged

No circularity detected: the derivation chain is self-contained against external benchmarks and standard results.

full rationale

The paper's central derivation chain is self-contained and does not exhibit circularity. Lemma 2 (design-unbiasedness of the HT estimator) follows from a direct application of the standard Horvitz-Thompson estimator [5], an external result from survey sampling. Theorem 1 (convergence) builds on the classical Ghadimi and Lan [4] framework, an external citation. Proposition 1 (compute savings) is a straightforward arithmetic identity: (n-k)+m backward passes out of n gives saving (1-N)k/n by definition of m=floor(Nk). Proposition 2 (impossibility for uncompensated selection) follows from the definition of the bias field b(theta) being non-vanishing at the minimizer. The empirical results (Tables 1-4) are validated against external baselines (full-batch SGD) and external methods (OHEM, Focal Loss, Global IS), not against the author's own prior results as ground truth. The self-citation [2] is a pedagogical companion volume and [3] is the code library; neither is load-bearing for any proof. The Poisson approximation in Assumption 1(b) is flagged as an approximation with measured residual bias (0.004), not presented as an exact result. No step in the derivation chain reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The paper introduces no new physical entities or postulated objects. The free parameters (K, N, alpha) are interpretable controls with clear semantic meanings, not ad hoc constants fitted to make a derivation work. The axioms are standard domain assumptions for survey sampling and SGD theory.

free parameters (3)

K (loss threshold) = 40th percentile
The loss threshold separating minors from majors, set as a fixed percentile of the current loss distribution. Recalibrated per epoch.
N (retention proportion) = 0.3
Governs the compute budget by fixing the number of retained minors m = floor(N*k). Swept over {0.1, 0.3, 0.5}.
alpha (defensive-mixture coefficient) = 0.3
Controls the mixture between uniform sampling and loss-proportional sampling. Swept over {0, 0.15, 0.3, 0.5, 1}.

axioms (3)

domain assumption Assumption 1(b): Poisson/rejective approximation for inclusion probabilities (pi = m * pi) is valid for high-entropy without-replacement designs when alpha < 1.
Invoked in Section 3.1 to handle the case where alpha < 1. The paper states the empirical error is measured (0.004 at n=20,000) but the approximation is an unproved background assumption for the variance bound (Lemma 4).
domain assumption Assumption 2: The estimator satisfies uniform bounds on bias (delta) and variance (sigma_hat^2) along the entire trajectory.
Invoked in Section 4.3 for Theorem 1. The paper argues this is a consequence of the calibration protocol (Remark 3) for standard losses with bounded features, but flags it as a limitation (L5) for exotic losses or unbounded features.
standard math L-smoothness of F and bounded below by F*.
Standard assumption for non-convex SGD convergence proofs, invoked in Section 4.3.

pith-pipeline@v1.1.0-glm · 17479 in / 2575 out tokens · 699474 ms · 2026-07-08T21:00:00.553280+00:00 · methodology

0 comments

read the original abstract

We present K-ABENA (K-Adaptive Backpropagation with Error-based N-exclusion Algorithm), a selective gradient computation framework that reduces per-iteration training cost by excluding a fraction of low-loss ("minor") observations from the backward pass. Its canonical form (v3) combines a defensive-mixture sampling design over the minor set with Horvitz-Thompson inverse-probability reweighting, yielding a design-unbiased Horvitz-Thompson gradient estimator (Lemma 2) and whose self-normalized practical variant carries a bias of order O(1/m) with an explicit constant (Lemma 3). We prove an O(1/sqrt(T)) non-convex convergence guarantee for SGD under the estimator, with an additive term that quantifies the residual bias (Theorem 1). We further prove that uncompensated loss-based selection - a family that includes OHEM, SBP, and the two earlier K-ABENA variants - admits no stationary point at any minimizer where its selection bias is bounded away from zero (Proposition 2), and we quantify this failure empirically: at 0.17% class imbalance, uncompensated variants reach test AUC 0.53-0.62 versus 0.9998 for full-batch SGD, while the compensated estimator attains 0.9991 at identical 28.4% compute savings. On real datasets (Breast Cancer, Digits, Wine, Diabetes) the compensated estimator is statistically indistinguishable from full-batch SGD (paired permutation tests, p >= 0.5; Section 7) while saving 28-54% of per-epoch gradient computation. A biased "regularized mode" (the earlier half-domain variant) is retained as an option with a proven exact bias decomposition (Lemma 5) and quantified contraindications: it collapses to 0.386 accuracy under 40% label noise (baseline: 0.832) and to 0.53 AUC under extreme imbalance. Every advantage and every limitation reported in this paper is either proved or measured; all experiments are CPU-scale (NumPy/scikit-learn) and their scope is stated explicitly.

discussion (0)

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · 1 internal anchor

[1]

Curriculum learn- ing

Yoshua Bengio, J´ erˆ ome Louradour, Ronan Collobert, and Jason Weston. Curriculum learn- ing. InProceedings of the 26th Annual International Conference on Machine Learning (ICML), pages 41–48. ACM, 2009

work page 2009
[2]

YekoElite University Press, Brazzaville, Republic of Congo, 2026

Jean-Fran¸ cois Bonbhel.The Art of Ignoring: K-ABENA and the Science of Selective Backpropagation — A Complete Theory of Error Thresholding and Proportional Exclusion in Machine Learning. YekoElite University Press, Brazzaville, Republic of Congo, 2026. English edition. Available upon request from the author

work page 2026
[3]

kabena-ml: K-abena python library (scikit-learn / pytorch / ten- sorflow).https://github.com/Bonbhel/kabena-ml, 2026

Jean-Fran¸ cois Bonbhel. kabena-ml: K-abena python library (scikit-learn / pytorch / ten- sorflow).https://github.com/Bonbhel/kabena-ml, 2026. MIT License. Includes both estimator modes, validation scripts, and tutorials

work page 2026
[4]

Stochastic first- and zeroth-order methods for noncon- vex stochastic programming.SIAM Journal on Optimization, 23(4):2341–2368, 2013

Saeed Ghadimi and Guanghui Lan. Stochastic first- and zeroth-order methods for noncon- vex stochastic programming.SIAM Journal on Optimization, 23(4):2341–2368, 2013

work page 2013
[5]

Horvitz and Donovan J

Daniel G. Horvitz and Donovan J. Thompson. A generalization of sampling without re- placement from a finite universe.Journal of the American Statistical Association, 47(260): 663–685, 1952

work page 1952
[6]

Accelerating Deep Learning by Focusing on the Biggest Losers

Angela H Jiang, Daniel L-K Wong, Giulio Yang, Daniel Meng, Minjia Yu, Wei Xu, Cyrus Noghabi, Greg R Ganger, and Hai Helen Ke. Accelerating deep learning by focusing on the biggest losers. InarXiv preprint arXiv:1910.00762, 2019

work page internal anchor Pith review Pith/arXiv arXiv 1910
[7]

Not all samples are created equal: Deep learning with importance sampling

Angelos Katharopoulos and Fran¸ cois Fleuret. Not all samples are created equal: Deep learning with importance sampling. InProceedings of the 35th International Conference on Machine Learning (ICML), 2018

work page 2018
[8]

Pawan Kumar, Benjamin Packer, and Daphne Koller

M. Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. InAdvances in Neural Information Processing Systems (NeurIPS), vol- ume 23, 2010. 11

work page 2010
[9]

Focal loss for dense object detection

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ ar. Focal loss for dense object detection. InProceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980–2988, 2017

work page 2017
[10]

Art B. Owen. Monte carlo theory, methods and examples. InAvailable at statweb.stanford.edu/ owen/mc, 2013

work page 2013
[11]

Springer, 1992

Carl-Erik S¨ arndal, Bengt Swensson, and Jan Wretman.Model Assisted Survey Sampling. Springer, 1992

work page 1992
[12]

Training region-based object detectors with online hard example mining

Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 761–769, 2016. A Proof of Lemma 3 (self-normalization bias) Write ˆT= P i /∈Mgi +P i∈S π−1 i gi and cW= (n−k) + P i∈S π−1 i , so ˆg= ˆT ...

work page 2016

[1] [1]

Curriculum learn- ing

Yoshua Bengio, J´ erˆ ome Louradour, Ronan Collobert, and Jason Weston. Curriculum learn- ing. InProceedings of the 26th Annual International Conference on Machine Learning (ICML), pages 41–48. ACM, 2009

work page 2009

[2] [2]

YekoElite University Press, Brazzaville, Republic of Congo, 2026

Jean-Fran¸ cois Bonbhel.The Art of Ignoring: K-ABENA and the Science of Selective Backpropagation — A Complete Theory of Error Thresholding and Proportional Exclusion in Machine Learning. YekoElite University Press, Brazzaville, Republic of Congo, 2026. English edition. Available upon request from the author

work page 2026

[3] [3]

kabena-ml: K-abena python library (scikit-learn / pytorch / ten- sorflow).https://github.com/Bonbhel/kabena-ml, 2026

Jean-Fran¸ cois Bonbhel. kabena-ml: K-abena python library (scikit-learn / pytorch / ten- sorflow).https://github.com/Bonbhel/kabena-ml, 2026. MIT License. Includes both estimator modes, validation scripts, and tutorials

work page 2026

[4] [4]

Stochastic first- and zeroth-order methods for noncon- vex stochastic programming.SIAM Journal on Optimization, 23(4):2341–2368, 2013

Saeed Ghadimi and Guanghui Lan. Stochastic first- and zeroth-order methods for noncon- vex stochastic programming.SIAM Journal on Optimization, 23(4):2341–2368, 2013

work page 2013

[5] [5]

Horvitz and Donovan J

Daniel G. Horvitz and Donovan J. Thompson. A generalization of sampling without re- placement from a finite universe.Journal of the American Statistical Association, 47(260): 663–685, 1952

work page 1952

[6] [6]

Accelerating Deep Learning by Focusing on the Biggest Losers

Angela H Jiang, Daniel L-K Wong, Giulio Yang, Daniel Meng, Minjia Yu, Wei Xu, Cyrus Noghabi, Greg R Ganger, and Hai Helen Ke. Accelerating deep learning by focusing on the biggest losers. InarXiv preprint arXiv:1910.00762, 2019

work page internal anchor Pith review Pith/arXiv arXiv 1910

[7] [7]

Not all samples are created equal: Deep learning with importance sampling

Angelos Katharopoulos and Fran¸ cois Fleuret. Not all samples are created equal: Deep learning with importance sampling. InProceedings of the 35th International Conference on Machine Learning (ICML), 2018

work page 2018

[8] [8]

Pawan Kumar, Benjamin Packer, and Daphne Koller

M. Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. InAdvances in Neural Information Processing Systems (NeurIPS), vol- ume 23, 2010. 11

work page 2010

[9] [9]

Focal loss for dense object detection

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ ar. Focal loss for dense object detection. InProceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980–2988, 2017

work page 2017

[10] [10]

Art B. Owen. Monte carlo theory, methods and examples. InAvailable at statweb.stanford.edu/ owen/mc, 2013

work page 2013

[11] [11]

Springer, 1992

Carl-Erik S¨ arndal, Bengt Swensson, and Jan Wretman.Model Assisted Survey Sampling. Springer, 1992

work page 1992

[12] [12]

Training region-based object detectors with online hard example mining

Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 761–769, 2016. A Proof of Lemma 3 (self-normalization bias) Write ˆT= P i /∈Mgi +P i∈S π−1 i gi and cW= (n−k) + P i∈S π−1 i , so ˆg= ˆT ...

work page 2016