Pith. sign in

REVIEW 2 major objections 4 minor 179 references

Parallel gradient boosting trains all output targets with a single shared descent direction and one base model per iteration, proving convergence to the optimal empirical risk while making conditional distribution estimation via multiple qu

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 04:51 UTC pith:VM4KAQYG

load-bearing objection Genuine algorithmic contribution with an honest but narrow convergence theorem; the practical claims rest on the empirical study, and the paper deserves peer review. the 2 major comments →

arxiv 2607.13550 v1 pith:VM4KAQYG submitted 2026-07-15 stat.ML cs.LG

Parallel gradient boosting for flexible estimation of conditional distributions

R\'emy Chapelle (CESP , CB , EVDG) , Nicolas Vayatis (CB) , Bruno Falissard (CESP) , Mohammed Sedki (CESP) This is my paper
classification stat.ML cs.LG MSC 62G0868T05
keywords parallel gradient boostingmulti-output regressionmultiple quantile regressionconditional distribution estimationgradient boosting convergencepinball lossnonparametric conditional density
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces parallel gradient boosting (PGB), a multi-output variant of gradient boosting in which a single univariate base model is trained per iteration and then scaled by a shared direction vector, so the number of targets no longer multiplies the training cost. The central theoretical result is that, for separable losses whose components are convex and smooth, PGB converges in empirical risk to the best predictor in the span of its base models—matching the guarantee of classical multi-output boosting while using only one weak-learner call per step. The authors apply PGB to multiple quantile regression, where training time scales almost independently of the number of quantiles, and show that the resulting conditional distribution estimates are competitive with or better than several nonparametric alternatives, especially with many covariates and mixed or missing data. The paper positions PGB as a general speeding-up layer that can be added to any boosting implementation, with trees as the default base learner but no requirement of tree-specific machinery.

Core claim

The paper's central discovery is that the computationally hard multi-output weak-learner problem—choosing a base model and direction to align with the full gradient—can be replaced by a much simpler two-step procedure without losing the convergence guarantee. At each iteration PGB picks a coordinate direction from the canonical basis (visited infinitely often), fits one univariate model to the projected pseudo-residuals, and then performs a full line search over all M components of the step. Theorem 3 shows that under Assumptions 3, 5 and 6, this procedure drives the empirical risk to the minimum over the span of the collinear base models, because the span of {h_w β} is exactly the M-fold pr

What carries the argument

The engine is the collinear base-model family φ_{w,β}(x) = h_w(x) β, where h_w is any univariate prediction rule and β is a unit vector giving the direction of the update. This factorization separates norm learning from direction selection: only h_w is learned by a weak learner, while β is chosen from the canonical basis in rotation (Equation 9) and refined by a multidimensional line search that sets the step size γ_t in R^M. Lemma 3 shows the span of these collinear models equals the product of univariate spans, which lets the proof reduce the multivariate descent to a separate univariate boosting run per coordinate, with Corollary 1 allowing the risk to stay flat on non-updated coordinates

Load-bearing premise

The convergence theorem requires each component of the loss to be convex and K-smooth, but the paper's flagship use case—multiple quantile regression—minimizes the pinball loss, which is non-differentiable at zero and lacks K-smoothness, so the formal guarantee does not cover the practical algorithm that delivers the reported speed and accuracy.

What would settle it

On a simulated dataset with known conditional quantiles, set the base-learner class to stumps and compute the minimum possible weighted interval score over all linear combinations of stumps via linear programming. If PGB with the pinball loss fails to approach that minimum after a very large number of iterations, with the gap remaining non-negligible, then the practical algorithm does not deliver the convergence that Theorem 3 would require, exposing the mismatch between the smooth-loss theory and the pinball application.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • PGB makes dense quantile grids affordable: in the baseball experiments, training 128 quantiles with a standard boosting library took more than 50 times longer than one quantile, while PGB's cost stayed nearly flat, enabling finer approximation of conditional distributions.
  • The convergence theorem applies to any univariate base learner satisfying the symmetry and approximation assumptions, not only trees; the paper demonstrates convergence with linear models on real datasets.
  • Because all targets share the same splits, PGB produced far fewer quantile crossings than standard boosting on the simulated example, simplifying post-processing.
  • The conditional distribution estimator built on PGB matches or beats distributional random forests, FlexCode, and LinCDE on most of the ten medical datasets tested, with the largest gains on high-dimensional and missing-data settings.
  • The isotonic-regression post-processing step is proven (by a cited result) to only improve the weighted interval score, so the correction for quantile crossings is safe.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the empirical speed-ups persist for non-smooth losses, the same collinear-direction idea could be applied to other multi-output objectives, such as multiclass classification with hinge-type losses, where the paper's convergence theory does not currently reach.
  • The per-coordinate line search gives PGB a block-coordinate-descent flavor; interpreting the canonical-basis sampling as a randomized block selection rule could let one import Gauss-Southwell or importance-sampling results to choose directions adaptively with provable rates (the paper only compares these empirically).
  • Since computation time is nearly independent of M, users could push quantile grids far beyond the paper's rule-of-thumb M≈2√n, trading negligible runtime for smaller CRPS approximation error; the paper's Appendix D suggests gains flatten near M=20, but this was tested only on multimodal circular data.
  • The shared-splits property suggests PGB might serve as a regularizer in the output direction: because all quantiles share the same tree structure, the effective number of parameters grows more slowly than in separate-per-quantile boosting, which may explain the reduced crossing and could be studied as a form of output-space regularization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces Parallel Gradient Boosting (PGB), a multi-output boosting procedure that trains a single univariate base learner per iteration by projecting the gradient onto a coordinate direction and then performing a multidimensional line search. The main theoretical result, Theorem 3 (Section 4.1), states that, under separability, convexity and K-smoothness of the loss, symmetry and boundedness of the base models, and per-coordinate compactness, the empirical risk of PGB converges to the minimum attainable over the linear span of the induced multi-output base models. The paper applies PGB to multiple quantile regression with the pinball loss and to conditional density estimation via quantile interpolation and kernel smoothing. Empirically, PGB is reported to match XGBoost in quantile-regression quality while being orders of magnitude faster, and to outperform or match DRF, FlexCode, and LinCDE on simulated and real medical data, with repeated trials and standard deviations reported. Code and an R package are provided.

Significance. If the results hold, PGB is a valuable contribution: it attacks a genuine computational bottleneck of multi-output boosting (one weak learner per target per iteration) with a simple, general construction that does not require specialized base learners. The proof strategy is transparent and the appendix is self-contained; Theorem 3 follows from separability and Corollary 1, and the coordinate-cycling argument is sound. The empirical study is more careful than typical for a methods paper: repeated runs, standard deviations, and comparisons on diverse real datasets. However, the main convergence theorem uses assumptions that exclude the pinball loss on which the paper's central application rests, and the implemented tree-fitting step solves an approximate objective rather than the exact weak-learner problem assumed by the theorem. These gaps mean the headline convergence guarantee currently applies to an idealized version of the procedure, not to the deployed algorithm. The paper's practical speed and quality claims still stand on the experiments, but the theoretical support for the main use case is weaker than the presentation suggests.

major comments (2)
  1. [§5.1, Theorem 3 (Assumption 6)] The central convergence result does not cover the loss used in the paper's main application. Theorem 3 requires Assumption 6, which says each elementary loss ℓ_m is convex and K-smooth in its second argument. Section 5.1 explicitly states that the pinball loss “does not meet the usual assumptions that are used in proofs of convergence of gradient boosting (besides being non-differentiable at 0, it notably lacks the K-smoothness property).” Since the multiple-quantile-regression experiments in Sections 5.1 and 6 minimize the pinball/WIS loss, the claim lim_{t→∞} R̂_n(g_t)=R̂_n(g*) is not established for the deployed method. The paper should either extend the convergence analysis to nonsmooth convex losses (e.g., via subgradient arguments or smoothing), prove a separate guarantee for the pinball case, or explicitly reframe Theorem 3 as applying only to smooth losses and remove the implicat
  2. [§4.2, Eq. (5)] The analyzed algorithm differs from the implemented algorithm in a load-bearing way. Theorem 3 assumes that w_t solves Eq. (5) exactly and that γ_t is a global minimizer of R̂_n(g_{t-1}+h_{w_t}γ). Section 4.2 states that the actual tree fitting minimizes the residual sum of squares ∑(-a_i-h_w(x_i))², which is an approximate proxy for Eq. (5), and that the implementation uses leaf-wise line search. For such approximate weak-learner fits, the update is not guaranteed to give the non-increasing empirical risk that Corollary 1 needs for intermediate steps. Thus even for smooth losses, the proof does not apply to the shipped algorithm. Please state the approximation explicitly, give conditions under which it preserves the descent property, or present Theorem 3 as a result for the exact procedure and provide separate theoretical or empirical justification for the approximation gap.
minor comments (4)
  1. [Algorithm 1] The algorithm box lists a 'differentiable and separable loss function' as input, but the paper's main experiments use the non-differentiable pinball loss. The surrounding text discusses this, but the box should be made consistent, e.g., by using 'suitable separable loss' and noting that nondifferentiable cases are handled in Section 5.1.
  2. [Figure 3] The caption says quantile crossing is plotted in red, and the text states that 'more than half of the quantiles are affected' with XGBoost. Please report the exact counts or proportions, and state the number of quantile levels used, to make the comparison reproducible and precise.
  3. [§6.1, Figure 7] The text says PGB and LinCDE 'perform significantly better' than DRF and FlexCode, and PGB is 'best-performing' in each scenario. Given that the reported error bars overlap in some settings and no formal significance tests are provided, the wording is stronger than the evidence. Add pairwise tests or phrase the conclusions more cautiously.
  4. [§5.2] The scaling formula for F_scaled mixes a condition on the raw c.d.f. taking values in {0,1} with a piecewise rescaling of the interior. Please clarify how the tails are handled before convolution: the text says the resulting p.d.f. is supported on a compact set and then smoothed with a Laplacian kernel, but the exact role of the {0,1} condition in making the c.d.f. absolutely continuous is not fully explained.

Circularity Check

0 steps flagged

No significant circularity: Theorem 3 is a genuine derivation from stated assumptions, and the empirical claims rely on held-out evaluation rather than fitted inputs.

full rationale

The paper's central theoretical claim is Theorem 3, and its proof is a direct derivation from Assumptions 3, 5, 6 and per-coordinate compactness. After Lemma 3 identifies span(phi_theta) with the product span(h_w)^M, the proof applies Corollary 1 coordinate-wise and uses separability to equate lim Rhat_n(g_t) with sum_m min_{f_m in span(h_w)} Rhat_{n,m}(f_m), which is Rhat_n(g*) by Lemma 3. This is a genuine implication, not a restatement of the conclusion: the theorem does not assume lim Rhat_n(g_t) = Rhat_n(g*), and the proof supplies the descent and alignment arguments. The empirical sections compare PGB to XGBoost, DRF, FlexCode, and LinCDE on held-out test sets, with hyperparameters chosen by validation or cross-validation and then evaluated on separate test folds; this is standard model selection, not fitting the reported metric. Self-citations are limited to the authors' R package for availability and to background work by co-authors (e.g., Lugosi & Vayatis 2004); neither is load-bearing for Theorem 3 or for the empirical ranking. The paper itself flags the known scope gaps: Section 5.1 states that the pinball loss 'does not meet the usual assumptions that are used in proofs of convergence of gradient boosting (besides being non-differentiable at 0, it notably lacks the K-smoothness property)', so Theorem 3 does not cover the exact loss used in the multiple-quantile application; and Section 4.2 says trees are fitted by minimizing residual sum of squares, an approximation to Equation (5) assumed by Theorem 3. These are correctness and coverage caveats, not circularity: they concern whether the analyzed object matches the deployed one, not whether the analyzed result is assumed as input. Proposition 1's proof is omitted in the main text but is supplied as Lemma 3 in Appendix A.3, so no load-bearing step is left as an unsupported self-citation. Overall, no derivation step reduces to its own inputs.

Axiom & Free-Parameter Ledger

3 free parameters · 8 axioms · 0 invented entities

The convergence result is built on five explicitly stated assumptions plus exact optimization of the weak-learner objective and infinite revisiting of output directions; the main application violates the smoothness assumption, so the practical claims are empirical. No new physical entities are introduced.

free parameters (3)
  • Number of quantile knots M = max(20, 2*sqrt(n)), rounded
    Rule-of-thumb set by the authors to balance approximation error and runtime; explicitly not theoretically justified (Section 5.2).
  • Laplacian bandwidth lambda = cross-validated per dataset to minimize logarithmic score
    Controls smoothness of the final density estimate; a fitted hyperparameter, not part of the boosting algorithm.
  • Boosting hyperparameters (alpha, subsample, depth, min_leaf, T) = alpha=0.02, subsample=0.5, depth=min(d,3), min_leaf=5, T by early stopping/CV
    Hand-chosen defaults used for all experiments; T is fit to validation data. They affect the empirical comparison but are not derived from theory.
axioms (8)
  • domain assumption Loss is convex and K-smooth in its second argument (Assumptions 1/6)
    Used in the descent lemmas and convergence proofs; violated by the pinball loss used in the main application.
  • domain assumption Base-model class is symmetric: if phi_theta is available then -phi_theta is available (Assumptions 2/5)
    Ensures the optimal weak learner gives a non-negative decrease kappa_t and allows two-sided bounds in Theorem 2.
  • domain assumption Base models are bounded: ||phi_theta(x_i)||^2 <= 1/n (Assumption 3)
    Expository normalization simplifying the constants in the proof.
  • domain assumption Sublevel sets of the empirical risk are compact (Assumption 4)
    Used with Bolzano-Weierstrass to extract a convergent subsequence and identify the limit as a minimizer.
  • domain assumption Loss is separable across output components (Assumption 6)
    Makes the multidimensional line search coordinate-wise and lets Theorem 3 follow from Corollary 1.
  • domain assumption The weak-learner oracle solves Equation (5) exactly
    The theorem assumes exact argmin over W; the implementation fits trees by minimizing residual sum of squares instead, so the proof does not strictly cover the shipped algorithm.
  • ad hoc to paper Each canonical direction e_m is visited infinitely often (Equation 9)
    Required to apply Corollary 1 to every coordinate; guaranteed almost surely by uniform random sampling but not deterministic.
  • standard math Standard convex-analysis and compactness results (Bolzano-Weierstrass, Borel-Lebesgue, Lipschitz-gradient inequalities)
    Used throughout Appendix A without proof.

pith-pipeline@v1.3.0-alltime-deepseek · 33319 in / 19250 out tokens · 194426 ms · 2026-08-02T04:51:41.795159+00:00 · methodology

0 comments
read the original abstract

Boosting is one of the most successful learning techniques for standard classification and regression tasks. Its extension to multi-output prediction problems has found an increasing number of applications in recent years. Among them is the prediction of entire conditional distributions rather than single functionals, which can often be framed as a multi-output regression problem, for example multiple quantile regression. Addressing such problems with classical implementations of boosting is computationally challenging, because usually one base model is trained for each target at every iteration. More efficient variants of boosting have been proposed to speed up training, but they tend to be tied to specific loss functions and classes of base learners, usually decision trees. In this work, we study a modification of the gradient boosting algorithm, which we call parallel gradient boosting, designed to circumvent all these limitations. The core idea is to use a common descent direction for all training observations. By doing so, only one base model is needed at each iteration, regardless of the number of targets, which allows for considerable performance gains. We establish sufficient conditions for the convergence of the algorithm, whose practical use is introduced via the multiple quantile regression setting. We show that in such a setting, it provides predictions of similar quality to state-of-the-art boosting libraries such as XGBoost, while being faster by several orders of magnitude. Then, we evaluate the properties of the resulting conditional distribution estimator, which is shown empirically to outperform other nonparametric and semiparametric estimators, especially in high-dimensional settings and in the presence of mixed and/or missing covariates.

Figures

Figures reproduced from arXiv: 2607.13550 by Bruno Falissard (CESP), CB, EVDG), Mohammed Sedki (CESP), Nicolas Vayatis (CB), R\'emy Chapelle (CESP.

Figure 1
Figure 1. Figure 1: Estimation of conditional distributions which are bimodal for some values [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Principle of parallel vs classical gradient descent. Here, [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Results of conditional quantile estimation with XGBoost and parallel [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Results of conditional quantile estimation with XGBoost and parallel [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Illustration of the linear interpolation between quantiles, and the resulting [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Prediction of the conditional density of [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Results of conditional density estimation on simulated datasets with an [PITH_FULL_IMAGE:figures/full_fig_p024_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Importance scores for data simulated according to the irrelevant covariates [PITH_FULL_IMAGE:figures/full_fig_p025_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Performance of various selection strategies for the projection directions in [PITH_FULL_IMAGE:figures/full_fig_p040_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Training curves for the 10 real datasets to predict the [PITH_FULL_IMAGE:figures/full_fig_p041_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Right: Quality of the estimations of conditional distributions provided by [PITH_FULL_IMAGE:figures/full_fig_p042_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

179 extracted references · 1 canonical work pages

  1. [1]

    Lee, Ann

    Izbicki, Rafael and B. Lee, Ann. Converting high-dimensional regression to high-dimensional conditional density estimation. Electronic Journal of Statistics

  2. [2]

    Journal of Approximation Theory , volume=

    Pointwise convergence of the Bernstein--Durrmeyer operators with respect to a collection of measures , author=. Journal of Approximation Theory , volume=. 2020 , publisher=

    2020

  3. [3]

    Health economics , volume=

    Healthcare cost regressions: going beyond the mean to estimate the full distribution , author=. Health economics , volume=. 2015 , publisher=

    2015

  4. [4]

    The KDD'21 Workshop on Causal Discovery , pages=

    Dirac delta regression: Conditional density estimation with clinical trials , author=. The KDD'21 Workshop on Causal Discovery , pages=. 2021 , organization=

    2021

  5. [5]

    Annual Review of Statistics and Its Application , volume=

    Probabilistic forecasting , author=. Annual Review of Statistics and Its Application , volume=. 2014 , publisher=

    2014

  6. [6]

    International journal of statistics in medical research , volume=

    Multiple imputation by fully conditional specification for dealing with missing data in a large epidemiologic study , author=. International journal of statistics in medical research , volume=

  7. [7]

    Applied Soft Computing , volume=

    Challenges and opportunities of generative models on tabular data , author=. Applied Soft Computing , volume=. 2024 , publisher=

    2024

  8. [8]

    Sur l'approximation de fonctions int

    Derriennic, Marie Madeleine , journal=. Sur l'approximation de fonctions int. 1981 , publisher=

    1981

  9. [9]

    Biometrics , pages=

    Robustness in real life: A study of clinical laboratory data , author=. Biometrics , pages=. 1982 , publisher=

    1982

  10. [10]

    Mid-South Educational Research Association Annual Meeting , year=

    A Call for Greater Use of Nonparametric Statistics , author=. Mid-South Educational Research Association Annual Meeting , year=

  11. [11]

    2017 , publisher=

    Classification and regression trees , author=. 2017 , publisher=

    2017

  12. [12]

    Journal of Machine Learning Research , volume=

    Learning conditional distributions on continuous spaces , author=. Journal of Machine Learning Research , volume=

  13. [13]

    Journal of Money, Credit and Banking , volume=

    Constructing density forecasts from quantile regressions , author=. Journal of Money, Credit and Banking , volume=. 2012 , publisher=

    2012

  14. [14]

    , TITLE =

    Rosenblatt, M. , TITLE =. Multivariate analysis,. 1969 , PAGES =

    1969

  15. [15]

    Annals of Economics and Finance , volume=

    Kernel estimation of multivariate conditional distributions , author=. Annals of Economics and Finance , volume=

  16. [16]

    Curse of dimensionality and related issues in nonparametric functional regression

    Geenens, Gery. Curse of dimensionality and related issues in nonparametric functional regression. Statistics Surveys

  17. [17]

    Computational Statistics & Data Analysis , volume=

    Bandwidth selection for kernel conditional density estimation , author=. Computational Statistics & Data Analysis , volume=. 2001 , publisher=

    2001

  18. [18]

    Journal of Machine Learning Research , volume=

    Distributional random forests: Heterogeneity adjustment and multivariate distributional regression , author=. Journal of Machine Learning Research , volume=

  19. [19]

    2012 , publisher=

    Neural networks for conditional probability estimation: Forecasting beyond point predictions , author=. 2012 , publisher=

    2012

  20. [20]

    ESAIM: Probability and Statistics , volume=

    Estimating the conditional density by histogram type estimators and model selection , author=. ESAIM: Probability and Statistics , volume=. 2017 , publisher=

    2017

  21. [21]

    Journal of the Korean Statistical Society , volume=

    Bernstein conditional density estimation with application to conditional distribution and regression functions , author=. Journal of the Korean Statistical Society , volume=. 2019 , publisher=

    2019

  22. [22]

    Journal of Nonparametric Statistics , volume=

    Estimation of the quantile function using Bernstein--Durrmeyer polynomials , author=. Journal of Nonparametric Statistics , volume=. 2014 , publisher=

    2014

  23. [23]

    IEEE Transactions on Knowledge and Data Engineering , volume=

    Label distribution learning , author=. IEEE Transactions on Knowledge and Data Engineering , volume=. 2016 , publisher=

    2016

  24. [24]

    Proceedings of the

    Theoretical analysis of label distribution learning , author=. Proceedings of the

  25. [25]

    Advances in neural information processing systems , volume=

    Label distribution learning forests , author=. Advances in neural information processing systems , volume=

  26. [26]

    International conference on machine learning , pages=

    Label distribution learning machine , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  27. [27]

    Proceedings of the

    Logistic boosting regression for label distribution learning , author=. Proceedings of the

  28. [28]

    Publications de l'Institut Mathematique , volume=

    On Lp-convergence of Bernstein-Durrmeyer operators with respect to arbitrary measure , author=. Publications de l'Institut Mathematique , volume=

  29. [29]

    Journal of Approximation Theory , volume=

    Approximation by multivariate Bernstein--Durrmeyer operators and learning rates of least-squares regularized regression with multivariate polynomial kernels , author=. Journal of Approximation Theory , volume=. 2013 , publisher=

    2013

  30. [30]

    Introduction to Nonparametric Estimation , year=

    Nonparametric estimators , author=. Introduction to Nonparametric Estimation , year=

  31. [31]

    Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages=

    Conditional density estimation via least-squares density ratio estimation , author=. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages=. 2010 , organization=

    2010

  32. [32]

    Riu, Benjamin , journal=

  33. [33]

    IEEE Transactions on Information Theory , volume=

    On the optimality of conditional expectation as a Bregman predictor , author=. IEEE Transactions on Information Theory , volume=. 2005 , publisher=

    2005

  34. [34]

    Artificial Intelligence Review , volume=

    A comparative analysis of gradient boosting algorithms , author=. Artificial Intelligence Review , volume=. 2021 , publisher=

    2021

  35. [35]

    Journal of the American Statistical association , volume=

    Methods for estimating a conditional distribution function , author=. Journal of the American Statistical association , volume=. 1999 , publisher=

    1999

  36. [36]

    1999 , publisher=

    Mathematical statistics , author=. 1999 , publisher=

    1999

  37. [37]

    2005 , publisher=

    Quantile regression , author=. 2005 , publisher=

    2005

  38. [38]

    arXiv preprint arXiv:2201.05340 , year=

    Machine learning for multi-output regression: When should a holistic multivariate approach be preferred over separate univariate ones? , author=. arXiv preprint arXiv:2201.05340 , year=

    Pith/arXiv arXiv

  39. [39]

    The Annals of statistics , volume=

    On the Bayes-risk consistency of regularized boosting methods , author=. The Annals of statistics , volume=. 2004 , publisher=

    2004

  40. [40]

    2024 , publisher=

    Learning theory from first principles , author=. 2024 , publisher=

    2024

  41. [41]

    Proceedings of the 22nd

    Xgboost: A scalable tree boosting system , author=. Proceedings of the 22nd

  42. [42]

    Advances in Neural Information Processing Systems , volume=

    Adaboost is consistent , author=. Advances in Neural Information Processing Systems , volume=

  43. [43]

    Journal of Applied Mathematics and Computing , volume=

    Durrmeyer variant of certain approximation operators , author=. Journal of Applied Mathematics and Computing , volume=. 2024 , publisher=

    2024

  44. [44]

    Advances in Contemporary Statistics and Econometrics: Festschrift in Honor of Christine Thomas-Agnan , pages=

    Optimization by gradient boosting , author=. Advances in Contemporary Statistics and Econometrics: Festschrift in Honor of Christine Thomas-Agnan , pages=. 2021 , publisher=

    2021

  45. [45]

    , biburl =

    Breiman, L. , biburl =

  46. [46]

    Prokhorenkova, Liudmila and Gusev, Gleb and Vorobev, Aleksandr and Dorogush, Anna Veronika and Gulin, Andrey , journal=

  47. [47]

    Advances in neural information processing systems , volume=

    Lightgbm: A highly efficient gradient boosting decision tree , author=. Advances in neural information processing systems , volume=

  48. [48]

    International conference on machine learning , pages=

    Ngboost: Natural gradient boosting for probabilistic prediction , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  49. [49]

    arXiv preprint arXiv:1907.03178 , year=

    M. arXiv preprint arXiv:1907.03178 , year=

    Pith/arXiv arXiv 1907

  50. [50]

    Advances in Neural Information Processing Systems , volume=

    Treeffuser: probabilistic prediction via conditional diffusions with gradient-boosted trees , author=. Advances in Neural Information Processing Systems , volume=

  51. [51]

    arXiv preprint arXiv:2204.00778 , year=

    Distributional gradient boosting machines , author=. arXiv preprint arXiv:2204.00778 , year=

    Pith/arXiv arXiv

  52. [52]

    Solar Energy , volume=

    Probabilistic forecasting of day-ahead solar irradiance using quantile gradient boosting , author=. Solar Energy , volume=. 2018 , publisher=

    2018

  53. [53]

    Comparing quantile regression methods for probabilistic forecasting of

    Vasseur, Sebastien P. Comparing quantile regression methods for probabilistic forecasting of. Scientific Reports , volume=. 2021 , publisher=

    2021

  54. [54]

    Remote Sensing , volume=

    Explanation and probabilistic prediction of hydrological signatures with statistical boosting algorithms , author=. Remote Sensing , volume=. 2021 , publisher=

    2021

  55. [55]

    Composite quantile regression with

    Sluijterman, Laurens and Kreuwel, Frank and Cator, Eric and Heskes, Tom , journal=. Composite quantile regression with. 2025 , publisher=

    2025

  56. [56]

    arXiv preprint arXiv:1905.07558 , year=

    Gradient tree boosting with random output projections for multi-label classification and multi-output regression , author=. arXiv preprint arXiv:1905.07558 , year=

    Pith/arXiv arXiv 1905

  57. [57]

    Joint European conference on machine learning and knowledge discovery in databases , pages=

    Random forests with random projections of the output space for high dimensional multi-label classification , author=. Joint European conference on machine learning and knowledge discovery in databases , pages=. 2014 , organization=

    2014

  58. [58]

    Computational statistics & data analysis , volume=

    Stochastic gradient boosting , author=. Computational statistics & data analysis , volume=. 2002 , publisher=

    2002

  59. [59]

    High-Dimensional Probability: An Introduction with Applications in Data Science , publisher=

    Vershynin, Roman , year=. High-Dimensional Probability: An Introduction with Applications in Data Science , publisher=

  60. [60]

    The Annals of Mathematical Statistics , pages=

    Mixture of distributions , author=. The Annals of Mathematical Statistics , pages=. 1948 , publisher=

    1948

  61. [61]

    1999 , publisher=

    Geometry of sets and measures in Euclidean spaces: fractals and rectifiability , author=. 1999 , publisher=

    1999

  62. [62]

    The Annals of Mathematical Statistics , volume=

    Choosing a point from the surface of a sphere , author=. The Annals of Mathematical Statistics , volume=. 1972 , publisher=

    1972

  63. [63]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Distribution-free prediction bands for non-parametric regression , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2014 , publisher=

    2014

  64. [64]

    2009 , howpublished =

    Tsanas, Athanasios and Little, Max , title =. 2009 , howpublished =

    2009

  65. [65]

    2024 , url =

    genridge: Generalized Ridge Trace Plots for Ridge Regression , author =. 2024 , url =

    2024

  66. [66]

    Peter Higgins , year =

  67. [67]

    Max Kuhn and Kjell Johnson , year =

  68. [68]

    Rémy Chapelle , year =

  69. [69]

    Ishwaran and U.B

    H. Ishwaran and U.B. Kogalur , publisher =. Fast Unified Random Forests for Survival, Regression, and Classification (. 2025 , url =

    2025

  70. [70]

    Nick Huntington-Klein and Malcolm Barrett , year =

  71. [71]

    Wayne Oldford and Adrian Waddell , year =

    R. Wayne Oldford and Adrian Waddell , year =

  72. [72]

    2016 , school=

    Tree boosting with xgboost-why does xgboost win "every" machine learning competition? , author=. 2016 , school=

    2016

  73. [73]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Bivariate quantile smoothing splines , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 1998 , publisher=

    1998

  74. [74]

    Statistics and its Interface , volume=

    Stepwise multiple quantile regression estimation using non-crossing constraints , author=. Statistics and its Interface , volume=. 2009 , publisher=

    2009

  75. [75]

    Journal of economic perspectives , volume=

    Quantile regression , author=. Journal of economic perspectives , volume=. 2001 , publisher=

    2001

  76. [76]

    Estimating conditional quantiles with the help of the pinball loss

    Steinwart, Ingo and Christmann, Andreas. Estimating conditional quantiles with the help of the pinball loss. Bernoulli (Andover.)

  77. [77]

    PLoS computational biology , volume=

    Evaluating epidemic forecasts in an interval format , author=. PLoS computational biology , volume=. 2021 , publisher=

    2021

  78. [78]

    Scoring interval forecasts: Equal-tailed, shortest, and modal interval

    Brehmer, Jonas R and Gneiting, Tilmann. Scoring interval forecasts: Equal-tailed, shortest, and modal interval. Bernoulli (Andover.)

  79. [79]

    arXiv preprint arXiv:2502.05157 , year=

    Efficient distributional regression trees learning algorithms for calibrated non-parametric probabilistic forecasts , author=. arXiv preprint arXiv:2502.05157 , year=

    Pith/arXiv arXiv

  80. [80]

    Frontiers in neurorobotics , volume=

    Gradient boosting machines, a tutorial , author=. Frontiers in neurorobotics , volume=. 2013 , publisher=

    2013

Showing first 80 references.