Pith. sign in

REVIEW 3 major objections 6 minor 14 references

OT-FairBoost: Optimal Transport-Guided Gradient Boosting for Fairness Regularization on Tabular Data

T0 review · 3 major / 6 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A Wasserstein-2 penalty inside gradient boosting yields the best accuracy–fairness trade-offs on tabular data.

desk verdict Usable discrete W₂ gradients for LightGBM fairness; solid engineering with one real but contained second-order approximation. read the letter →

arxiv 2607.28014 v1 pith:EBGAUDR2 submitted 2026-07-30 math.ST stat.TH

Veronika Shilova (IMT) , Abdoulaye Sakho (LPSM (UMR_8001)) , Younes Boumoussou , Laurent Risser (Comue de Toulouse , IMT) , Jean-Michel Loubes (IMT , REGALIA) , Emmanuel Malherbe This is my paper
classification math.STstat.TH MSC 62H3090C0868T05
keywords algorithmicfairnessgradientboostingoptimaltransportWasserstein-2distanceDemographicParityEqualizedOddstabulardatain-processing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Gradient-boosted trees remain the workhorse for tabular prediction, yet they routinely produce different score distributions across demographic groups. The paper shows that this gap can be closed during training itself by adding a discrete Wasserstein-2 distance between the groups’ predictions directly to the boosting objective. Sample-wise gradients and diagonal Hessians of that distance are derived so the fairness term fits the second-order leaf-split criterion used by LightGBM. Across binary classification, regression, multi-group attributes and a large real-world job-ad dataset, the resulting OT-FairBoost models dominate existing in-processing and post-processing baselines on a scalarized accuracy–fairness score. The method therefore supplies a practical, controllable way to enforce Demographic Parity or Equalized Odds without freezing the model or requiring the sensitive attribute at inference time.

What carries the argument

The sample-wise gradient 2/n_s (ẑ_i - T(ẑ_i)) and constant diagonal Hessian 2/n_s of the discrete squared Wasserstein-2 distance, obtained via the optimal transport map (or its CDF approximation) and treated as additive corrections to the ordinary loss gradient and Hessian inside each boosting iteration.

What would settle it

On any of the reported benchmarks, re-run the identical hyper-parameter sweep while replacing the diagonal-only Hessian with a full (or block) Hessian that retains the neglected couplings; if the Pareto front of OT-FairBoost then collapses below the baselines, the claim fails.

Watch

Extended reading notes

Core claim

By deriving exact sample-wise first and second derivatives of the discrete Wasserstein-2 distance between group-conditional prediction distributions and inserting them into LightGBM’s objective, OT-FairBoost simultaneously optimizes predictive loss and group fairness, producing superior accuracy–fairness trade-offs to prior in- and post-processing methods on classification, regression and multi-group tasks.

Load-bearing premise

All off-diagonal and cross-group second derivatives of the Wasserstein term are set to zero so the fairness penalty remains sample-separable inside the leaf-split formula.

Editorial extensions

If this is right

  • Practitioners can enforce Demographic Parity or Equalized Odds on production LightGBM models without a separate post-processing stage or access to the sensitive attribute at inference.
  • The same derivative formulae extend immediately to multi-class sensitive attributes and to continuous regression targets.
  • A single scalar λ smoothly traces the accuracy–fairness frontier, giving operators an explicit dial for regulatory compliance.
  • Because the method stays inside the standard second-order boosting loop, existing hardware and distributed LightGBM pipelines remain usable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same transport-map gradient could be attached to other leaf-wise or histogram-based boosters (XGBoost, CatBoost) with only an API change for custom gradients.
  • If the neglected cross-sample Hessian terms prove material on highly discrete leaf distributions, a low-rank or sketch approximation of those blocks might restore fidelity without destroying the O(n) per-iteration cost.
  • The CDF-based transport map already used for speed suggests a natural extension to streaming or mini-batch boosting where full empirical measures cannot be stored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes OT-FairBoost, an in-processing method that adds a discrete Wasserstein-2 penalty between group-conditional prediction measures to the LightGBM second-order objective. The authors derive sample-wise (one-sided) gradients and diagonal Hessians of W₂² for atomic empirical measures (Proposition 1 / Eq. 11), justify them via a matching continuous Gâteaux calculation, and approximate the transport map through CDFs. The regularized loss is plugged into standard leaf-split / leaf-weight formulae. Experiments on Folktables, Communities & Crime, and FairJob cover binary DP, Equalized Odds, regression DP, multi-group DP, and a large-scale setting, reporting superior Obj_α=0.75 trade-offs versus FairGS, FairEG, FairGBM, and two OT post-processors.

Significance. In-processing fairness for GBDTs remains sparse relative to neural methods; a distribution-matching regularizer that targets DP and EOdds (and extends to regression and multi-group attributes) is a genuine and practically relevant contribution. Strengths include explicit discrete and continuous derivative derivations (Appendices C–D), finite-difference oracle checks (Figure 1), empirical evidence that the penalty decreases over boosting iterations (Appendix F), integration into a widely used library, and a broad multi-task evaluation with large hyperparameter sweeps and 5-fold CV. If the approximation quality of the sample-separable second-order model is adequately characterized, the work would be a solid reference method for fair tabular boosting.

major comments (3)
  1. [Section 3.3, Eq. (11)–(12); Appendix E] Section 3.3 (after Eq. 11) and Appendix E set all cross-group mixed partials ∂²W₂²/∂ẑ⁰_i ∂ẑ¹_k to zero by freezing the opposite group inside each boosting iteration, and also set same-group off-diagonal Hessians to zero. Same-group off-diagonals vanish for order-preserving moves of distinct atoms, so that part is exact; the residual issue is cross-group. A single pooled tree f_t is grown whose leaves routinely mix S=0 and S=1, yet leaf gains and optimal weights use only the diagonal sample-wise (g,h) of Eq. 12. The joint quadratic model of the fairness term is therefore incorrect whenever a leaf is mixed. This approximation is load-bearing for the claim that the method optimizes W₂² inside LightGBM’s split criterion. The manuscript should either (i) quantify the error (e.g., magnitude of neglected −2γ*_ik terms, or an ablation that compares mixed-leaf vs. group-stratified trees / alterna
  2. [Section 3.1–3.2; Table 2 (Multi-Group row)] The multi-group extension is used in Table 2 (Folktables ethnicity, ΔmaxDP) and claimed in the abstract and contributions, yet the body never states how the OT regularizer M is defined for |S|>2 (pairwise sum of W₂², transport to a Wasserstein barycenter, max-pairwise, etc.). Because the sample-wise gradient/Hessian formulae and the CDF transport map are written for two measures, the multi-group case is not a trivial notational extension. Please specify the multi-marginal or reduced objective actually implemented and confirm that Proposition 1 still applies term-by-term.
  3. [Section 3.1; Table 2 (EOdds block)] For Equalized Odds the paper states that M can be conditioned on Y, but the concrete construction (separate W₂² on {Y=0} and {Y=1} slices, weighted sum, how empty cells are handled, interaction with the sigmoid chain rule) is not written down. Given that EOdds is one of the four headline settings and yields the largest reported Obj_α gap (Table 2), the precise regularizer and its derivatives should be stated at the same level of detail as DP.
minor comments (6)
  1. [Table 2; Section 4 Protocol] Table 2 marks FairGBM as “-” for DP rows with the note that it is not designed for DP; that is fine, but the caption/protocol should also state clearly which methods are inapplicable to regression and multi-group so readers do not interpret blanks as failures.
  2. [Appendix A, Table 5; Section 3.4] Hyperparameter table (Appendix A) lists a KDE bandwidth for OT-FairBoost, yet the main text describes CDF-based transport maps. Clarify whether KDEs are used for density estimation, for smoothing CDFs, or only in an unused ablation.
  3. [Figure 3] Figure 3 x-axis is log-scaled λ but the tick labels are hard to read; adding the exact λ grid and the fixed companion hyperparameters would improve reproducibility of the trade-off curves.
  4. [Section 3; Section 4 Evaluation Metrics] Minor notation: ẑ vs z, μ_{Z_s} vs μ_s, and the occasional switch between W₂²(μ_{Z0},μ_{Z1}) and W₂²(Z0,Z1) should be unified. Also “Wassertein” typo in the regression metrics bullet.
  5. [Section 4 Baselines] Post-processors are correctly flagged as requiring S at inference; consider adding a short remark on whether any in-processing baseline (e.g., FairGBM) also leaks S through constraint construction at train time only, to keep the comparison symmetric.
  6. [Appendix D.3; Section 3.3] The continuous second-Gâteaux calculation (Appendix D.3) yields a vanishing second derivative under the chosen scaling; a one-sentence pointer in the main text linking this to the diagonal-Hessian choice would help readers who skip the appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: discrete W2 samplewise derivatives are derived independently and empirical wins are measured on held-out folds against external baselines.

full rationale

The load-bearing chain is (i) discrete/continuous samplewise gradients and diagonal Hessians of W2 between group-conditional prediction measures (Proposition 1; Appendices C–E), (ii) insertion of those g/h into LightGBM’s second-order leaf objective (Eqs. 11–12), and (iii) empirical Pareto comparison via Obj_α on held-out folds against FairGS, FairEG, FairGBM, W1Post, and W2Post (Tables 2–3, Figs. 2–4). The derivative formulas are obtained from the discrete coupling definition and from a Gâteaux perturbation of the 1D continuous W2, then checked against finite-difference oracles (Fig. 1); they are not defined in terms of the reported fairness gaps or Obj_α. Prior OT-fairness citations (Risser et al. 2022, Jiang et al. 2020, Gouic/Chzhen barycenter post-processors) supply motivation and baselines, not a uniqueness theorem that forces the tree result. Hyperparameter λ and the reporting weight α=0.75 are free knobs for trade-off display, not fitted targets renamed as predictions. Cross-group Hessian sparsification is an approximation (correctness risk), not a circular reduction. No step reduces the central claim to its own inputs by construction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 1 invented entities

The claim rests on standard GBDT second-order boosting, 1-D Wasserstein geometry, group-fairness definitions, and several modeling approximations that make the penalty LightGBM-compatible. Free parameters are the usual fairness–accuracy knobs and HPO ranges. No new physical entities are postulated.

free parameters (4)
  • λ (fairness penalty weight) = search [0.1, 20]; FairJob example λ=10
    User-chosen multiplier on W₂² in L_fair; swept on a log-uniform range and used to trace trade-offs; selected configs maximize Obj_α.
  • α in Obj_α scalarization = 0.75
    Preference weight that defines the reported ‘best’ configuration (α=0.75); not derived from data but fixes which hyperparameter run is declared winner.
  • LightGBM and baseline hyperparameters = random search ranges in Appendix A
    num_leaves, max_depth, reg_lambda, feature/bagging fractions, method-specific bounds (Tables 4–5); 100 random configs per model shape the Pareto comparison.
  • KDE bandwidth (OT-FairBoost-specific) = search [0.01, 2.0]
    Listed in method-specific search space; affects density/CDF-side numerical behavior if used in the implementation path.
assumptions (6)
  • domain assumption Second-order Taylor expansion of the boosted objective with sample-wise grad/Hess is a valid split criterion (standard GBDT / LightGBM).
    Section 3.1 Eqs. 1–4; inherited from Chen & Guestrin / Ke et al.
  • domain assumption Group fairness operationalized as small W₂ (or conditional W₂) between group-conditional score laws implies DP / EOdds control on continuous scores.
    Section 3 opening; follows Jiang et al. and Risser et al. line of work.
  • ad hoc to paper Within one boosting iteration the other group’s distribution may be treated as fixed (cross-group mixed partials zero).
    Section 3.3 explicit modeling choice to keep the custom objective sample-separable.
  • ad hoc to paper Off-diagonal same-group Hessian entries of discrete W₂² can be set to 0 even with duplicate leaf values (one-sided derivatives).
    Section 3.3 and Appendix E; justified by continuous limit and ‘minority duplicates’ argument, not proved negligible for finite trees.
  • domain assumption CDF inverse composition H1^{-1}(H0(z)) is an adequate scalable surrogate for the discrete optimal transport map under ties.
    Section 3.4; standard 1-D OT fact plus engineering approximation for speed.
  • ad hoc to paper Right-hand derivatives at atoms sufficiently represent the non-C¹ discrete W₂² for large n.
    Proposition 1 and Appendix C; left/right gap said to vanish as |Z|→∞.
invented entities (1)
  • OT-FairBoost objective (W₂²-regularized GBDT loss with discrete sample-wise grad/Hess)
    purpose: Name the in-processing procedure that injects OT fairness into LightGBM leaf splits.
    Composite of known W₂ penalty + new discrete derivative estimates; not a new physical object, but the paper’s central constructed method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OT-FairBoost: Optimal Transport-Guided Gradient Boosting for Fairness Regularization on Tabular Data." pith.science (2026). https://pith.science/paper/EBGAUDR2

@misc{pith2026260728014,
  author       = {Pith},
  title        = {Pith review of: OT-FairBoost: Optimal Transport-Guided Gradient Boosting for Fairness Regularization on Tabular Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EBGAUDR2}},
  note         = {Machine review of arXiv:2607.28014}
}
read the original abstract

Although neural-based machine learning models have received a lot of attention recently, tree-based models such as gradient boosting are competitive for tabular data and therefore remain widely used in various applications of AI. As when using other machine learning predictive models, they can however yield discriminative predictions across demographic groups, due to so-called algorithmic biases. These undesirable phenomena have motivated the emergence of new regulatory frameworks and various AI fairness strategies. While several pre-and post-processing methodologies exist to mitigate such bias on gradient boosting models, only a few in-processing methods have been proposed. To bridge this gap, we introduce OT-FairBoost, a novel in-processing framework that incorporates a Wasserstein-2 distance penalty directly into the objective function of gradient-boosted trees. This OT-based mitigation strategy has been shown to efficiently optimize group fairness criteria such as Demographic Parity and Equalized Odds on neural-based predictions. To adapt this approach for gradient boosting, we extend the sample-wise gradient estimation of the Wasserstein-2 distance between group predictions to discrete distributions and hessian diagonals. We then integrate our approach into the LightGBM training procedure and evaluate it across binary classification, regression, and multi-group sensitive attribute settings. Experimental results in each of these settings demonstrate that OT-FairBoost achieves best accuracy-fairness trade-offs against alternatives.

Figures

Figures reproduced from arXiv: 2607.28014 by the authors.

Figure 1
Figure 1. Simulated dataset distributions Z 0 and Z 1 , and corresponding Wasserstein gradients and hessians for our estimate and the oracle on Z 0 samples. The first row of [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. PR AUC vs. ∆DP of all models on Folktables Tennessee for sampled hyperparameters configurations [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. OT-FairBoost metrics with respect to λ compared to other methods (other hyperpa￾rameters fixed). Trade-off between fairness and accuracy Finally, we analyze the sensitivity and operational stability of the proposed framework by tracking its behavior under a fixed hyperparameter configuration while varying the training fairness penalty parameter λ. The resulting trajectories are depicted in [PITH_FULL_IMAGE:figures/… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Performance comparisons across different classification and regression tasks. [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Hessian matrix from oracle estimates, on the first group [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]
Figure 6
Figure 6. Figure 6: CDFs of Z 0 and Z 1 with respect to boosting iteration t: each line is for a λ value (λ = 0, 0.1, 1 and 10), and each row for an iteration value (t = 1, 10, 25, 50 and 100). This optimization is a trade-off between the predictive performance and the fairness, where the…
Figure 7
Figure 7. Figure 7: W2 normalized with respect to iterations. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 8 linked inside Pith

  1. [1]

    In the following, we follow Risser et al

    = ∫1 0 ( H−1 0 (τ)−H −1 1 (τ) )2 dτ,(17) where H−1 s is the inverse of the cumulative distribution functionHs. In the following, we follow Risser et al. (2022) that give a meaning to the pseudo-derivative ofW 2 2(µ0,µ 1)with respect to a specific observationˆzi. In the following, we omitˆ·for readability. D.1 General Gâteaux Differentiability Model The tr...

    2022

  2. [5]

    Projection to fairness in statistical learning.arXiv preprint arXiv:2005.11720,

    Thibaut Le Gouic, Jean-Michel Loubes, and Philippe Rigollet. Projection to fairness in statistical learning.arXiv preprint arXiv:2005.11720,

    arXiv 2005

  3. [7]

    Tabicl: A tabular foundation model for in-context learning on large data.arXiv preprint arXiv:2502.05564,

    Jingang Qu, David Holzmuller, Gael Varoquaux, and Marine Le Morvan. Tabicl: A tabular foundation model for in-context learning on large data.arXiv preprint arXiv:2502.05564,

    arXiv

  4. [12]

    Fairness-aware grouping for continuous sensitive variables: Application for debiasing face analysis with respect to skin tone.arXiv preprint arXiv:2507.11247,

    Veronika Shilova, Emmanuel Malherbe, Giovanni Palma, Laurent Risser, and Jean-Michel Loubes. Fairness-aware grouping for continuous sensitive variables: Application for debiasing face analysis with respect to skin tone.arXiv preprint arXiv:2507.11247,

    arXiv

  5. [1996]

    Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel

    DOI: 10.24432/C5XW20. Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoencoder.arXiv preprint arXiv:1511.00830,

    arXiv

  6. [2002]

    Regulation (eu) 2024/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence (artificial intelligence act)

    Regulation (EU) 2024/1689. Regulation (eu) 2024/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence (artificial intelligence act). Official Journal of the European Union, L 2024/1689.,

    2024

  7. [2013]

    Mitigating unwanted biases with adversarial learning

    Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. InProceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340,

    2018

  8. [2019]

    On the power of randomization in fair classification and representation

    Sushant Agarwal and Amit Deshpande. On the power of randomization in fair classification and representation. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1542–1551,

    2022

Show all 14 references
  1. [2020]

    The measure and mismeasure of fairness: A critical review of fair machine learning.arXiv preprint arXiv:1808.00023, 14,

    Sam Corbett-Davies and Sharad Goel. The measure and mismeasure of fairness: A critical review of fair machine learning.arXiv preprint arXiv:1808.00023, 14,

    arXiv

  2. [2021]

    Fairgbm: Gradient boosting with fairness constraints.arXiv preprint arXiv:2209.07850,

    André F Cruz, Catarina Belém, Sérgio Jesus, João Bravo, Pedro Saleiro, and Pedro Bizarro. Fairgbm: Gradient boosting with fairness constraints.arXiv preprint arXiv:2209.07850,

    arXiv

  3. [2022]

    Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney

    doi: 10.1080/00031305.2021.1952897. Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre-processing for discrimination prevention.Advances in neural information processing systems, 30,

    2021 arXiv

  4. [2024]

    Laurent Risser, Alberto Gonzalez Sanz, Quentin Vincenot, and Jean-Michel Loubes

    https://eur- lex.europa.eu/eli/reg/2024/1689/oj/eng. Laurent Risser, Alberto Gonzalez Sanz, Quentin Vincenot, and Jean-Michel Loubes. Tackling algorithmic bias in neural-network classifiers using wasserstein-2 regularization.Journal of Mathematical Imaging and Vision, 64(6):672–689,

    2024

  5. [2025]

    Tabiclv2: A better, faster, scalable, and open tabular foundation model.arXiv preprint arXiv:2602.11139,

    Jingang Qu, David Holzmuller, Gael Varoquaux, and Marine Le Morvan. Tabiclv2: A better, faster, scalable, and open tabular foundation model.arXiv preprint arXiv:2602.11139,

  6. [2026]

    Fairxgboost: Fairness-aware classification in xgboost.arXiv preprint arXiv:2009.01442,

    Srinivasan Ravichandran, Drona Khurana, Bharath Venkatesh, and Narayanan Unny Edakunni. Fairxgboost: Fairness-aware classification in xgboost.arXiv preprint arXiv:2009.01442,

    2009 arXiv

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.