Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

DRO: A Python Library for Distributionally Robust Optimization in Machine Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new Python library implements 14 distributionally robust optimization formulations across 9 model backbones, and claims 10x to 1000x speedups over baseline implementations through vectorization and approximation techniques.

desk verdict A genuinely useful DRO library, but the exactness and speedup claims are overstated, especially for Marginal-DRO; deserves peer review after honest relabeling and validation. read the letter →

arxiv 2505.23565 v1 pith:KD3TMJLG submitted 2025-05-29 cs.LG cs.MScs.NAmath.NA

Jiashuo Liu , Tianyu Wang , Henry Lam , Hongseok Namkoong , Jose Blanchet This is my paper
classification cs.LGcs.MScs.NAmath.NA MSC 90C1568T05
keywords distributionallyrobustoptimizationDROPythonlibraryWassersteindistancef-divergencekernelmethodsscikit-learnPyTorch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces dro, an open-source Python library that unifies distributionally robust optimization (DRO) for machine learning. The library covers 14 DRO formulations and 9 backbone models, yielding 79 distinct methods, and is compatible with both scikit-learn and PyTorch. The authors aim to make DRO practical for large-scale ML tasks by replacing expensive symbolic reformulations with modular, vectorized implementations that run orders of magnitude faster. They report speedups of 10x to over 1000x compared to baseline implementations, with larger gains at larger sample sizes. If these claims hold, dro would be the first comprehensive, scalable DRO library that researchers and practitioners can drop into existing workflows.

What carries the argument

The load-bearing mechanism is the vectorized constraint construction inside CVXPY, where all N constraints are created as a single batched expression rather than through a Python loop. This is supplemented by two approximation techniques: Nyström kernel approximation for kernel-based methods like MMD-DRO, and constraint reduction that subsamples constraints or replaces a dense coupling matrix with row and column marginals on a k-nearest-neighbor graph. These techniques are what allow the library to solve DRO problems that would otherwise exceed solver limits on datasets of a few thousand samples.

What would settle it

Run the library and a baseline implementation on a fixed dataset of, say, 5000 samples with a specified hardware configuration, and compare both the objective value and the wall-clock time; if the speedup falls well below 10x on a standard benchmark, or if the sparse Marginal-DRO solution differs materially from the full coupling-matrix solution on a small dataset where the full problem is solvable, the central efficiency and exactness claims would be undermined.

Watch

Extended reading notes

Core claim

The central discovery is that a carefully engineered combination of constraint vectorization, kernel approximation, and formulation-specific constraint reduction can make a wide family of DRO problems tractable at scale without abandoning convex solvers. The paper implements exact optimization for linear and kernel models through disciplined convex programming, and approximate optimization for tree ensembles and neural networks. The performance gains come from batching CVXPY constraint construction, applying Nyström low-rank kernel embeddings, and reformulating expensive constraints such as the dense n-by-n coupling matrix in Marginal-DRO into sparse k-NN graph form. The authors present runtime comparisons showing speedups from 9.7x to over 2686x on datasets with 1000 to 10000 samples.

Load-bearing premise

The speedup claims rest on the assumption that the baseline implementations in Table 2 are representative and fairly timed, and the sparse k-NN reformulation for Marginal-DRO is an approximation that is not proven to preserve the original DRO objective.

Editorial extensions

If this is right

  • If the speedups are reproducible, DRO can move from toy problems to industrial-scale tabular and image datasets, enabling robust training where it was previously computationally prohibitive.
  • The unified interface across 14 formulations means practitioners can test multiple ambiguity sets (Wasserstein, f-divergence, kernel, hybrid) on the same model with minimal code changes, making DRO formulation selection an empirical question.
  • The compatibility with scikit-learn and PyTorch allows DRO models to plug into existing pipelines, hyperparameter tuning, and model selection tools.
  • The library's diagnostics, including worst-case distribution generation and out-of-sample performance evaluation, give users a way to audit robustness rather than just fit a single robust model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same vectorization and constraint-reduction recipe could be applied to other metric-based robust optimization problems beyond the 14 listed, such as robust Markov decision processes or robust portfolio optimization, where symbolic constraint construction is also a bottleneck.
  • The k-NN graph radius in the sparse Marginal-DRO reformulation introduces a second scale parameter that is not present in the original formulation; its interaction with the DRO radius epsilon is unexplored and could be an implicit regularizer worth explicit study.
  • The speedup ratios in Table 2 are not accompanied by hardware or baseline code details; a head-to-head reproduction study on a standardized benchmark would be a natural falsification test for the advertising claim.
  • The library's modular loss-function hooks suggest that user-defined losses for DRO are as easy to write as for ordinary supervised learning, which could lower the barrier for applying DRO to custom objectives in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript presents dro, an open-source Python library that claims to implement 14 DRO formulations over 9 backbone model classes, yielding 79 distinct DRO methods, with a scikit-learn/PyTorch-compatible API. It reports 10x to over 1000x speedups over baseline implementations on large-scale datasets, obtained through vectorized constraint construction, Nyström kernel approximation, and formulation-specific constraint reductions. The paper describes the package architecture, a method matrix, acceleration details, synthetic data generators, and a short benchmark table. If its claims are correct, dro would be the first comprehensive, scalable DRO library for machine learning.

Significance. A working, well-documented library that unifies many DRO formulations under one API would be a valuable community resource. The authors deserve credit for releasing source code, a PyPI package, documentation, unit tests, and synthetic data generators, and for attempting order-of-magnitude speedups. However, the significance is currently limited by two unresolved issues: the approximate reformulations in Appendix B are not shown to preserve the original DRO objectives, and the benchmark in Table 2 lacks the metadata needed to verify the speedup claims. These are correctness and reproducibility concerns, not mere presentation issues.

major comments (3)
  1. [Appendix B.3] The sparse reformulation of Marginal-DRO in Appendix B.3 is not a reformulation but a different optimization problem: the full n x n coupling matrix B from Duchi et al. (2023) is replaced by row/column marginals plus a k-NN cost graph, which changes both the feasible set and the objective. No theorem, error bound, or numerical validation shows that the resulting optimum matches the original Marginal-DRO value or solution. Consequently, Table 2's Marginal-DRO speedups (e.g., >2686.6x at n=10000) are not like-for-like comparisons of solvers, and the Introduction's claim that the library solves exact optimization problems 'without sacrificing computational exactness' is unsupported for this method. Please either prove equivalence, provide an approximation error bound, or clearly label Marginal-DRO as approximate and benchmark against an appropriate baseline.
  2. [Introduction, Section 3, Appendix B] The paper is internally inconsistent about exactness. The Abstract and Introduction state that the library solves exact optimization problems and that speedups come 'without sacrificing computational exactness'; Section 3 says the optimizations achieve 'near-exact solutions'; Appendix B.2 applies Nyström approximation, Appendix B.3 applies constraint subsampling for MMD-DRO and the sparse reformulation for Marginal-DRO, and Appendix A.3 describes RS-WDRO as 'approximately reformulated.' The manuscript needs an explicit delimitation of which of the 14 formulations are exact and which are approximate, under which hyperparameters, together with error measures for the approximate cases. Without this, the central claims of '14 formulations' and 'without sacrificing computational exactness' cannot be evaluated.
  3. [Table 2] Table 2 lacks essential experimental metadata: hardware and software versions, dataset names and dimensions, the number of repeated runs behind the standard errors, and, most importantly, a definition of the baseline implementation. If the baseline is a naive for-loop CVXPY construction and the accelerated version uses batched constraints plus constraint reduction or subsampling, the speedup ratios are not like-for-like comparisons. Also, several entries are truncated with '>' (e.g., '>230.6x', '>2686.6x'), which is not a measured quantity. Please provide the baseline code, dataset details, run counts, and a reproducibility script so the 10-1000x speedup claim is verifiable.
minor comments (6)
  1. [Table 1 and Abstract] By my count, Table 1 contains 78 check marks, not the 79 distinct DRO methods stated in the Abstract; please reconcile the count or the table.
  2. [Table 1] The header of Table 1 lists 'Personal' twice under Approximate Optimization; the intended column names (e.g., 'Personalized linear' and 'Personalized NN') should be clarified.
  3. [Appendix B.3 heading] The heading 'Constrain Reduction' appears to be a typo for 'Constraint Reduction'.
  4. [Appendix B.1] The code snippet for vectorized KL-DRO constraints uses variables eta and epi_g without definitions; please define these quantities or refer to the corresponding derivation.
  5. [Table 2] Table 2 reports standard errors but does not state the number of repeated trials; please report the number of runs.
  6. [Appendix A.3] The RS-WDRO formulation is described as 'approximately reformulated' without stating the nature or size of the approximation; this should be reconciled with the exactness claims in the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's DRO formulations are imported from published literature (some co-authored), and the software claims are backed by code and measurements rather than by self-referential derivation.

full rationale

The paper is a software library description. Its DRO formulations are presented as implementations of published results (e.g., Duchi and Namkoong 2019, 2021; Duchi et al. 2023; Blanchet et al. 2023; Sahoo et al. 2022), not as derivations performed here. Citing one's own earlier papers for the formulations is normal attribution and does not make the software claim circular, because the cited results are externally published and independently checkable. The speedup claims in Table 2 compare the optimized library against its own unoptimized baseline; this is an internal benchmark, not a derived prediction, and no fitted parameter is renamed as a forecast. The approximations in Appendices B.2/B.3 (Nyström, constraint subsampling, sparse marginal-DRO reformulation) are explicitly labelled as approximations; the Introduction's phrase 'without sacrificing computational exactness' is in tension with Section 3's 'near-exact' and with Appendix B.3's 'with approximation', but this is an internal consistency/correctness issue rather than a circularity, since the approximate problem is not claimed to be mathematically identical to the original by construction. No equation in the paper reduces to its own input, and no load-bearing premise is justified solely by a self-citation chain. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The central software contribution relies on standard reformulation results from the DRO literature (treated as axioms) and on the paper's own approximate Marginal-DRO reformulation. No new physical or mathematical entities are introduced. The hyperparameters listed are user choices, not fitted values.

free parameters (2)
  • n_components (Nystrom approximation rank)
    User-chosen parameter for kernel approximation in MMD-DRO and other kernel methods; affects accuracy of the kernel matrix but is not fitted to data.
  • k (k-NN graph size in Marginal-DRO sparse reformulation)
    User-chosen parameter for the sparse reformulation in Appendix B.3; controls the approximation quality but not fitted.
assumptions (2)
  • domain assumption The duality and reformulation theorems from cited DRO papers (e.g., Esfahani and Kuhn 2018, Shafieezadeh-Abadeh et al. 2019, Zhu et al. 2021) correctly convert the DRO problems into solvable convex programs.
    The library relies on these external results to implement exact optimization. Section 2 and Appendix A.3 invoke these reformulations without re-deriving them.
  • ad hoc to paper The sparse reformulation of Marginal-DRO (Appendix B.3) preserves the intended DRO model sufficiently well for practical purposes.
    Replacing the full coupling matrix B with row/column marginals on a k-NN graph is an approximation introduced by this paper. No equivalence proof or error bound is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DRO: A Python Library for Distributionally Robust Optimization in Machine Learning." pith.science (2026). https://pith.science/paper/KD3TMJLG

@misc{pith2026250523565,
  author       = {Pith},
  title        = {Pith review of: DRO: A Python Library for Distributionally Robust Optimization in Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KD3TMJLG}},
  note         = {Machine review of arXiv:2505.23565}
}
read the original abstract

We introduce dro, an open-source Python library for distributionally robust optimization (DRO) for regression and classification problems. The library implements 14 DRO formulations and 9 backbone models, enabling 79 distinct DRO methods. Furthermore, dro is compatible with both scikit-learn and PyTorch. Through vectorization and optimization approximation techniques, dro reduces runtime by 10x to over 1000x compared to baseline implementations on large-scale datasets. Comprehensive documentation is available at https://python-dro.org.

Figures

Figures reproduced from arXiv: 2505.23565 by the authors.

Figure 1
Figure 1. Overview of the dro library. exist. Correspondingly, existing software tools often fall short in two key aspects. (i) Scalability and ML integration: Many rely on symbolic reformulations of individual objectives or constraints to distributionally robust ones and solve the resulting problems using general￾purpose solvers (Chen et al., 2020; Vayanos et al., 2022). While helpful, these tools lack encapsulation of the D… view at source ↗
Figure 2
Figure 2. Overview of the documentation website. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Unregularized limit of stochastic gradient method for Wasserstein distributionally robust optimization

    math.OC 2025-06 accept novelty 6.0 of 10

    Gradients of the entropically smoothed and sampled WDRO objective converge to Clarke subgradients of the unregularized objective as regularization vanishes, yielding O(log N/√N) SGD convergence rates up to sampling error.

  2. Data Heterogeneity Modeling for Trustworthy Machine Learning

    cs.LG 2025-06 conditional novelty 3.0 of 10

    A survey that frames heterogeneity-aware machine learning as a paradigm spanning data collection, training, evaluation, and deployment, drawing mostly on the authors' prior results.

Reference graph

Works this paper leans on

14 extracted references · 7 canonical work pages · cited by 2 Pith papers

  1. [1]

    (2) (Generalized) f -divergence

    of the best empirical performance with 10 DRO Package E(X,Y )∼ bP [ℓ(bθ; (X, Y))] with bθ ∈ argminθ∈Θ E bP [ℓ(bθ; (X, Y))] obtained in the corresponding empirical risk minimization problem. (2) (Generalized) f -divergence. When d is set as the generalizedf -divergence (including CVaR), all distances there can be formulated as follows: d(P, Q) = EQ f dP dQ...

    work page 2023

  2. [3]

    Unifying distributionally robust optimization via optimal transport theory

    Jose Blanchet, Daniel Kuhn, Jiajin Li, and Bahar Taskesen. Unifying distributionally robust optimization via optimal transport theory. arXiv preprint arXiv:2308.05414 ,

    arXiv

  3. [4]

    Distributionally robust optimization and robust statistics

    Jose Blanchet, Jiajin Li, Sirui Lin, and Xuhui Zhang. Distributionally robust optimization and robust statistics. arXiv preprint arXiv:2401.14655 ,

    arXiv

  4. [7]

    Rethinking Distribution Shifts: Empirical Analysis and Modeling for Tabular Data

    Jiashuo Liu, Tianyu Wang, Peng Cui, and Hongseok Namkoong. On the need for a language describing distribution shifts: Illustrations on tabular datasets. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2023a. Jiashuo Liu, Tianyu Wang, Peng Cui, and Hongseok Namkoong. Rethinking distribution shifts: Empi...

    work page Pith review arXiv

  5. [9]

    skwdro: a library for wasserstein distributionally robust machine learning

    Florian Vincent, Wa ¨ ıss Azizian, Franck Iutzeler, and J´ erˆ ome Malick. skwdro: a library for wasserstein distributionally robust machine learning. arXiv preprint arXiv:2410.21231 ,

    arXiv

  6. [11]

    F unction Source Description classificationbasic Custom Multi-class Gaussian blobs on a sphere; baseline data generator

    Table 3: Summary of synthetic classification datasets and their sources. F unction Source Description classificationbasic Custom Multi-class Gaussian blobs on a sphere; baseline data generator. classificationDN21 Duchi and Namkoong (2021) (Sec 3.1.1) Linear decision boundary with controlled label noise. classificationSNVD20Sinha et al. (2018) (Sec 5.1) Ri...

    work page 2021

  7. [12]

    lad”, “svm

    Each function returns covariates and targets in NumPy format, which simulates challenging distribution shifts for robust model evaluation. Real-W orld Datasets. Real-world datasets can be easily imported using existing Python libraries such as whyshift (Liu et al., 2023a) and tableshift (Gardner et al., 2023). 9 Table 4: Summary of synthetic regression da...

    work page 2021

  8. [2000]

    Learning from a biased sample

    Roshni Sahoo, Lihua Lei, and Stefan Wager. Learning from a biased sample. arXiv preprint arXiv:2209.01754,

    arXiv

Show all 14 references
  1. [2013]

    Optimizer’s information criterion: Dissecting and correcting bias in data-driven optimization

    Garud Iyengar, Henry Lam, and Tianyu Wang. Optimizer’s information criterion: Dissecting and correcting bias in data-driven optimization. arXiv preprint arXiv:2306.10081 ,

    arXiv

  2. [2018]

    Distributionally robust optimization

    Daniel Kuhn, Soroosh Shafiee, and Wolfram Wiesemann. Distributionally robust optimization. arXiv preprint arXiv:2411.02549 ,

    arXiv

  3. [2022]

    Data-driven optimal transport cost selection for distributionally robust optimization

    Jose Blanchet, Yang Kang, Karthyek Murthy, and Fan Zhang. Data-driven optimal transport cost selection for distributionally robust optimization. In 2019 winter simulation conference (WSC), pages 3740–3751. IEEE,

    2019

  4. [2023]

    Holistic robust data-driven decisions

    Amine Bennouna and Bart Van Parys. Holistic robust data-driven decisions. arXiv preprint arXiv:2207.09560,

    arXiv

  5. [2024]

    Sinkhorn distributionally robust optimization

    Jie Wang, Rui Gao, and Yao Xie. Sinkhorn distributionally robust optimization. arXiv preprint arXiv:2109.11926,

    arXiv

  6. [5000]

    fit_transform ) ( batch ) for batch in batches ) This reduces wall-clock time significantly while keeping memory usage controlled

    ] 2 K_approx_list = Parallel ( n_jobs =4) ( 3 delayed ( nystroem . fit_transform ) ( batch ) for batch in batches ) This reduces wall-clock time significantly while keeping memory usage controlled. B.3 Constrain Reduction For MMD-DRO and Marginal-DRO that are quite time-consum...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.