Pith. sign in

REVIEW 3 major objections 4 minor 82 references

Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves that the discontinuous indicator functions inside precision, recall, and Fβ can be replaced exactly by a continuous piecewise-linear constraint, making direct metric optimization for imbalanced classification amenable to…

desk verdict Genuinely new exact lifted reformulation for indicator-based metrics, but the exactness guarantee relies on a non-singularity condition that the solver does not actually enforce. read the letter →

arxiv 2507.15240 v1 pith:UVX2LDR7 submitted 2025-07-21 cs.LG stat.ML

Le Peng , Yash Travadi , Chuan He , Ying Cui , Ju Sun This is my paper
classification cs.LGstat.ML MSC 49M3765K0590C2690C30
keywords imbalancedclassificationdirectmetricoptimizationprecision-recalltradeoffF-betascoreexactpenaltymethodsindicatorfunctionreformulationconstrainednonconvex
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Directly optimizing precision, recall, or Fβ for a binary classifier is hard because these metrics are built from indicator functions that have zero gradient almost everywhere, so existing methods approximate the indicators with smooth surrogates and lose exact control of the metric constraints. The paper claims that these indicator constraints can instead be replaced exactly by a continuous piecewise-linear condition, provided no training point sits exactly on the decision threshold, and proves that the reformulated problems are equivalent to the original FPOR, FROP, and OFBS problems in feasibility and global optimality. It then solves the reformulations with an exact penalty method and reports that this consistently finds feasible, higher-objective solutions than smooth-surrogate baselines on four benchmark datasets. If the equivalence holds, direct metric optimization becomes a standard constrained-optimization task rather than a search for good approximations, which matters for applications that must guarantee a precision or recall level.

What carries the argument

The load-bearing object is the piecewise-linear function $H_t(a,s) = s + [s + a - 1 - t]_+ - [s + a - t]_+$, whose zero-level set coincides with the zero-level set of $G_t(a,s) = s - \mathbf{1}\{a > t\}$ everywhere off the line $a = t$. It converts the non-smooth metric constraints into continuous constraints that still induce nonzero gradients on a large region of the unit square, which is what makes gradient-based optimization possible. The second mechanism is the coordinate-wise monotonicity of precision, recall, and $F_\beta$ as functions of the lifted variables $s_i$, which lets the equality constraints be relaxed to inequalities while preserving exact feasibility and optimality. The third is an exact penalty formulation that minimizes the negative objective plus a finite penalty on constraint violations, so that for a sufficiently large but finite penalty parameter a feasible point is obtained exactly, rather than only in the limit.

What would settle it

Enumerate a small binary dataset exhaustively: for every threshold $t$ and every $s \in \{0,1\}^N$, check whether feasibility and optimality of the reformulation (3.18) imply feasibility and optimality of the original problem (3.1) whenever $f_\theta(x_i) \neq t$ for all $i$; any mismatch would refute Theorems 3.8 and 3.11. Separately, instrument the ERO solver on a real dataset and record $\min_i |f_\theta(x_i) - t|$; if a returned optimum is singular, the algorithm is operating outside the theorem's premise.

Watch

Extended reading notes

Core claim

The central discovery is an exact algebraic stand-in for the thresholding constraint. For any fixed threshold $t$, the discontinuous equation $s = \mathbf{1}\{a > t\}$ is equivalent, for all $a \neq t$, to $s + [s + a - 1 - t]_+ - [s + a - t]_+ = 0$, where $[\cdot]_+$ is the positive part; the left-hand side is continuous and piecewise linear, so it has a useful Clarke subdifferential while the original indicator has none. The inequality version of this identity lets the equality be relaxed to $s \leq \mathbf{1}\{a > t\}$ for positive samples and $s \geq \mathbf{1}\{a > t\}$ for negative samples without changing the feasible set, under the same non-singularity condition. Theorems 3.8 and 3.11 (and their analogues in Appendix B) show that global solutions of the lifted reformulation (3.18) correspond exactly to global solutions of the original problem (3.1), and the same pattern covers FROP and OFBS. This makes the paper the first to optimize exact binary imbalanced-classification metrics with gradient-based methods rather than smooth surrogates.

Load-bearing premise

The whole equivalence rests on the assumption that no training sample's predicted score exactly equals the decision threshold, a condition the algorithm encourages with regularization but never hard-enforces.

Editorial extensions

If this is right

  • Global solutions of the original FPOR, FROP, and OFBS problems can be read off from global solutions of the inequality-constrained reformulations, so any globally convergent solver for the reformulation solves the original nonsmooth problem exactly.
  • Every non-singular feasible point found for the reformulation certifies feasibility of the original precision- or recall-constrained problem, something smooth-surrogate methods cannot do.
  • Because the reformulation is exact, constraint violations are optimization failures rather than approximation artifacts; the reported experiments show the exact-penalty solver returning feasible solutions with the highest objectives on all eight FPOR/FROP training tasks while baselines fail feasibility on several.
  • The general theorem in Appendix B implies the same reformulation strategy applies to any metric that is coordinate-wise monotone in the per-sample threshold indicators, including accuracy, balanced accuracy, and other count-based metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to enforce the non-singularity condition as a hard margin constraint $|f_\theta(x_i) - t| \geq \varepsilon$ rather than relying on the logit regularizer; this would keep every iterate inside the theorem's premise and might also improve generalization by forcing a decision margin, but the paper does not pursue this.
  • Because the equivalence only needs coordinate-wise monotonicity in the per-sample indicators, the same level-set identity should transfer to other count-based metrics such as precision@k, recall@k, and NDCG; the paper states this as future work, and working out the extra top-k selection constraints is the obvious next step.
  • The singular set where $f_\theta(x_i) = t$ is measure zero for a generic continuous model, so in practice the equivalence may hold automatically except at pathological optima; an empirical check of whether standard deep-learning optimizers ever land on such points would clarify how often the regularizer is actually load-bearing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies direct optimization of precision/recall/F-beta at a single operating point for binary imbalanced classification. It introduces a lifted continuous reformulation in which the indicator 1{a>t} is replaced by a piecewise-linear function H_t(a,s), proves equivalence between the original problems (3.1) and the inequality-constrained reformulation (3.18) under a non-singularity condition f_theta(x_i) != t, and solves (3.18) by an l1 exact penalty method with a logit regularization. Experiments on four datasets compare the method with WCE, TFCO, and SigmoidF1.

Significance. The reformulation idea is elegant and the proofs are self-contained; if the non-singularity condition were guaranteed, the paper would offer a principled alternative to smooth surrogates with exact metric constraints. The code is released and the ablation on the regularization is informative. However, the main exactness theorem is not connected to the solutions that Algorithm 3.1 actually returns, because non-singularity is neither enforced nor certified, and the empirical superiority claim rests on a small set of baselines.

major comments (3)
  1. [§3.2–§3.3, Eq. (3.18), (3.26), Theorems 3.8/3.11] The equivalence results are conditional on non-singularity, but the algorithm does not enforce or certify it. At any singular point a=t one computes H_t(t,s)=s+[s-1]_+ - [s]_+ = 0 for every s in [0,1], so the eta-constraints in (3.18) impose no restriction on s. Consequently, (theta,s,t) with f_theta ≡ t, s_i=1 for i in P and s_i=0 for i in N is feasible in (3.18) and attains objective 1 and precision 1, while the corresponding indicator classifier 1{f_theta>t} is identically zero and is infeasible in (3.1) for alpha>0. The regularizer psi in (3.26) is finite at f_theta(x_i)=t (e.g., s_i=1, f=0.5 gives log 0.5), so it is not a barrier; the text in §3.1 promises that a barrier-style regularization suffices to ensure f_theta(x_i) != t, but §3.3 only shows that psi encourages extreme values, and no post-hoc check such as min_i |f_theta(x_i)-t| is reported. Theorems 3.8, 3.11, B.2, and B.5 therefore do not cover feasible points that a solver can easily find. The paper should either modify the algorithm so non-singularity is guaranteed (e.g., an explicit margin constraint with a proof that exactness survives), or prove that singular points cannot be optimal for the regularized problem, or at minimum verify and report non-singularity of the returned solutions and restrict the exactness claims accordingly.
  2. [§4.1, Tables 1–3] The claim of practical superiority over state-of-the-art methods is supported by only three baselines, and one of them (WCE) is an unconstrained method that does not target DMO. The other two are smooth-surrogate methods (TFCO, SigmoidF1). Methods discussed in §2.1, such as AP-perf [24] or black-box differentiation [63], are not compared, and the paper itself notes that structural SVM-based methods form a distinct line. The empirical claim should be scoped to the considered baselines, or additional comparisons and/or ablations with a surrogate-free or exact-penalty baseline should be provided.
  3. [§3.4, Algorithm 3.1, Eq. (3.30)–(3.31)] The paper establishes equivalence between global solutions of (3.1) and (3.18), but Algorithm 3.1 only finds approximate solutions of the nonsmooth, nonconvex penalty subproblem via ADAM with a fixed iteration budget. No convergence or feasibility guarantee is proved for the sequence generated by Algorithm 3.1; the exact penalty property cited from [20] applies to exact (or sufficiently accurate) solutions of the penalty problem. Thus the global-solution equivalence does not by itself justify the solutions reported in Tables 1–3. The paper should either provide a stationarity/feasibility guarantee under appropriate assumptions, or explicitly frame the algorithm as heuristic and validate it accordingly.
minor comments (4)
  1. [§3.3, after Eq. (3.26)] The text says 'To avoid finding singular points, i.e., (theta,s,t) so that f_theta(x_i) != t for all i', but Definition 3.3 defines non-singular points as those with f_theta(x_i) != t; the phrase should be 'f_theta(x_i) = t for some i'.
  2. [Tables 1–3] The dataset name 'wildfire' is used in the tables, but Section 4.1 lists the dataset as 'Fire'; the naming should be consistent.
  3. [Appendix B] The heading 'OBFS' should be 'OFBS', and the title of Theorem B.5 refers to (3.1)/(3.18) but should refer to (B.5)/(B.6).
  4. [Eq. (3.26)] The logit regularization is undefined when f_theta(x_i) is exactly 0 or 1; the paper does not specify how these boundary values are handled in the optimizer.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reformulation is derived from a self-contained algebraic identity, and the paper's equivalence theorems do not reduce to their inputs or to load-bearing self-citation.

full rationale

The paper's central claim is that the lifted inequality reformulation (3.18) is exactly equivalent to the original FPOR problem (3.1), and similarly for FROP and OFBS, under a non-singularity condition. The derivation chain is transparent: the paper introduces auxiliary variables s, replaces the indicator constraint s_i = 1{f_theta(x_i)>t} by the continuous equation H_t(f_theta(x_i),s_i)=0 via Lemma 3.1, then relaxes equalities to inequalities via Lemma 3.5, and finally proves feasibility and global-optimality equivalence in Theorems 3.8 and 3.11 (and their general forms B.2 and B.5). Lemmas 3.1 and 3.5 are proven algebraically in Appendices A.1 and A.2 rather than imported from cited work or fitted to data; they make no use of the target optimization result. The equivalence proofs proceed by monotonicity of phi_1 and phi_2 and by rounding feasible s to the indicator vector of f_theta(x)>t, which is a constructive argument and not circular. The paper's only self-citation, [68], is an extended abstract of the same authors' preliminary work; it is invoked only to say that the augmented Lagrangian method used there 'works fine also' in an algorithmic comparison, not as evidence for exactness or as an imported uniqueness theorem. The regularization term (3.26) is presented as a heuristic to push f_theta(x_i) away from t and toward extremes; its status is empirical and tested in ablations, and the conditional nature of the equivalence under non-singularity is acknowledged rather than hidden. The fact that the solver does not certify f_theta(x_i) != t for every i is a correctness or robustness gap, not a circularity: the reformulation's exactness is a conditional mathematical statement and does not reduce to the algorithm's outputs. The empirical comparison is against external baselines (WCE, TFCO, SigmoidF1) and external datasets, so there is no fitted parameter renamed as a prediction. Overall, the derivation is self-contained and no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 2 invented entities

The central reformulation does not require fitted parameters; its correctness rests on the non-singularity axiom and the monotonicity structure of the metrics. The listed free parameters are algorithmic choices that affect practical performance but not the equivalence theorems. The lifted variable and logit regularization are internal constructs justified by proofs and ablations.

free parameters (3)
  • penalty parameter lambda = 100 (initial), multiplied by 1.3 each outer iteration
    Chosen by hand in Algorithm 3.1; controls the trade-off between feasibility and objective in the exact penalty function (3.30).
  • regularization weight gamma = 0.5 * 1.3^k at outer iteration k
    Chosen by hand in Section 4.2 (implementation details); controls the logit regularization (3.26) to avoid singular points.
  • decision threshold t = 0.5
    Fixed in experiments (Section 4.1) rather than optimized; model bias is adjusted instead.
assumptions (4)
  • ad hoc to paper Non-singularity fθ(x_i) ≠ t for all training samples i
    Required for Lemma 3.1, Lemma 3.5, and Theorems 3.8, 3.11, B.2, B.5; introduced in Definition 3.3. The algorithm only promotes this via logit regularization and does not enforce it.
  • domain assumption Monotonicity of the target metrics in the lifted variables: recall, precision, and Fβ are non-decreasing in s_i for positive-class samples and non-increasing for negative-class samples
    Proven as Fact 3.9/B.3. This property makes the inequality relaxation (3.18) exact; it holds for the three studied metrics but is a structural condition for the general DMO claim in Theorem B.6.
  • domain assumption The training data is i.i.d. from a distribution; empirical constraints approximate population-level constraints
    Standard ML assumption stated in Section 2.1; test-time feasibility is only approximate because constraints are sample-level (acknowledged in Section 4.2).
  • standard math Exact penalty method properties: a feasible point is found for finite penalty parameter λ
    Relies on Han and Mangasarian (1979); in the nonconvex setting with approximate subproblem solves this is not guaranteed, but the paper uses it as the basis of Algorithm 3.1.
invented entities (2)
  • Auxiliary lifted variable s_i in [0,1] per sample
    purpose: Encodes the predicted label relative to threshold t in the reformulated problem (3.18).
    Internal mathematical lifting; its validity is established by the equivalence proofs rather than by an external falsifiable prediction.
  • Logit regularization term ψ(θ,s)
    purpose: Encourages fθ(x_i) to take extreme values (near 0 or 1) and align with s_i, avoiding singular points fθ(x_i)=t and improving numerical behavior.
    Introduced in Section 3.3 as a heuristic regularizer; its benefit is demonstrated in ablation (Table 5) but it is not independently testable outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification." pith.science (2026). https://pith.science/paper/UVX2LDR7

@misc{pith2026250715240,
  author       = {Pith},
  title        = {Pith review of: Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UVX2LDR7}},
  note         = {Machine review of arXiv:2507.15240}
}
abstract

For classification with imbalanced class frequencies, i.e., imbalanced classification (IC), standard accuracy is known to be misleading as a performance measure. While most existing methods for IC resort to optimizing balanced accuracy (i.e., the average of class-wise recalls), they fall short in scenarios where the significance of classes varies or certain metrics should reach prescribed levels. In this paper, we study two key classification metrics, precision and recall, under three practical binary IC settings: fix precision optimize recall (FPOR), fix recall optimize precision (FROP), and optimize $F_\beta$-score (OFBS). Unlike existing methods that rely on smooth approximations to deal with the indicator function involved, \textit{we introduce, for the first time, exact constrained reformulations for these direct metric optimization (DMO) problems}, which can be effectively solved by exact penalty methods. Experiment results on multiple benchmark datasets demonstrate the practical superiority of our approach over the state-of-the-art methods for the three DMO problems. We also expect our exact reformulation and optimization (ERO) framework to be applicable to a wide range of DMO problems for binary IC and beyond. Our code is available at https://github.com/sun-umn/DMO.

Figures

Figures reproduced from arXiv: 2507.15240 by the authors.

Figure 1
Figure 1. Data distribution, classifier, and surrogates for t = 0.2, T ∈ {1, 2, 10} To tackle Challenge 2, optimization methods capable of reliably handling nonlinear constraints are needed. Penaliza￾tion methods, including penalty methods, Lagrangian methods, and augmented Lagrangian methods (ALMs), have been popu￾larly used for this purpose [54]. For example, the TensorFlow￾based library TFCO [16] has implemented Lagrangian… view at source ↗
Figure 2
Figure 2. Illustration of issues with using smooth approximations/surrogates when solving [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Heatmap visualization of Gt and Ht for t = 0.3, their 0-level sets L0(Gt) and L0(ft), as well as their gradient fields. Note that for our purposes, t ∈ [0, 1] and a ∈ [0, 1], s ∈ [0, 1]. Clearly, (3.7) L0(Gt) = {(a, s) : s − 1{a > t} = 0} , (3.8) L0(Ht) = {(a, s) : s + [s + a − 1 − t]+ − [s + a − t]+ = 0} . The following result can be observed directly from [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Contour plot of r(a, s) .= log |R(a, s)| To see why this works, recall that s ∈ [0, 1]N and fθ : X → [0, 1]. Consider the function R(a, s) .= a log s + (1 − a) log(1 − s) over [0, 1] × [0, 1]. It is maximized when a = s = 0 and a = s = 1; see [PITH_FULL_IMAGE:figures/…
Figure 5
Figure 5. Figure 5: Histograms of the normalized prediction logits with and without the proposed logit [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

82 extracted references · 71 canonical work pages

  1. [24]

    F athony and Z

    R. F athony and Z. Kolter, Ap-perf: Incorporating generic performance metrics in differentiable learn- ing, in International Conference on Artificial Intelligence and Statistics, PMLR, 2020, pp. 4130–4140

    work page 2020

  2. [63]

    Rol´ınek, V

    M. Rol´ınek, V. Musil, A. Paulus, M. Vlastelica, C. Michaelis, and G. Martius , Optimizing rank-based metrics with blackbox differentiation , in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 7620–7630

    work page 2020

  3. [20]

    Di Pillo, Exact penalty methods, Algorithms for continuous optimization: the state of the art, (1994), pp

    G. Di Pillo, Exact penalty methods, Algorithms for continuous optimization: the state of the art, (1994), pp. 209–253

    work page 1994

  4. [1]

    H. S. Ahmed Gamaleldin, Ahmed Atef and A. Shaheen, firedataset, 2020, https://www.kaggle.com/ datasets/phylake1337/fire-dataset/data

    2020

  5. [2]

    Alacaoglu and S

    A. Alacaoglu and S. J. Wright , Complexity of single loop algorithms for nonlinear programming with stochastic objective and constraints , in International Conference on Artificial Intelligence and Statistics, PMLR, 2024, pp. 4627–4635

    2024

  6. [3]

    P. L. Bartlett, A. Montanari, and A. Rakhlin, Deep learning: a statistical viewpoint , Acta numer- ica, 30 (2021), pp. 87–201

    work page 2021

  7. [4]

    Belkin, Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation, Acta Numerica, 30 (2021), pp

    M. Belkin, Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation, Acta Numerica, 30 (2021), pp. 203–248

    work page 2021

  8. [5]

    sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classification

    G. B´en´edict, V. Koops, D. Odijk, and M. de Rijke , sigmoidf1: A smooth f1 score surrogate loss for multilabel classification, arXiv preprint arXiv:2108.10566, (2021)

    work page Pith review arXiv 2021

Show all 82 references
  1. [6]

    D. P. Bertsekas, Nonlinear Programming, Athena Scientific, Belmont, Massachusetts, 3rd ed., 2016

    2016

  2. [7]

    Brown, W

    A. Brown, W. Xie, V. Kalogeiton, and A. Zisserman , Smooth-ap: Smoothing the path towards large-scale image retrieval , in European Conference on Computer Vision, Springer, 2020, pp. 677– 694

    2020

  3. [8]

    R. H. Byrd, J. Nocedal, and R. A. W altz, KNITRO: An integrated package for nonlinear optimiza- tion, Large-Scale Nonlinear Optimization, (2006), pp. 35–59

    2006

  4. [9]

    Cakir, K

    F. Cakir, K. He, X. Xia, B. Kulis, and S. Sclaroff , Deep metric learning to rank , in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 1861–1870

    2019

  5. [10]

    Chapelle and M

    O. Chapelle and M. Wu , Gradient descent optimization of smoothed information retrieval metrics , Information retrieval, 13 (2010), pp. 216–235

    2010

  6. [11]

    K. Chen, W. Lin, J. Li, J. See, J. W ang, and J. Zou, Ap-loss for accurate one-stage object detection , IEEE Transactions on Pattern Analysis and Machine Intelligence, 43 (2020), pp. 3782–3798

    2020

  7. [12]

    F. H. Clarke , Optimization and nonsmooth analysis , SIAM, 1990

    1990

  8. [13]

    Clason and T

    C. Clason and T. V alkonen , Introduction to nonsmooth analysis and optimization , arXiv preprint arXiv:2001.00216, (2020)

    2020 arXiv

  9. [14]

    N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, H. Kittler, et al. , Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the i...

    2017

  10. [15]

    Cotter, M

    A. Cotter, M. Gupta, H. Jiang, N. Srebro, K. Sridharan, S. W ang, B. Woodworth, and S. You , Training well-generalizing classifiers for fairness metrics and other data-dependent con- straints, in International Conference on Machine Learning, PMLR, 2019, pp. 1397–1405

    2019

  11. [16]

    Cotter, H

    A. Cotter, H. Jiang, and K. Sridharan , Two-player games for efficient non-convex constrained optimization, in Algorithmic Learning Theory, PMLR, 2019, pp. 300–332

    2019

  12. [17]

    Y. Cui, X. W ang, and X. Xiao , A two-phase stochastic momentum-based algorithm for nonconvex expectation-constrained optimization, Journal of Scientific Computing, 104 (2025), pp. 1–27

    2025

  13. [18]

    R. R. Curtin, M. Edel, R. G. Prabhu, S. Basak, Z. Lou, and C. Sanderson, The ensmallen library for flexible numerical optimization. , J. Mach. Learn. Res., 22 (2021), pp. 166–1

    2021

  14. [19]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, Bert: Pre-training of deep bidirectional trans- 20 L. PENG, Y. TRA VADI, C. HE, Y. CUI AND J. SUN formers for language understanding , arXiv preprint arXiv:1810.04805, (2018)

    2018 arXiv

  15. [21]

    E. Eban, M. Schain, A. Mackey, A. Gordon, R. Rifkin, and G. Elidan , Scalable learning of non-decomposable objectives, in Artificial intelligence and statistics, PMLR, 2017, pp. 832–840

    2017

  16. [22]

    Emma Dugas, Jared and W

    J. Emma Dugas, Jared and W. Cukierski , Diabetic retinopathy detection, 2015, https://kaggle.com/ competitions/diabetic-retinopathy-detection

    2015

  17. [23]

    Engilberge, L

    M. Engilberge, L. Chevallier, P. P ´erez, and M. Cord , Sodeep: a sorting deep net to learn rank- ing loss surrogates , in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 10792–10801

    2019

  18. [25]

    Fern´andez, S

    A. Fern´andez, S. Garcia, F. Herrera, and N. V. Chawla, Smote for learning from imbalanced data: progress and challenges, marking the 15-year anniversary , Journal of artificial intelligence research, 61 (2018), pp. 863–905

    2018

  19. [26]

    Gurulingappa, A

    H. Gurulingappa, A. M. Rajput, A. Roberts, J. Fluck, M. Hofmann-Apitius, and L. Toldo , Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports , Journal of Biomedical Informatics, 45 (2012), pp. 885 – 892...

    2012 doi

  20. [27]

    S. P. Han and O. L. Mangasarian , Exact penalty functions in nonlinear programming , Mathematical programming, 17 (1979), pp. 251–269

    1979

  21. [28]

    C. He, L. Peng, and J. Sun , Federated learning with convex global and local constraints , Transactions on machine learning research, 2024 (2024), pp. https–openreview

    2024

  22. [29]

    Herland, T

    M. Herland, T. M. Khoshgoftaar, and R. A. Bauder , Big data fraud detection using multiple medicare data sources, Journal of Big Data, 5 (2018), pp. 1–21

    2018

  23. [30]

    Huang, S

    C. Huang, S. Zhai, P. Guo, and J. Susskind , Metricopt: Learning to optimize black-box evaluation metrics, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 174–183

    2021

  24. [31]

    Irtaza, S

    A. Irtaza, S. M. Adnan, K. T. Ahmed, A. Jaffar, A. Khan, A. Javed, and M. T. Mahmood , An ensemble based evolutionary approach to the class imbalance problem with applications in cbir , Applied Sciences, 8 (2018), p. 495

    2018

  25. [32]

    Irvin, P

    J. Irvin, P. Rajpurkar, M. Ko, Y. Yu, S. Ciurea-Ilcus, C. Chute, H. Marklund, and et al. , Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison , in Proceed- ings of the AAAI conference on artificial intelligence, vol. 33, 2019, pp. 590–597

    2019

  26. [33]

    Joachims, A support vector method for multivariate performance measures, in Proceedings of the 22nd international conference on Machine learning, 2005, pp

    T. Joachims, A support vector method for multivariate performance measures, in Proceedings of the 22nd international conference on Machine learning, 2005, pp. 377–384

    2005

  27. [34]

    J. M. Johnson and T. M. Khoshgoftaar , Survey on deep learning with class imbalance , Journal of Big Data, 6 (2019), pp. 1–54

    2019

  28. [35]

    P. Kar, H. Narasimhan, and P. Jain , Surrogate functions for maximizing precision at the top , in International Conference on Machine Learning, PMLR, 2015, pp. 189–198

    2015

  29. [36]

    Khatami, M

    A. Khatami, M. Babaie, A. Khosravi, H. R. Tizhoosh, and S. Nahavandi , Parallel deep solutions for image retrieval from imbalanced medical imaging archives , Applied Soft Computing, 63 (2018), pp. 197–205

    2018

  30. [37]

    Kumar, H

    A. Kumar, H. Narasimhan, and A. Cotter , Implicit rate-constrained optimization of non- decomposable objectives, in International Conference on Machine Learning, PMLR, 2021, pp. 5861– 5871

    2021

  31. [38]

    S. Laue, M. Blacher, and J. Giesen, Optimization for classical machine learning problems on the gpu , in Proceedings of the AAAI conference on artificial intelligence, 2022, pp. 7300–7308

    2022

  32. [39]

    S. Laue, M. Mitterreiter, and J. Giesen , Geno–generic optimization for classical machine learning , Advances in Neural Information Processing Systems, 32 (2019)

    2019

  33. [40]

    N. Lee, H. Yang, and H. Yoo , A surrogate loss function for optimization of fβ score in binary classi- fication with imbalanced data , arXiv preprint arXiv:2104.01459, (2021)

    2021 arXiv

  34. [41]

    Z. Li, K. Kamnitsas, and B. Glocker , Analyzing overfitting under class imbalance in neural networks EXACT REFORMULATION AND OPTIMIZATION FOR BIC 21 for image segmentation , IEEE transactions on medical imaging, 40 (2020), pp. 1065–1077

    2020

  35. [42]

    Liang, T

    B. Liang, T. Mitchell, and J. Sun , Ncvx: A general-purpose optimization solver for constrained machine and deep learning , arXiv preprint arXiv:2210.00973, (2022)

    2022 arXiv

  36. [43]

    Liang, T

    H. Liang, T. Li, and J. Sun , A baseline method for removing invisible image watermarks using deep image prior , Transactions on Machine Learning Research, (2025)

    2025

  37. [44]

    Liang, B

    H. Liang, B. Liang, L. Peng, Y. Cui, T. Mitchell, and J. Sun , Optimization and optimizers for adversarial robustness, arXiv preprint arXiv:2303.13401, (2023)

    2023 arXiv

  38. [45]

    Liang, L

    H. Liang, L. Peng, and J. Sun , Selective classification under distribution shifts , Transactions on Ma- chine Learning Research, (2024)

    2024

  39. [46]

    T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, Focal loss for dense object detection , in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988

    2017

  40. [47]

    Z. C. Lipton, C. Elkan, and B. Narayanaswamy, Thresholding classifiers to maximize f1 score , stat, 1050 (2014), p. 14

    2014

  41. [48]

    Z. Lu, S. Mei, and Y. Xiao , Variance-reduced first-order methods for deterministically constrained sto- chastic nonconvex optimization with strong convergence guarantees, arXiv preprint arXiv:2409.09906, (2024)

    2024 arXiv

  42. [49]

    Lu and Y

    Z. Lu and Y. Xiao , First-order methods for stochastic and finite-sum convex optimization with deter- ministic constraints, arXiv preprint arXiv:2506.20630, (2025)

    2025 arXiv

  43. [50]

    Majnik and Z

    M. Majnik and Z. Bosni ´c, Roc analysis of classifiers in machine learning: A survey , Intelligent data analysis, 17 (2013), pp. 531–558

    2013

  44. [51]

    Menon, H

    A. Menon, H. Narasimhan, S. Agarwal, and S. Chawla, On the statistical consistency of algorithms for binary classification under class imbalance , in International Conference on Machine Learning, PMLR, 2013, pp. 603–611

    2013

  45. [52]

    Y. Nan, K. M. Chai, W. S. Lee, and H. L. Chieu , Optimizing f-measure: A tale of two approaches , arXiv preprint arXiv:1206.4625, (2012)

    2012 arXiv

  46. [53]

    Narasimhan, A

    H. Narasimhan, A. Cotter, and M. Gupta , Optimizing generalized rate metrics with three players , Advances in Neural Information Processing Systems, 32 (2019)

    2019

  47. [54]

    Nocedal and S

    J. Nocedal and S. J. Wright , Numerical Optimization, Springer, NY, USA, 2nd ed., 2006

    2006

  48. [55]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. , Dinov2: Learning robust visual features without supervision, arXiv preprint arXiv:2304.07193, (2023)

    2023 arXiv

  49. [56]

    Patel, G

    Y. Patel, G. Tolias, and J. Matas , Recall@ k surrogate loss with large batches and similarity mixup , in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 7502–7511

    2022

  50. [57]

    L. Peng, Y. Travadi, R. Zhang, Y. Cui, and J. Sun, Imbalanced classification in medical imaging via regrouping, NeurIPS Workshop on Medical Imaging Meets NeurIPS, (2022)

    2022

  51. [58]

    M. V. Poganci ´c, A. Paulus, V. Musil, G. Martius, and M. Rolinek , Differentiation of blackbox combinatorial solvers, in International Conference on Learning Representations, 2019

    2019

  52. [59]

    Puthiya Parambath, N

    S. Puthiya Parambath, N. Usunier, and Y. Grandvalet , Optimizing f-measures by cost-sensitive classification, Advances in neural information processing systems, 27 (2014)

    2014

  53. [60]

    Q. Qi, Y. Luo, Z. Xu, S. Ji, and T. Yang , Stochastic optimization of areas under precision-recall curves with provable convergence , Advances in Neural Information Processing Systems, 34 (2021), pp. 1752–1765

    2021

  54. [61]

    Qin, T.-Y

    T. Qin, T.-Y. Liu, and H. Li, A general approximation framework for direct optimization of information retrieval measures, Information retrieval, 13 (2010), pp. 375–397

    2010

  55. [62]

    Rath and M

    P. Rath and M. Hughes , Optimizing early warning classifiers to control false alarms via a minimum precision constraint, in International Conference on Artificial Intelligence and Statistics, PMLR, 2022, pp. 4895–4914

    2022

  56. [64]

    Saito and M

    T. Saito and M. Rehmsmeier , The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets , PloS one, 10 (2015), p. e0118432

    2015

  57. [65]

    Sanyal, P

    A. Sanyal, P. Kumar, P. Kar, S. Chawla, and F. Sebastiani , Optimizing non-decomposable mea- 22 L. PENG, Y. TRA VADI, C. HE, Y. CUI AND J. SUN sures with deep networks , Machine Learning, 107 (2018), pp. 1597–1620

    2018

  58. [66]

    S. A. Taghanaki, Y. Zheng, S. K. Zhou, B. Georgescu, P. Sharma, D. Xu, D. Comaniciu, and G. Hamarneh, Combo loss: Handling input and output imbalance in multi-organ segmentation , Computerized Medical Imaging and Graphics, 75 (2019), pp. 24–33

    2019

  59. [67]

    X. Tong, Y. Feng, and A. Zhao, A survey on neyman-pearson classification and suggestions for future research, Wiley Interdisciplinary Reviews: Computational Statistics, 8 (2016), pp. 64–81

    2016

  60. [68]

    Travadi, L

    Y. Travadi, L. Peng, Y. Cui, and J. Sun , Direct metric optimization for imbalanced classification , in 2023 IEEE 11th International Conference on Healthcare Informatics (ICHI), IEEE, 2023, pp. 698–700

    2023

  61. [69]

    Tsochantaridis, T

    I. Tsochantaridis, T. Joachims, T. Hofmann, Y. Altun, and Y. Singer, Large margin methods for structured and interdependent output variables. , Journal of machine learning research, 6 (2005)

    2005

  62. [70]

    Vishwakarma, Y

    H. Vishwakarma, Y. Chen, S. J. Tay, S. S. S. Namburi, F. Sala, and R. Korlakai Vinayak , Pearls from pebbles: Improved confidence functions for auto-labeling , Advances in Neural Information Processing Systems, 37 (2024), pp. 15983–16015

    2024

  63. [71]

    W¨achter and L

    A. W¨achter and L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Mathematical programming, 106 (2006), pp. 25–57

    2006

  64. [72]

    Wei and K

    J. Wei and K. Zou , Eda: Easy data augmentation techniques for boosting performance on text classifi- cation tasks, arXiv preprint arXiv:1901.11196, (2019)

    2019 arXiv

  65. [73]

    W. Wei, J. Li, L. Cao, Y. Ou, and J. Chen , Effective detection of sophisticated online banking fraud on extremely imbalanced data , World Wide Web, 16 (2013), pp. 449–475

    2013

  66. [74]

    P. Wen, Q. Xu, Z. Yang, Y. He, and Q. Huang , Exploring the algorithm-dependent generalization of auprc optimization with list stability , Advances in Neural Information Processing Systems, 35 (2022), pp. 28335–28349

    2022

  67. [75]

    Werner, A review on instance ranking problems in statistical learning , Machine Learning, 111 (2022), pp

    T. Werner, A review on instance ranking problems in statistical learning , Machine Learning, 111 (2022), pp. 415–463

    2022

  68. [76]

    C. K. Williams, The effect of class imbalance on precision-recall curves, Neural Computation, 33 (2021), pp. 853–857

    2021

  69. [77]

    R. Xie, R. Barber, and E. Candes , Boosted conformal prediction intervals, Advances in Neural Infor- mation Processing Systems, 37 (2024), pp. 71868–71899

    2024

  70. [78]

    L. Yang, H. Jiang, Q. Song, and J. Guo , A survey on long-tailed visual recognition , International Journal of Computer Vision, (2022), pp. 1–36

    2022

  71. [79]

    Yang, Algorithmic foundation of deep x-risk optimization , arXiv preprint arXiv:2206.00439, (2022)

    T. Yang, Algorithmic foundation of deep x-risk optimization , arXiv preprint arXiv:2206.00439, (2022)

    2022 arXiv

  72. [80]

    Yang and Y

    T. Yang and Y. Ying , Auc maximization in the era of big data and ai: A survey , ACM Computing Surveys, 55 (2022), pp. 1–37

    2022

  73. [81]

    Yeung, E

    M. Yeung, E. Sala, C.-B. Sch ¨onlieb, and L. Rundo , Unified focal loss: Generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation , Computerized Medical Imaging and Graphics, 95 (2022), p. 102026

    2022

  74. [82]

    1”) and negative (“0

    Y. Yue, T. Finley, F. Radlinski, and T. Joachims , A support vector method for optimizing average precision, in Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, 2007, pp. 271–278. Appendix A. Proofs of auxi...

    2007

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.