Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Stochastic AUC Maximization with Deep Neural Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper extends provable AUC maximization from linear scoring models to deep neural networks, achieving an O~(1/ε) stochastic convergence rate through a proximal primal-dual method that exploits the Polyak-Łojasiewicz condition.

desk verdict First O~(1/epsilon) rates for deep AUC via PL in a primal-dual setting, with an honest but load-bearing unverified PL assumption. read the letter →

arxiv 1908.10831 v5 pith:QLE64HXU submitted 2019-08-28 cs.LG math.OCstat.ML

Mingrui Liu , Zhuoning Yuan , Yiming Ying , Tianbao Yang This is my paper · ORCID
classification cs.LGmath.OCstat.ML
keywords AUCmaximizationdeepneuralnetworksnon-convexconcavemin-maxoptimizationPolyak-Łojasiewiczconditionprimal-dualstochasticgradientAdaGradimbalancedclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AUC maximization, the standard objective for learning from imbalanced data, had provable stochastic algorithms only for linear scoring models, leaving deep networks without convergence guarantees for this objective. This paper claims to close that gap: it rewrites the squared-loss surrogate of AUC as a saddle-point problem (non-convex min over network weights plus two offsets, concave max over one dual scalar) and proves that two stochastic primal-dual algorithms reach an ε-accurate objective under the Polyak-Łojasiewicz condition. PPD-SG uses geometrically decaying step sizes and a closed-form dual refresh to achieve $\widetilde{O}(L G^2/(\mu^2 \epsilon))$ iterations and $\widetilde{O}(L^3 \sigma^2/(\mu^2 \epsilon))$ samples; PPD-AdaGrad achieves an adaptive $\widetilde{O}((L\delta^2 d/(\mu^2 \epsilon))^{1/(2(1-\alpha))})$ rate when cumulative gradients grow slowly. Both rates improve the generic $\widetilde{O}(1/\epsilon^3)$ bound for one-sided non-convex concave min-max problems, and experiments on four image benchmarks with a ResNet-20 support the effectiveness claim.

What carries the argument

Four objects carry the argument. The saddle-point reformulation (Proposition 1) expresses the pairwise AUC probability through a min over weights $(w,a,b)$ and a max over a single dual scalar $\alpha$, making the stochastic gradient computable from one example. On top of it, the Polyak-Łojasiewicz condition $\mu(\varphi(v)-\varphi(v^*)) \le \frac12\|\nabla\varphi(v)\|^2$ is the engine: it asserts that small gradients force near-optimality of $\varphi$, which turns the per-epoch suboptimality bound of each proximal solve into a geometric contraction of the gap across epochs, and Lemma 1 shows this property is inherited from a PL condition on the plain minimization objective $P(w)$. The algorithmic skeleton is the inexact proximal point method — each outer loop approximately solves the strongly convex-concave subproblem $\min_v\max_\alpha \{f(v,\alpha)+\frac{1}{2\gamma}\|v-\bar v_{k-1}\|^2\}$; PPD-SG differs from the generic template in its geometrically decaying step size $\eta_k=\eta_0\exp(-(k-1)(\mu/L)/(5+\mu/L))$ and its minibatch estimate of the optimal dual variable. PPD-AdaGrad swaps the inner primal-dual update for an AdaGrad-preconditioned step with $H_t=\delta I+\mathrm{diag}(s_t)$, whose per-coordinate scaling converts a bound on cumulative gradient growth into an adaptive iteration count.

What would settle it

Run PPD-SG on a deep network (e.g., ResNet-20 on an imbalanced CIFAR10 split) and along the trajectory estimate the surrogate gap $\varphi(v)-\varphi(v^*)$ and the squared gradient norm $\|\nabla\varphi(v)\|^2$; if the effective PL coefficient $\|\nabla\varphi(v)\|^2/(2(\varphi(v)-\varphi(v^*)))$ is near zero or shrinks toward zero as training proceeds, the premise fails and the claimed $\widetilde{O}(1/\epsilon)$ rate cannot hold. A sharper, easier disproof: exhibit a dataset and architecture for which the squared-loss AUC surrogate has a spurious local minimum, since PL forbids that by definition.

Watch

Extended reading notes

Core claim

The paper's central claim is that the Polyak-Łojasiewicz (PL) gradient-dominance property, known to hold for several classes of overparameterized networks trained with square loss, transfers to the AUC min-max surrogate and unlocks provably faster convergence for deep AUC maximization. The route is: replace the AUC indicator by the squared loss, which by Proposition 1 turns the pairwise objective $\min_w P(w)$ into $\min_{w,a,b}\max_{\alpha} \mathbb{E}_z[F(w,a,b,\alpha;z)]$, a non-convex-concave stochastic min-max problem in which each gradient step consumes one sample. Assuming $\mu(\varphi(v)-\varphi(v^*)) \le \frac12 \|\nabla\varphi(v)\|^2$ for $\varphi(v)=\max_\alpha f(v,\alpha)$ (Assumption 1(1)), Theorem 2 shows that PPD-SG returns a point with $\mathbb{E}[\varphi(\bar v_K)-\varphi(v^*)]\le\epsilon$ in $\widetilde{O}(L G^2/(\mu^2\epsilon))$ iterations using $\widetilde{O}(L^3\sigma^2/(\mu^2\epsilon))$ samples, where the two algorithmic keys are a geometrically decaying step size across outer loops and a minibatch refresh of the dual variable to its closed-form optimum $\alpha^*=\mathbb{E}[h(w;x)|y=-1]-\mathbb{E}[h(w;x)|y=1]$. Theorem 3 gives the AdaGrad-style variant a matching sample bound and an adaptive iteration count $\widetilde{O}((L\delta^2 d/(\mu^2\epsilon))^{1/(2(1-\alpha))})$ under a growth condition on cumulative gradients. The paper presents these as the first non-asymptotic guarantees for stochastic AUC maximization with deep networks and as strictly better than the $\widetilde{O}(1/\epsilon^3)$ cost of the generic proximal framework.

Load-bearing premise

The load-bearing premise is the Polyak-Łojasiewicz condition on the AUC surrogate objective, $\mu(\varphi(v)-\varphi(v^*)) \le \frac12\|\nabla\varphi(v)\|^2$: the assumption that every low-gradient point is within a proportional gap of the global minimum, which the paper supports only with PL results for square-loss networks and a one-hidden-layer example rather than a proof or empirical check for deep networks, and without which all claimed $\widetilde{O}(1/\epsilon)$ and adaptive rates fall back to the generic $\widetilde{O}(1/\epsilon^3)$.

Editorial extensions

If this is right

  • If the PL condition holds, deep AUC maximization is asymptotically no harder than plain SGD training: the $\widetilde{O}(1/\epsilon)$ iteration rate matches the standard stochastic-gradient rate under PL, and the sample count $\widetilde{O}(L^3\sigma^2/(\mu^2\epsilon))$ is independent of the model dimension.
  • The generic inexact proximal framework for non-convex concave min-max problems costs $\widetilde{O}(1/\epsilon^3)$; this paper's rates improve on that in both $\epsilon$ and the dependence on the conditioning parameters, so any one-sided min-max problem whose outer objective satisfies PL inherits the speedup.
  • PPD-AdaGrad supplies the first adaptive convergence guarantee for non-convex concave min-max problems: when cumulative gradient norms grow like $\delta T^\alpha$ with $\alpha < 1/2$, its iteration bound $\widetilde{O}((L\delta^2 d/(\mu^2\epsilon))^{1/(2(1-\alpha))})$ beats the non-adaptive rate.
  • The closed-form dual refresh, estimating $\alpha^*=\mathbb{E}[h(w;x)|y=-1]-\mathbb{E}[h(w;x)|y=1]$ from a small minibatch, removes dual-error accumulation and extends to any min-max problem with a conditional-expectation maximizer.
  • The same framework covers multi-class AUC and online settings with unknown class prior (Section 4.3), so the rates carry over with only the overhead of maintaining estimators for $p$ and $p(1-p)$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The empirical value of the construction rests on how often the PL condition actually holds along real training trajectories; since the paper's PL evidence is by analogy (square-loss networks, a one-hidden-layer example), a direct measurement of the effective PL coefficient during PPD-SG runs on deep networks would settle the question.
  • A natural extension the paper does not explore: combining the geometric-restart schedule with variance-reduced inner loops (an SVRG-style estimator) could plausibly remove the $\sigma^2$ sample-factor, since the contraction argument only needs unbiased gradients.
  • The same saddle-point-plus-PL blueprint should transfer to other pairwise objectives, such as ranking surrogates and metric learning, whenever the inner maximization is convex and admits a closed form; the dual-refresh trick is the piece that generalizes most directly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies stochastic AUC maximization when the scoring function is a deep neural network. It uses a squared surrogate loss and the saddle-point reformulation of Ying et al. to cast the problem as a nonconvex-concave min-max problem in the primal variables (w,a,b) and dual variable α. The authors propose two stochastic primal-dual algorithms built on the proximal-point framework of Rafique et al.: PPD-SG, which uses a geometrically decaying step-size sequence and a closed-form dual correction, and PPD-AdaGrad, which uses AdaGrad-style adaptive preconditioning. Under Assumption 1, which includes a Polyak-Łojasiewicz condition on φ(v)=max_α f(v,α), they prove an O~(LG^2/(μ^2 ε)) iteration complexity and O~(L^3 σ^2/(μ^2 ε)) sample complexity for PPD-SG (Theorem 2), and for PPD-AdaGrad an iteration bound of O~((L δ^2 d/(μ^2 ε))^{1/(2(1-α))}) under a cumulative-gradient-growth condition (Theorem 3). Experiments on Cat&Dog, CIFAR10, CIFAR100, and STL10 compare test AUC against PGA, OAUC, and SGD baselines.

Significance. The conditional rates are substantial improvements over the O(1/ε^3) baseline from Rafique et al., and the proof structure is careful: the one-epoch analyses in Lemmas 2 and 3, the use of strong convexity of the proximal subproblem, and the function-gap contraction via the PL condition are transparent and internally consistent. The paper is honest about the main gap: the remark after Lemma 1 states that proving PL for P(w) is not the focus, and Appendix A.7 gives only a one-hidden-layer example. The idea of exploiting PL in primal-dual nonconvex-concave optimization, and the adaptive AdaGrad-style extension, are likely to be useful beyond AUC maximization. The main weakness is that the advertised improvements for deep networks are conditional on an unverified landscape condition and on unknown constants, so the contribution is best understood as a conditional complexity analysis rather than an end-to-end guarantee for deep AUC maximization as implemented.

major comments (3)
  1. [Assumption 1(1), Lemma 1, Appendix A.7] The advertised O~(1/ε) iteration and sample rates in Theorems 2 and 3 all require the PL condition μ(φ(v)−φ(v*)) ≤ (1/2)‖∇φ(v)‖^2. The paper does not establish this condition for the deep AUC surrogate objective (1) with networks such as ResNet-20. Lemma 1 only transfers PL from P(w) to φ, and the remark after Lemma 1 explicitly defers the proof of PL for P(w); Theorem 4 in Appendix A.7 covers only a one-hidden-layer Leaky ReLU network under zero-mean conditional features and E[xx'^T | y=1, y'=−1] = 0, which does not cover the experimental architectures or data. The experiments report only test AUC, not the ratio (φ(v)−φ(v*))/‖∇φ(v)‖^2, so the central rate claims rest on an unverified condition. I request that the authors either prove PL for a broader class of deep AUC objectives relevant to the experiments, or add an empirical measurement of the PL ratio during training on the benchmark datasets, and if this is not possible, state explicitly in the abstract and introduction that all rates are conditional on a PL assumption that is not verified for the studied models.
  2. [Sections 4.1, 4.3, and 5] The theoretical schedules in Theorems 2 and 3 are specified in terms of unknown constants μ, L, G, and σ; Section 4.3 only provides heuristics such as decreasing η by a constant factor when validation performance saturates. The experiments in Section 5 use η_s = η0/3^s and T_s = T0·3^s, which is not the exact schedule exp(−(k−1) μ/L / (5+μ/L)) required by the theorems, and no estimates of μ, L, G, or σ are reported. Consequently, the experimental results do not directly instantiate the theoretical guarantees. Please either provide a parameter-free or easily estimable schedule with a matching guarantee, or clearly separate the heuristic implementation from the theoretical algorithm and explain why the reported experiments are a meaningful test of the conditional rates.
  3. [Theorem 3] The claimed adaptive advantage of PPD-AdaGrad rests on the assumption ‖ĝ_{1:T_k,i}‖_2 ≤ δ T_k^α for 0 ≤ α ≤ 1/2. This is an assumption on the realized stochastic gradient path that is neither derived from the model class nor verified empirically; the paper does not report cumulative gradient norms on the benchmark tasks. As with the PL condition, the adaptive improvement is therefore conditional on an unmeasured quantity. I ask the authors to state this caveat more prominently and, if possible, to provide empirical evidence about the growth exponent α on the datasets considered.
minor comments (4)
  1. [Algorithm 2, Step 12] The dual correction in Step 12 divides by the numbers of negative and positive examples in the minibatch; if a minibatch happens to contain no positive or no negative example, the denominators are zero. The paper should state how this event is handled in the implementation or in the analysis.
  2. [Figures 1 and 3] In the version I received, the legends of Figures 1 and 3 contain garbled text (e.g., 'uni00000006/...'), which makes it impossible to identify which curve corresponds to which method; please replace these with readable labels.
  3. [Section 4.3] The extension for unknown p claims that the estimators for p and p(1−p) are unbiased, but the update rule in Algorithm 4 is described without a proof of unbiasedness; a short derivation or a reference would help.
  4. [Appendix A.8] There is a typo in the dataset description: 'positve' should be 'positive'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence rates are conditional theorems from explicit assumptions, and no prediction reduces to a fitted input or to a self-citation chain.

full rationale

The paper's central claims (Theorems 2 and 3) are of the form: under Assumption 1, Algorithm 2 (resp. Algorithm 3) returns v-bar_K with E[phi(v-bar_K)-phi(v*)] <= epsilon after the stated iteration and sample counts. Assumption 1(1) is a Polyak-Lojasiewicz condition on phi, and the proofs use it explicitly as a premise, e.g., in Eqs. (16) and (32) with the words "(b) holds by the PL property of phi." This is an assumption, not a conclusion derived from the target rate; there is no fitted parameter that is later renamed as a prediction. The min-max reformulation (Proposition 1) is proved from the squared-loss AUC surrogate, and the algorithms are not defined in terms of phi(v*) or the final guarantee. The self-citations to Rafique et al. (2018), Liu et al. (2018), and Chen et al. (2019) supply the proximal-point template and AdaGrad tools, but the one-epoch Lemmas 2 and 3 and the recursion analyses in Appendices A.2-A.5 are carried out in the paper and do not reduce to those citations. The paper itself flags the main limitation: after Lemma 1 it states "The PL condition of P(w) could be proved for learning a neural network similar to existing studies, which is not the main focus of this paper," and Appendix A.7 only proves PL for a one-hidden-layer Leaky ReLU example. That is an unverified assumption and a correctness risk for the deep AUC objective, not a circularity, because the theorems are explicitly conditional on Assumption 1. For the same reason, the paper's claim that PL "has been proved and observed in deep learning" is a literature-support statement for an assumption, not a derivation of the paper's own rates from that statement. No equation in the paper is shown to equal its input by construction, and no benchmark prediction is generated from data fitted to that same prediction. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 8 assumptions · 0 invented entities

The theoretical claims depend on Assumption 1 plus gradient bounds needed for the AdaGrad analysis. No constants are fitted to data in the derivations; however, the experimental schedules require tuning eta0, T0, and gamma on validation data, and the theoretical parameters mu, L, G, sigma, and delta are unknown in practice. The PL assumption is the most fragile entry because it is not proven for the deep AUC objective.

free parameters (3)
  • eta0 (initial step size) = tuned in [0.1, 0.05, 0.01, 0.008, 0.005] on validation
    Experimental hyperparameter; the theory treats eta0 as a free constant in the O~ notation, and experiments tune it per dataset and algorithm.
  • T0 (initial inner-loop length) = tuned in [200, 2000] on validation
    Controls stage lengths in experiments; the theory would set T_k from L, mu, and eta0, but the experiments use the heuristic T_s = T0 * 3^s.
  • gamma (proximal parameter) = tuned for PGA and reused for PPD-SG and PPD-AdaGrad
    In theory gamma = 1/(2L), but in experiments it is tuned rather than set from the unknown smoothness constant.
assumptions (8)
  • domain assumption Assumption 1(1): phi(v) satisfies the PL inequality mu(phi(v)-phi(v*)) <= (1/2)||grad phi(v)||^2 for mu > 0.
    Load-bearing. Not proven for the deep AUC surrogate; cited PL results apply to other losses and architectures. Appendix A.7 only covers one-hidden-layer Leaky ReLU with zero-mean uncorrelated features.
  • domain assumption Assumption 1(2): h(w;x) is \tilde L-Lipschitz in w for all x.
    Used in Lemma 1 and Lemma 2 to bound dual drift between epochs. Plausible for smooth networks but unproven for arbitrary deep networks.
  • domain assumption Assumption 1(3): phi(v) is L-smooth.
    Standard smoothness required for the proximal point analysis and the Nesterov inequality used in Theorems 2 and 3.
  • domain assumption Assumption 1(4): Var[h(w;x)|y=+1] <= sigma^2 and Var[h(w;x)|y=-1] <= sigma^2.
    Used to bound the mini-batch estimation error of the dual variable alpha in Lemmas 2 and 3.
  • domain assumption Assumption 1(5): 0 <= h(w;x) <= 1.
    Holds when h is a sigmoid-composed network output; required for the variance bound and for the closed-form alpha update.
  • domain assumption Assumption 1(6): phi(bar v0) - phi(v*) <= Delta0 for a given initial solution.
    Used to set the number of outer stages K in Theorems 2 and 3.
  • domain assumption Bounded stochastic gradients: ||hat g_t^k||^2 <= G in Lemma 2 and ||hat g_t^k||_infty <= delta in Lemma 3.
    Required for the SGD and AdaGrad analyses; not derived from the previous assumptions.
  • ad hoc to paper Cumulative gradient growth: ||hat g_{1:T_k,i}||_2 <= delta T_k^alpha for 0 <= alpha <= 1/2 in Theorem 3.
    This assumption defines the regime in which PPD-AdaGrad is claimed adaptive. It is not standard and may fail for noisy stochastic gradients, where cumulative gradients typically grow like sqrt(T), i.e., alpha = 1/2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stochastic AUC Maximization with Deep Neural Networks." pith.science (2026). https://pith.science/paper/QLE64HXU

@misc{pith2026190810831,
  author       = {Pith},
  title        = {Pith review of: Stochastic AUC Maximization with Deep Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QLE64HXU}},
  note         = {Machine review of arXiv:1908.10831}
}
read the original abstract

Stochastic AUC maximization has garnered an increasing interest due to better fit to imbalanced data classification. However, existing works are limited to stochastic AUC maximization with a linear predictive model, which restricts its predictive power when dealing with extremely complex data. In this paper, we consider stochastic AUC maximization problem with a deep neural network as the predictive model. Building on the saddle point reformulation of a surrogated loss of AUC, the problem can be cast into a {\it non-convex concave} min-max problem. The main contribution made in this paper is to make stochastic AUC maximization more practical for deep neural networks and big data with theoretical insights as well. In particular, we propose to explore Polyak-\L{}ojasiewicz (PL) condition that has been proved and observed in deep learning, which enables us to develop new stochastic algorithms with even faster convergence rate and more practical step size scheme. An AdaGrad-style algorithm is also analyzed under the PL condition with adaptive convergence rate. Our experimental results demonstrate the effectiveness of the proposed algorithms.

Figures

Figures reproduced from arXiv: 1908.10831 by the authors.

Figure 1
Figure 1. Comparison of testing AUC on Cat&Dog, CIFAR10, CIFAR100 and STL10. [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Comparison of testing AUC on Cat&Dog, CIFAR10, CIFAR100 and STL10. [PITH_FULL_IMAGE:figures/full_fig_p024_2.png] view at source ↗
Figure 3
Figure 3. Comparison of testing AUC on Cat&Dog, CIFAR10, CIFAR100 and STL10. For CIFAR10 [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Enhancing Topic Interpretability for Neural Topic Modeling through Topic-wise Contrastive Learning

    cs.AI 2024-12 conditional novelty 6.0 of 10

    A topic-wise contrastive regularizer using precomputed NPMI similarities improves coherence and diversity of neural topic model topics on 20NG, Yahoo, and NYTimes.

Reference graph

Works this paper leans on

54 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    A convergence theory for deep learning via over-parameterization

    Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over-parameterization. arXiv preprint arXiv:1811.03962, 2018

    arXiv 2018

  2. [2]

    A convergence analysis of gradient descent for deep linear neural networks

    Sanjeev Arora, Nadav Cohen, Noah Golowich, and Wei Hu. A convergence analysis of gradient descent for deep linear neural networks. arXiv preprint arXiv:1810.02281, 2018

    arXiv 2018

  3. [3]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014

    arXiv 2014

  4. [4]

    Stability and Generalization of Learning Algorithms that Converge to Global Optima

    Zachary Charles and Dimitris Papailiopoulos. Stability and generalization of learning algorithms that converge to global optima. arXiv preprint arXiv:1710.08402, 2017

    work page Pith review arXiv 2017

  5. [5]

    Universal stagewise learning for non-convex problems with convergence on averaged solutions

    Zaiyi Chen, Zhuoning Yuan, Jinfeng Yi, Bowen Zhou, Enhong Chen, and Tianbao Yang. Universal stagewise learning for non-convex problems with convergence on averaged solutions. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Syx5V2CcFm

    work page 2019

  6. [6]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

  7. [7]

    Gradient descent finds global minima of deep neural networks

    Simon S Du, Jason D Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. Gradient descent finds global minima of deep neural networks. arXiv preprint arXiv:1811.03804, 2018 a

    arXiv 2018

  8. [8]

    Gradient descent provably optimizes over-parameterized neural networks

    Simon S Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh. Gradient descent provably optimizes over-parameterized neural networks. arXiv preprint arXiv:1810.02054, 2018 b

    arXiv 2018

Show all 54 references
  1. [9]

    Adaptive subgradient methods for online learning and stochastic optimization

    John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12 0 (Jul): 0 2121--2159, 2011

    2011

  2. [10]

    Composite objective mirror descent

    John C Duchi, Shai Shalev-Shwartz, Yoram Singer, and Ambuj Tewari. Composite objective mirror descent. In COLT, pp.\ 14--26, 2010

    2010

  3. [11]

    The foundations of cost-sensitive learning

    Charles Elkan. The foundations of cost-sensitive learning. In International joint conference on artificial intelligence, volume 17, pp.\ 973--978. Lawrence Erlbaum Associates Ltd, 2001

    2001

  4. [12]

    One-pass auc optimization

    Wei Gao, Rong Jin, Shenghuo Zhu, and Zhi-Hua Zhou. One-pass auc optimization. In ICML (3), pp.\ 906--914, 2013

    2013

  5. [13]

    Generating sequences with recurrent neural networks

    Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013

    2013 arXiv

  6. [14]

    A simple generalisation of the area under the roc curve for multiple class classification problems

    David J Hand and Robert J Till. A simple generalisation of the area under the roc curve for multiple class classification problems. Machine learning, 45 0 (2): 0 171--186, 2001

    2001

  7. [15]

    The meaning and use of the area under a receiver operating characteristic (roc) curve

    James A Hanley and Barbara J McNeil. The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology, 143 0 (1): 0 29--36, 1982

    1982

  8. [16]

    A method of comparing the areas under receiver operating characteristic curves derived from the same cases

    James A Hanley and Barbara J McNeil. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology, 148 0 (3): 0 839--843, 1983

    1983

  9. [17]

    Identity matters in deep learning

    Moritz Hardt and Tengyu Ma. Identity matters in deep learning. arXiv preprint arXiv:1611.04231, 2016

    2016 arXiv

  10. [18]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  11. [19]

    Deep neural networks for acoustic modeling in speech recognition

    Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Brian Kingsbury, et al. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal processing magazine, 29, 2012

    2012

  12. [20]

    Minmax optimization: Stable limit points of gradient descent ascent are locally optimal

    Chi Jin, Praneeth Netrapalli, and Michael I Jordan. Minmax optimization: Stable limit points of gradient descent ascent are locally optimal. arXiv preprint arXiv:1902.00618, 2019

    1902 arXiv

  13. [21]

    Linear convergence of gradient and proximal-gradient methods under the polyak- ojasiewicz condition

    Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximal-gradient methods under the polyak- ojasiewicz condition. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.\ 795--811. Springer, 2016

    2016

  14. [22]

    An alternative view: When does sgd escape local minima? arXiv preprint arXiv:1802.06175, 2018

    Robert Kleinberg, Yuanzhi Li, and Yang Yuan. An alternative view: When does sgd escape local minima? arXiv preprint arXiv:1802.06175, 2018

    2018 arXiv

  15. [23]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp.\ 1097--1105, 2012

    2012

  16. [24]

    Non-convex finite-sum optimization via scsg methods

    Lihua Lei, Cheng Ju, Jianbo Chen, and Michael I Jordan. Non-convex finite-sum optimization via scsg methods. In Advances in Neural Information Processing Systems, pp.\ 2348--2358, 2017

    2017

  17. [25]

    Learning overparameterized neural networks via stochastic gradient descent on structured data

    Yuanzhi Li and Yingyu Liang. Learning overparameterized neural networks via stochastic gradient descent on structured data. In Advances in Neural Information Processing Systems, pp.\ 8157--8166, 2018

    2018

  18. [26]

    Convergence analysis of two-layer neural networks with relu activation

    Yuanzhi Li and Yang Yuan. Convergence analysis of two-layer neural networks with relu activation. In Advances in Neural Information Processing Systems, pp.\ 597--607, 2017

    2017

  19. [27]

    A simple proximal stochastic gradient method for nonsmooth nonconvex optimization

    Zhize Li and Jian Li. A simple proximal stochastic gradient method for nonsmooth nonconvex optimization. In Advances in Neural Information Processing Systems, pp.\ 5564--5574, 2018

    2018

  20. [28]

    Solving weakly-convex-weakly-concave saddle-point problems as weakly-monotone variational inequality

    Qihang Lin, Mingrui Liu, Hassan Rafique, and Tianbao Yang. Solving weakly-convex-weakly-concave saddle-point problems as weakly-monotone variational inequality. arXiv preprint arXiv:1810.10207, 2018

    2018 arXiv

  21. [29]

    Fast stochastic auc maximization with o (1/n)-convergence rate

    Mingrui Liu, Xiaoxuan Zhang, Zaiyi Chen, Xiaoyu Wang, and Tianbao Yang. Fast stochastic auc maximization with o (1/n)-convergence rate. In International Conference on Machine Learning, pp.\ 3195--3203, 2018

    2018

  22. [30]

    Hybrid block successive approximation for one-sided non-convex min-max problems: Algorithms and applications

    Songtao Lu, Ioannis Tsaknakis, Mingyi Hong, and Yongxin Chen. Hybrid block successive approximation for one-sided non-convex min-max problems: Algorithms and applications. arXiv preprint arXiv:1902.08294, 2019

    1902 arXiv

  23. [31]

    Acoustic modeling using deep belief networks

    Abdel-rahman Mohamed, George E Dahl, and Geoffrey Hinton. Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing, 20 0 (1): 0 14--22, 2012

    2012

  24. [32]

    Stochastic proximal algorithms for auc maximization

    Michael Natole, Yiming Ying, and Siwei Lyu. Stochastic proximal algorithms for auc maximization. In International Conference on Machine Learning, pp.\ 3707--3716, 2018

    2018

  25. [33]

    Robust stochastic approximation approach to stochastic programming

    Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization, 19 0 (4): 0 1574--1609, 2009

    2009

  26. [34]

    Introductory lectures on convex optimization: A basic course, volume 87

    Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013

    2013

  27. [35]

    Stochastic recursive gradient algorithm for nonconvex optimization

    Lam M Nguyen, Jie Liu, Katya Scheinberg, and Martin Takac. Stochastic recursive gradient algorithm for nonconvex optimization. arXiv preprint arXiv:1705.07261, 2017

    2017 arXiv

  28. [36]

    Gradient methods for minimizing functionals

    Boris Teodorovich Polyak. Gradient methods for minimizing functionals. Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki, 3 0 (4): 0 643--653, 1963

    1963

  29. [37]

    Non-convex min-max optimization: Provable algorithms and applications in machine learning

    Hassan Rafique, Mingrui Liu, Qihang Lin, and Tianbao Yang. Non-convex min-max optimization: Provable algorithms and applications in machine learning. arXiv preprint arXiv:1810.02060, 2018

    2018 arXiv

  30. [38]

    Stochastic variance reduction for nonconvex optimization

    Sashank J Reddi, Ahmed Hefny, Suvrit Sra, Barnabas Poczos, and Alex Smola. Stochastic variance reduction for nonconvex optimization. In International conference on machine learning, pp.\ 314--323, 2016

    2016

  31. [39]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pp.\ 91--99, 2015

    2015

  32. [40]

    Monotone operators and the proximal point algorithm

    R Tyrrell Rockafellar. Monotone operators and the proximal point algorithm. SIAM journal on control and optimization, 14 0 (5): 0 877--898, 1976

    1976

  33. [41]

    Solving non-convex non-concave min-max games under polyak-l ojasiewicz condition

    Maziar Sanjabi, Meisam Razaviyayn, and Jason D Lee. Solving non-convex non-concave min-max games under polyak-l ojasiewicz condition. arXiv preprint arXiv:1812.02878, 2018

    2018 arXiv

  34. [42]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  35. [43]

    Sequence to sequence learning with neural networks

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp.\ 3104--3112, 2014

    2014

  36. [44]

    Spiderboost: A class of faster variance-reduced algorithms for nonconvex optimization

    Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, and Vahid Tarokh. Spiderboost: A class of faster variance-reduced algorithms for nonconvex optimization. arXiv preprint arXiv:1810.10690, 2018

    2018 arXiv

  37. [45]

    Stochastic online auc maximization

    Yiming Ying, Longyin Wen, and Siwei Lyu. Stochastic online auc maximization. In Advances in Neural Information Processing Systems, pp.\ 451--459, 2016

    2016

  38. [46]

    Online auc maximization

    Peilin Zhao, Rong Jin, Tianbao Yang, and Steven C Hoi. Online auc maximization. In Proceedings of the 28th international conference on machine learning (ICML-11), pp.\ 233--240, 2011

    2011

  39. [47]

    Stochastic nested variance reduced gradient descent for nonconvex optimization

    Dongruo Zhou, Pan Xu, and Quanquan Gu. Stochastic nested variance reduced gradient descent for nonconvex optimization. In Advances in Neural Information Processing Systems, pp.\ 3921--3932, 2018

    2018

  40. [48]

    Characterization of gradient dominance and regularity conditions for neural networks

    Yi Zhou and Yingbin Liang. Characterization of gradient dominance and regularity conditions for neural networks. arXiv preprint arXiv:1710.06910, 2017

    2017 arXiv

  41. [49]

    An improved analysis of training over-parameterized deep neural networks

    Difan Zou and Quanquan Gu. An improved analysis of training over-parameterized deep neural networks. arXiv preprint arXiv:1906.04688, 2019

    1906 arXiv

  42. [50]

    Stochastic gradient descent optimizes over-parameterized deep relu networks

    Difan Zou, Yuan Cao, Dongruo Zhou, and Quanquan Gu. Stochastic gradient descent optimizes over-parameterized deep relu networks. arXiv preprint arXiv:1811.08888, 2018

    2018 arXiv

  43. [51]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  44. [52]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  45. [53]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  46. [54]

    1h A XHT J e ,..b ] K Lxb-

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.