Pith. sign in

REVIEW 2 major objections 5 minor 46 references

PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PSMGD claims that periodically reusing the dynamic weights of a multi-gradient descent algorithm—rather than solving for them at every step—preserves the best known convergence rates and, when the period is at least the number of…

desk verdict Useful periodic-weight-reuse idea and solid nonconvex/convex analyses, but the strongly convex proof has a load-bearing absorption error, so the headline O(1/T) rate and BP-complexity claim are not established as written. read the letter →

arxiv 2412.10961 v2 pith:ZY2T6ZVI submitted 2024-12-14 cs.LG cs.AI

Mingjing Xu , Peizhong Ju , Jia Liu , Haibo Yang This is my paper · ORCID
classification cs.LGcs.AI MSC 90C2990C2568T07
keywords multi-objectiveoptimizationgradientmanipulationmethodsperiodicweightupdatestochasticmulti-gradientdescentconvergenceratebackpropagationcomplexitymulti-tasklearningParetostationarity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the most expensive step in gradient-manipulation methods for multi-objective optimization—recomputing dynamic weights by solving a small constrained problem at every iteration—is largely unnecessary. Its proposed algorithm, PSMGD, solves for the weights once every $R$ iterations and reuses them in between, with a momentum update $\lambda_t = \alpha_t \lambda_{t-R} + (1-\alpha_t)\hat{\lambda}^*_t$ to keep them stable. The authors prove that this periodic schedule preserves the best known convergence rates: $O(1/T)$ for strongly convex objectives and $O(1/\sqrt{T})$ for general convex and non-convex objectives. They also introduce backpropagation complexity, the total number of backward passes to reach accuracy $\epsilon$, and show that when $R = \Omega(S)$ PSMGD needs only $O(1/\epsilon)$ passes for strongly convex problems and $O(1/\epsilon^2)$ for convex and non-convex ones—the same order as single-objective SGD. If correct, the result makes multi-objective training almost as cheap per step as single-objective training while retaining the conflict-avoiding behavior of gradient manipulation.

What carries the argument

The mechanism is periodic weight recalculation with momentum. Every $R$ iterations the algorithm solves the constrained quadratic program $\min_\lambda \|\sum_{s\in[S]} \lambda_s \nabla f_s(x_t,\xi_t)\|^2$ subject to $\sum_s \lambda_s = 1$, obtaining a fresh weight vector $\hat{\lambda}^*_t$, and then mixes it with the previous weights via $\lambda_t = \alpha_t \lambda_{t-R} + (1-\alpha_t)\hat{\lambda}^*_t$. For the intervening $R-1$ steps the weights are frozen, so the model update requires only one scalarized backpropagation. The proof engine is a variance bound $V_\xi[\lambda_t] \le S^2 B^2 (1-\alpha_t)^2$ on the stochastic weight vector, which controls the extra error created by stale weights and lets the recalculated-weight error be absorbed into $\eta_t^2$ terms in the convergence recurrences. That absorption is what turns the per-iteration quadratic-program solve into an amortized cost and yields the objective-independent backpropagation complexity.

What would settle it

Substitute the stated momentum schedule $1-\alpha_t=\eta_t$ into the strongly convex recurrence and check whether the stale-weight term $4F(1-\alpha_t)SB$ is bounded by a constant times $\eta_t^2$ for the full trajectory; if it is not, the displayed $\Phi$ and the $O(1/T)$ bound in Theorem 3.7 do not follow. Re-running the calculation with $1-\alpha_t=\eta_t^2$ would show whether a modified schedule restores the rate, and a synthetic strongly convex two-objective problem with a known Pareto point could test the claimed $O(1/\epsilon)$ backpropagation complexity empirically.

Watch

Extended reading notes

Core claim

The central claim is that the dynamic weight vector defining a common descent direction does not need to be recomputed every iteration. Theorem 3.5 proves $O(1/\sqrt{T})$ convergence in the non-convex case, Theorem 3.6 proves $O(1/\sqrt{T})$ in the general convex case, and Theorem 3.7 proves $O(1/T)$ in the $\mu$-strongly convex case, each matching the rates of the best existing stochastic multi-objective methods without assuming Lipschitz continuity of the optimal weight function. The new backpropagation complexity measure counts total backward passes; with the recalculation period set to $R = \Omega(S)$, the bounds become $O(1/\epsilon)$ and $O(1/\epsilon^2)$, independent of the number of objectives $S$ and identical to single-objective SGD. The experiments on multi-task learning datasets are offered as evidence that the same performance as full gradient manipulation is reached in less wall-clock time.

Load-bearing premise

The load-bearing premise is that the error from reusing stale weights shrinks as the square of the step size, yet the momentum schedule chosen in the strongly convex proof only guarantees it shrinks as the step size itself.

Editorial extensions

If this is right

  • If the central claim holds, setting the recalculation period $R=\Omega(S)$ makes PSMGD's total backward passes to accuracy $\epsilon$ of the same order as single-objective SGD: $O(1/\epsilon)$ for strongly convex objectives and $O(1/\epsilon^2)$ for convex and non-convex objectives.
  • The convergence-rate results extend to settings without Lipschitz continuity of the optimal weight function, a condition required by earlier stochastic multi-gradient algorithms.
  • In multi-task learning, the wall-clock cost of the weight-solving step is amortized over $R$ iterations, so the speedup over full gradient manipulation should grow roughly linearly with $R$ until the weight-stability assumption degrades.
  • Backpropagation complexity gives practitioners a common metric—total backward passes to a target loss—for comparing multi-objective optimizers on equal footing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same periodic-reuse wrapper could be applied to other gradient-manipulation methods whose weight vectors drift slowly (for example PCGrad, CAGrad, or FAMO); benchmarking each method with and without an $R$-step reuse schedule would test whether the speedup transfers.
  • Because backpropagation complexity counts only backward passes, it may understate the practical cost of computing all $S$ objective gradients before each weight solve; in architectures where gradient memory traffic dominates, the real speedup could be smaller than $O(S/R)$.
  • The weight-stability observation suggests an adaptive rule for choosing $R$: monitor the empirical variation of $\hat{\lambda}^*_t$ and set $R$ to the largest window with acceptable drift, with the paper's bounds indicating how convergence degrades as the drift term $|\hat{\lambda}^*_t-\lambda_{t-R}|$ grows.
  • If the strongly convex proof is repaired by changing the momentum schedule, the same periodic-reuse idea could be combined with adaptive optimizers to reduce multi-objective training overhead further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes PSMGD, a periodic stochastic multi-gradient descent method for multi-objective optimization. Instead of recomputing the MGDA-type dynamic weights at every iteration, PSMGD recomputes them every R iterations and reuses them in between, with a momentum-style update λ_t = α_t λ_{t-R} + (1−α_t) λ̂*_t at recalculation steps. The authors claim O(1/T) convergence for strongly convex objectives and O(1/√T) for general convex and non-convex objectives, matching the best existing MOO rates, and introduce a 'backpropagation complexity' measure under which PSMGD achieves objective-independent complexity when R = Ω(S). The paper also reports extensive multi-task learning experiments on QM-9, NYU-v2, Multi-MNIST, CityScapes, and CelebA, with a public code repository.

Significance. If the theoretical claims were fully established, the periodic weight-reuse idea would be a useful and practical contribution: it directly addresses the dominant per-iteration cost of gradient-manipulation MOO methods, and the backpropagation-complexity measure is a sensible metric for comparing such methods. The experimental study is broad, includes several standard MTL benchmarks and many baselines, and the public code is a strength. However, the central strongly-convex result currently rests on an invalid algebraic step in the appendix, and the proof's parameter regime is not stated consistently. The general-convex and non-convex proofs appear structurally sound, and the flaw in the strongly-convex proof is localized, so the contribution is plausibly repairable; in its present form, the headline theoretical guarantee and the associated BP-complexity claims are not established.

major comments (2)
  1. [Appendix A.5, Eqs. (118)–(122)] The strongly-convex proof contains an invalid absorption step. After deriving the recursion with the periodic weight-recalculation term 4F(1−α_t)SB·1{(t+1)%R=0} in equation (118), the proof replaces it inside the η_t²Φ term with 4FSB·1{(t+1)%R=0}, annotated 'where we set (1−α_t)=η_t.' Since 1−α_t=η_t, the original term is O(η_t), not O(η_t²). With η_t=c/T and η_t<1 for large T, summing this term over the roughly T/R recalculation steps contributes an additive Ω(1) constant, so the contraction argument in equation (125) does not yield the claimed O(1/T) bound in equation (8). Because Theorem 3.7 is the basis for the strongly-convex row in Table 1 and for the strongly-convex BP complexity in Remarks 3.9 and 3.10, those claims are not proven as written. A repair would require, at minimum, changing the schedule so that 1−α_t is O(η_t²) (e.g., 1−α_t=η_t²) and then re-checking all affected bounds.
  2. [Theorem 3.7 and Appendix A.5] The statement of Theorem 3.7 is internally incomplete and inconsistent. The theorem states 'Set η=c/T with c>1/μ', but the proof's final display uses the condition c>1/(2μ), and the proof introduces the condition 'where we set (1−α_t)=η_t' only at equation (120), even though the theorem statement does not specify a schedule for α_t. The contraction coefficient and the conditions on c must be stated consistently and derived from the descent inequality actually used; otherwise the theorem cannot be verified as written. This is a separate load-bearing issue from the absorption error in the previous comment, because it affects the reproducible statement of the claimed O(1/T) rate.
minor comments (5)
  1. [Theorem 3.5 and Appendix A.3] The theorem states that each objective is bounded by F in the form f_s(x)≤F, but the proof of equation (59) uses |f_s(x)|≤F. Please state the needed two-sided boundedness assumption explicitly.
  2. [Lemma A.1] The lemma's bounds contain the undefined symbol m inside expressions such as √(mσ² V_ξ[λ_t]); from the context this should be S (or the bound should be written with explicit constants).
  3. [Appendix B.3] The experiments fix a 'momentum λ=0.9' and select R∈{4,8,16}, whereas the convergence proofs analyze time-varying α_t with 1−α_t=η_t or the min-schedule in Theorem 3.5. The paper should explain how the experimental hyperparameters relate to the analyzed parameter regime, or state which theoretical setting the experiments are intended to validate.
  4. [Definition 3 and Remark 3.9] Backpropagation complexity is defined informally as the number of backpropagation operations needed to 'achieve a specified performance threshold, denoted ε'. Since the convergence theorems are stated in expectation, the BP-complexity claims should be tied to a precise formal statement (e.g., expectation of the relevant metric being below ε) to be checkable.
  5. [Reproducibility Checklist] The checklist says that all theoretical claims are demonstrated empirically to hold, but the experiments report task metrics and wall-clock time rather than the convergence metrics used in Theorems 3.5–3.7. Please temper this checklist entry or add measurements of the theoretical convergence metrics.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation chain; the sole self-citation (Yang et al. 2024) is a non-load-bearing metrics comparison, so the paper sits at the low end (2) of the no-significant-circularity band.

full rationale

I walked the derivation chain of PSMGD. The algorithm is motivated by an empirical weight-stability observation, but the convergence proofs do not assume that observation: the recalculation error is controlled by the explicit schedule for 1−α_t (Theorem 3.5: 1−α_t = min{η_t/η_1, η_t/(η_1√t max_s |λhat_t^s−λ_{t−R}^s|)}; Theorem 3.6: 1−α_t = η_t), and the rates are derived from L-smoothness, bounded variance, bounded weights/gradients, and the external Lemmas A.1–A.2 from Zhou et al. (2022). No parameter is fit to a target rate and then renamed as a prediction. The BP-complexity claims in Remarks 3.9–3.10 are arithmetic consequences of Definition 3 plus the iteration rates (S backprops at the T/R recalculation steps, one backprop elsewhere), so they are not circular. The only self-citation (Yang et al. 2024) appears in Appendix A.1 as an example of an alternative convergence metric; it carries no load in any proof. The skeptical reader's Appendix A.5 concern — absorbing the 4F(1−α_t)SB term into η_t²Φ by setting (1−α_t)=η_t — is an algebraic/validity problem in the strongly convex proof, not a circular reduction: the theorem's O(1/T) claim does not become equal to its assumptions by construction. Hence no circularity steps are identified; the score reflects only the minor non-load-bearing self-citation.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claims rest on standard smoothness, variance, bounded-gradient, and strong-convexity assumptions common in the MOO literature, plus hand-chosen hyperparameters R and momentum. No ad hoc entities are introduced.

free parameters (2)
  • R (period) = R=4 (Multi-MNIST), R=8 (QM-9, NYU-v2, CityScapes, CelebA)
    The period between weight recalculations is chosen by validation loss in experiments and is the key hyperparameter behind the claimed speedup.
  • Momentum coefficient for weight update = λ=0.9 in experiments; α_t schedule in theory (1−α_t set to η_t in Theorem 3.7)
    The momentum coefficient controls how much of the old weight vector is kept. In experiments a fixed value is chosen by hand; in theory it is scheduled to make error terms small.
assumptions (5)
  • standard math Each objective f_s is L-smooth (Assumption 3.1)
    Used throughout the convergence proofs to apply the descent lemma.
  • domain assumption Stochastic gradients are unbiased with bounded variance (Assumption 3.2)
    Standard in stochastic optimization; needed to bound gradient noise terms.
  • domain assumption Bounded weights with 0≤λ_t,s≤B and sum_s λ_t,s≥1 (Assumption 3.3)
    Ensures the weighted gradient direction is a valid convex combination; standard in MOO analysis.
  • domain assumption Bounded gradients ||∇f_s||≤H (Assumption 3.4)
    Used to bound the bias from the dependence of λ_t on the current stochastic gradients.
  • domain assumption For the strongly convex result, each f_s is μ-strongly convex and bounded above by F (Theorem 3.7)
    Needed for the O(1/T) claim; strong convexity gives the quadratic growth inequality (91).

how reviews work

0 comments
Cite this review

Pith. "Pith review of PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization." pith.science (2026). https://pith.science/paper/ZY2T6ZVI

@misc{pith2026241210961,
  author       = {Pith},
  title        = {Pith review of: PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZY2T6ZVI}},
  note         = {Machine review of arXiv:2412.10961}
}
read the original abstract

Multi-objective optimization (MOO) lies at the core of many machine learning (ML) applications that involve multiple, potentially conflicting objectives (e.g., multi-task learning, multi-objective reinforcement learning, among many others). Despite the long history of MOO, recent years have witnessed a surge in interest within the ML community in the development of gradient manipulation algorithms for MOO, thanks to the availability of gradient information in many ML problems. However, existing gradient manipulation methods for MOO often suffer from long training times, primarily due to the need for computing dynamic weights by solving an additional optimization problem to determine a common descent direction that can decrease all objectives simultaneously. To address this challenge, we propose a new and efficient algorithm called Periodic Stochastic Multi-Gradient Descent (PSMGD) to accelerate MOO. PSMGD is motivated by the key observation that dynamic weights across objectives exhibit small changes under minor updates over short intervals during the optimization process. Consequently, our PSMGD algorithm is designed to periodically compute these dynamic weights and utilizes them repeatedly, thereby effectively reducing the computational overload. Theoretically, we prove that PSMGD can achieve state-of-the-art convergence rates for strongly-convex, general convex, and non-convex functions. Additionally, we introduce a new computational complexity measure, termed backpropagation complexity, and demonstrate that PSMGD could achieve an objective-independent backpropagation complexity. Through extensive experiments, we verify that PSMGD can provide comparable or superior performance to state-of-the-art MOO algorithms while significantly reducing training time.

Figures

Figures reproduced from arXiv: 2412.10961 by the authors.

Figure 1
Figure 1. The convergence behaviors on a pedagogical example with 10 runs for each algorithm: (a) Solutions obtained from [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Test loss in training (300 epochs) on QM-9. [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Test loss of segmentation, depth, and surface normal in training (200 epochs) on NYU-v2. [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: PSMGD achieves a favorable balance between test accuracy and training time of epoch and iteration. (a) (Left) Average test accuracy: mean and 95% CI (10 runs). (b) (Middle) Box plots for the average training time of an epoch (10 runs). (c) (Right) Box plots for the ave…
Figure 5
Figure 5. Figure 5: PSMGD achieves a favorable balance between each task’s accuracy and convergence complexity time of epoch in MultiMNIST. (a) (Left) The results for MultiMNIST with Task1&2 accuracy. (b) (Middle) Average backpropagation complexity for convergence: mean and 95% CI (10 run…
Figure 6
Figure 6. Figure 6: The dynamic weight vectors for MGDA and lin [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Test loss in training (300 epochs) on QM-9. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 20 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Badrinarayanan, V.; Kendall, A.; and Cipolla, R. 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12): 2481--2495

    2017

  4. [4]

    Ban, H.; and Ji, K. 2024. Fair Resource Allocation in Multi-Task Learning. arXiv preprint arXiv:2402.15638

    arXiv 2024

  5. [5]

    K.; and Doppa, J

    Belakaria, S.; Deshwal, A.; Jayakodi, N. K.; and Doppa, J. R. 2020. Uncertainty-aware search framework for multi-objective Bayesian optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 10044--10052

    2020

  6. [6]

    C.; and Reymond, J.-L

    Blum, L. C.; and Reymond, J.-L. 2009. 970 million druglike small molecules for virtual screening in the chemical universe database GDB-13. Journal of the American Chemical Society, 131(25): 8732--8733

    2009

  7. [7]

    Caruana, R. 1997. Multitask learning. Machine learning, 28: 41--75

    1997

  8. [8]

    Chen, L.; Fernando, H.; Ying, Y.; and Chen, T. 2024. Three-way trade-off in multi-objective learning: Optimization, generalization and conflict-avoidance. Advances in Neural Information Processing Systems, 36

    work page 2024

Show all 46 references
  1. [9]

    Chen, Z.; Badrinarayanan, V.; Lee, C.-Y.; and Rabinovich, A. 2018. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, 794--803. PMLR

    2018

  2. [10]

    Chen, Z.; Ngiam, J.; Huang, Y.; Luong, T.; Kretzschmar, H.; Chai, Y.; and Anguelov, D. 2020. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. Advances in Neural Information Processing Systems, 33: 2039--2050

    2020

  3. [11]

    Cordts, M.; Omran, M.; Ramos, S.; Rehfeld, T.; Enzweiler, M.; Benenson, R.; Franke, U.; Roth, S.; and Schiele, B. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3213--3223

    2016

  4. [12]

    Deb, K.; Pratap, A.; Agarwal, S.; and Meyarivan, T. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation, 6(2): 182--197

    2002

  5. [13]

    D \'e sid \'e ri, J.-A. 2012. Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. Comptes Rendus Mathematique, 350(5-6): 313--318

    2012

  6. [14]

    D.; Shen, H.; Liu, M.; Chaudhury, S.; Murugesan, K.; and Chen, T

    Fernando, H. D.; Shen, H.; Liu, M.; Chaudhury, S.; Murugesan, K.; and Chen, T. 2022. Mitigating gradient bias in multi-objective learning: A provably convergent approach. In The Eleventh International Conference on Learning Representations

    2022

  7. [15]

    Fey, M.; and Lenssen, J. E. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428

    2019 arXiv

  8. [16]

    Fliege, J.; and Svaiter, B. F. 2000. Steepest descent methods for multicriteria optimization. Mathematical methods of operations research, 51: 479--494

    2000

  9. [17]

    Fliege, J.; Vaz, A. I. F.; and Vicente, L. N. 2019. Complexity of gradient descent for multiobjective optimization. Optimization Methods and Software, 34(5): 949--959

    2019

  10. [18]

    M.; and Fleming, P

    Fonseca, C. M.; and Fleming, P. J. 1996. On the performance assessment and comparison of stochastic multiobjective optimizers. In International conference on parallel problem solving from nature, 584--593. Springer

    1996

  11. [19]

    a llstr \

    Hayes, C. F.; R a dulescu, R.; Bargiacchi, E.; K \"a llstr \"o m, J.; Macfarlane, M.; Reymond, M.; Verstraeten, T.; Zintgraf, L. M.; Dazeley, R.; Heintz, F.; et al. 2022. A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agen...

    2022

  12. [20]

    Javaloy, A.; and Valera, I. 2021. Rotograd: Gradient homogenization in multitask learning. arXiv preprint arXiv:2103.02631

    2021 arXiv

  13. [21]

    Kendall, A.; Gal, Y.; and Cipolla, R. 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7482--7491

    2018

  14. [22]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  15. [23]

    Kurin, V.; De Palma, A.; Kostrikov, I.; Whiteson, S.; and Mudigonda, P. K. 2022. In defense of the unitary scalarization for deep multi-task learning. Advances in Neural Information Processing Systems, 35: 12169--12183

    2022

  16. [24]

    Laumanns, M.; and Ocenasek, J. 2002. Bayesian optimization algorithms for multi-objective optimization. In International Conference on Parallel Problem Solving from Nature, 298--307. Springer

    2002

  17. [25]

    LeCun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278--2324

    1998

  18. [26]

    Lin, B.; Feiyang, Y.; and Zhang, Y. 2021. A closer look at loss weighting in multi-task learning

    2021

  19. [27]

    Liu, B.; Feng, Y.; Stone, P.; and Liu, Q. 2024. Famo: Fast adaptive multitask optimization. Advances in Neural Information Processing Systems, 36

    2024

  20. [28]

    Liu, B.; Liu, X.; Jin, X.; Stone, P.; and Liu, Q. 2021 a . Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems, 34: 18878--18890

    2021

  21. [29]

    Liu, L.; Li, Y.; Kuang, Z.; Xue, J.; Chen, Y.; Yang, W.; Liao, Q.; and Zhang, W. 2021 b . Towards impartial multi-task learning. iclr

    2021

  22. [30]

    Liu, S.; Johns, E.; and Davison, A. J. 2019. End-to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 1871--1880

    2019

  23. [31]

    Liu, S.; and Vicente, L. N. 2021. The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning. Annals of Operations Research, 1--30

    2021

  24. [32]

    Liu, Z.; Luo, P.; Wang, X.; and Tang, X. 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, 3730--3738

    2015

  25. [33]

    Mahapatra, D.; Dong, C.; Chen, Y.; and Momma, M. 2023. Multi-label learning to rank through multi-objective optimization. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 4605--4616

    2023

  26. [34]

    Maheshwari, G.; and Perrot, M. 2022. Fairgrad: Fairness aware gradient descent. arXiv preprint arXiv:2206.10923

    2022 arXiv

  27. [35]

    Miettinen, K. 1999. Nonlinear multiobjective optimization, volume 12. Springer Science & Business Media

    1999

  28. [36]

    Navon, A.; Shamsian, A.; Achituve, I.; Maron, H.; Kawaguchi, K.; Chechik, G.; and Fetaya, E. 2022. Multi-task learning as a bargaining game. arXiv preprint arXiv:2202.01017

    2022 arXiv

  29. [37]

    M.; Z ilinskas, A.; Z ilinskas, J.; et al

    Pardalos, P. M.; Z ilinskas, A.; Z ilinskas, J.; et al. 2017. Non-convex multi-objective optimization. Springer

    2017

  30. [38]

    Sener, O.; and Koltun, V. 2018. Multi-task learning as multi-objective optimization. Advances in neural information processing systems, 31

    2018

  31. [39]

    Silberman, N.; Hoiem, D.; Kohli, P.; and Fergus, R. 2012. Indoor segmentation and support inference from rgbd images. In Computer Vision--ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part V 12, 746--760. Springer

    2012

  32. [40]

    H.; and Yamashita, N

    Tanabe, H.; Fukuda, E. H.; and Yamashita, N. 2019. Proximal gradient methods for multiobjective optimization and their applications. Computational Optimization and Applications, 72: 339--361

    2019

  33. [41]

    Xiao, P.; Ban, H.; and Ji, K. 2024. Direction-oriented multi-objective learning: Simple and provable stochastic algorithms. Advances in Neural Information Processing Systems, 36

    2024

  34. [42]

    Xin, D.; Ghorbani, B.; Gilmer, J.; Garg, A.; and Firat, O. 2022. Do current multi-task optimization methods in deep learning even help? Advances in neural information processing systems, 35: 13597--13609

    2022

  35. [43]

    Yang, H.; Liu, Z.; Liu, J.; Dong, C.; and Momma, M. 2024. Federated multi-objective learning. Advances in Neural Information Processing Systems, 36

    2024

  36. [44]

    Yu, T.; Kumar, S.; Gupta, A.; Levine, S.; Hausman, K.; and Finn, C. 2020. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33: 5824--5836

    2020

  37. [45]

    Zhang, Q.; and Li, H. 2007. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on evolutionary computation, 11(6): 712--731

    2007

  38. [46]

    Zhou, S.; Zhang, W.; Jiang, J.; Zhong, W.; Gu, J.; and Zhu, W. 2022. On the convergence of stochastic multi-objective gradient manipulation and beyond. Advances in Neural Information Processing Systems, 35: 38103--38115

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.