Pith. sign in

REVIEW 3 major objections 5 minor 41 references

Gradient Similarity Surgery in Multi-Task Deep Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A scalar measure of gradient magnitude similarity can guide multi-task optimization, the paper claims, reaching global optima in synthetic tests and state-of-the-art results on three benchmarks.

desk verdict A plausible gradient surgery idea, but Algorithm 1 is an identity in the branch that should equalize gradients, so the method as printed is not defined and the results are not reproducible. read the letter →

arxiv 2506.06130 v1 pith:DQ5GTTDC submitted 2025-06-06 cs.LG cs.CV

Thomas Borsani , Andrea Rosani , Giuseppe Nicosia , Giuseppe Di Fatta This is my paper
classification cs.LGcs.CV
keywords multi-tasklearninggradientsurgeryconflictinggradientsmagnitudesimilaritymomentumregularisationoptimisationdeep
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Similarity-Aware Momentum Gradient Surgery (SAM-GS), a gradient surgery method for multi-task deep learning. It claims that the deciding factor in whether task gradients conflict is their magnitude similarity, not their direction, and that a single scalar similarity signal can therefore guide the whole optimization: equalize gradients when they disagree in magnitude, and let momentum accelerate when they agree. On a synthetic two-task problem with one global optimum, the paper reports that SAM-GS is the only tested method to reach the optimum from all seven starting points. On the NYU-V2, CelebA, and MetaWorld MT10 benchmarks it matches or improves on state-of-the-art methods, with CityScapes as the one benchmark where an angle-focused method does better. If the claim holds, multi-task training gains a cheap, scalable alternative to costly per-pair conflict resolution.

What carries the argument

The load-bearing object is the magnitude similarity $\psi(g_i,g_j) = \tfrac{2\|g_i\|_2\|g_j\|_2}{\|g_i\|_2^2+\|g_j\|_2^2}$, a scalar in $[0,1]$ that is large when two task gradients have comparable norms and small when one dominates. Averaged over all task pairs it gives $\Psi$, which controls two mechanisms in Algorithm 1: gradient equalisation, applied when $\Psi<\gamma$, and a momentum term $m_{k,t}$ whose influence is modulated by an exponential moving average of $(1-\Psi)^2$. This scalar signal is what lets SAM-GS avoid expensive per-pair projections or optimization loops while still adapting the learning dynamics to the state of the task gradients.

What would settle it

Run SAM-GS on a two-task problem where both gradients have identical magnitudes but exactly opposite directions, while keeping the same total step budget as an aligned case. If the method slows down or fails to converge relative to an angle-aware baseline such as Aligned-MTL, the paper's claim that angle conflicts only affect convergence speed is false; the paper's own CityScapes result already points in that direction.

Watch

Extended reading notes

Core claim

SAM-GS's central claim is that magnitude conflicts, not angle conflicts, are the primary obstacle in multi-task deep learning optimization, and that a gradient magnitude similarity measure is enough to drive adaptive surgery. The method computes the pairwise magnitude similarity $\psi(g_i,g_j) = \frac{2\|g_i\|_2\|g_j\|_2}{\|g_i\|_2^2+\|g_j\|_2^2}$, averages it over task pairs to get $\Psi$, and compares it to a threshold $\gamma$. If $\Psi<\gamma$, task gradients are equalized so no single task dominates the update; otherwise the update is scaled by a bias-corrected momentum divided by a regularizer that tracks $(1-\Psi)^2$, so momentum is trusted more when gradients are similar. The paper reports that this reaches the global optimum from all seven initial points on the first synthetic problem, and achieves the best or near-best $\Delta m\%$ on NYU-V2, CelebA, and MetaWorld MT10, while remaining competitive on CityScapes.

Load-bearing premise

The claim stands or falls on the assumption that gradient magnitude imbalances, rather than opposing gradient directions, are the main cause of poor multi-task convergence, so the method has no correction for angle conflicts.

Editorial extensions

If this is right

  • On the one-global-optimum synthetic problem, SAM-GS is the only method among those tested to reach the global optimum from all seven starting points within 20,000 steps.
  • On NYU-V2, SAM-GS reports the best mean rank (2.4) and best $\Delta m\%$ ($-5.3\%$) among all compared methods.
  • On CelebA's 40 tasks, SAM-GS reports $\Delta m\% = 3.33\%$, the best result in the comparison table.
  • On MetaWorld MT10, SAM-GS matches Nash-MTL's success rate (0.91) and beats FAMO, CAGrad, and PCGrad, with the smallest reported standard error.
  • The ablation study shows that neither pure gradient equalisation ($\gamma=0$) nor pure momentum ($\gamma=1$) is optimal, so the similarity threshold is what balances the two mechanisms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors do not develop is to feed the same magnitude-similarity signal into other aggregators, such as loss-balancing weights, turning SAM-GS into a cheap regulariser for methods that currently ignore gradient geometry.
  • The CityScapes result, where an angle-alignment method outperforms SAM-GS, suggests a boundary condition: on problems where inter-sample or angular conflicts dominate, magnitude-only surgery may need a hybrid switch that detects angle conflicts before falling back to alignment.
  • The switch from average $\Psi$ on supervised benchmarks to $\min\psi(g_i,g_j)$ in the RL experiment hints that the choice of pairwise aggregation is itself a hyperparameter; testing order statistics of the similarity distribution across task counts would be a direct follow-up.
  • Because the paper lists convergence theory as future work, a testable extension is to replace the fixed threshold $\gamma$ with an adaptive schedule that shrinks as training progresses, which might remove the need for per-benchmark tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces SAM-GS, a gradient surgery method for multi-task learning that uses a pairwise gradient-magnitude similarity measure Ψ to decide when to equalize task gradients and when to modulate a first-order momentum. The authors argue that magnitude conflicts, rather than angular conflicts, are the main obstacle in multi-task deep learning, and they support this with experiments on synthetic 2D problems, three vision benchmarks (CityScapes, NYU-v2, CelebA), and the MetaWorld MT10 reinforcement-learning benchmark, reporting state-of-the-art or comparable performance. The manuscript includes an ablation over the similarity threshold γ and an appendix with synthetic loss formulations, experimental settings, and standard errors for SAM-GS.

Significance. The problem is relevant and the proposal is simple enough to be useful if it works: a cheap magnitude-based conflict detector with adaptive momentum modulation could complement existing gradient-surgery methods. The synthetic analysis, the breadth of benchmarks, the ablation over γ, and the availability of code are strengths, and the paper's central assumption—that angular conflicts only affect convergence speed—is a clearly falsifiable claim. I do not see a circularity problem: γ is a hyperparameter tuned on validation data, and the similarity measure is fixed a priori. However, as printed, Algorithm 1 does not implement the described mechanism, so the method is currently undefined and the empirical claims cannot be reproduced from the manuscript alone. This is a load-bearing issue that must be corrected before the contribution can be assessed.

major comments (3)
  1. [§4, Algorithm 1] The low-similarity branch is an identity: if Ψ < γ, then w_k = (∥g_k∥_2 / ∥g_k∥_2) g_k = g_k, so the update reduces to the ordinary sum of raw task gradients. This branch therefore performs no gradient equalization and no conservative step-size reduction when magnitude conflicts are detected. The high-similarity else branch, w_k = |cm_{k,t}| / sqrt(ĥ_t + ε), is the only place where momentum scaling is applied, which is the reverse of the prose, where momentum should be trusted less when gradients are dissimilar. The same inconsistency appears in the γ ablation: a value of γ = 1 makes the algorithm always enter the identity branch, not 'rely exclusively on the equalisation component' as claimed. Please correct the pseudocode, state explicitly which branch performs equalization and which performs momentum modulation, and ensure that the reported trajectories and benchmark scores are generated by that corrected algorithm.
  2. [§4 and §5/B] The update rule θ_t ← θ_{t-1} − α Σ_k w_k ⊙ g_k is a plain steepest-descent step, not an AdaBelief-style per-parameter adaptive rule as claimed in Section 4. The quantity h_t is a scalar EMA of (1−Ψ)^2, so it cannot provide coordinate-wise adaptation, and the algorithm adds ε inside the h_t recurrence rather than only in the denominator. The experiments in Section 5 and Appendix B state that the Adam optimizer is used, but it is unclear whether SAM-GS is applied before Adam, replaces Adam, or modifies Adam's update. Please specify the exact integration with the base optimizer and define whether w_k is a scalar per task or a vector per parameter.
  3. [§4, 'Angle-based gradient conflicts are intentionally disregarded'] The premise that angular conflicts 'only impact convergence speed' is load-bearing for the design of SAM-GS, but it is supported only by a citation and not by a direct experiment. The paper's own Table 1 shows Aligned-MTL, which explicitly targets angular conflicts, achieving a better Δm% than SAM-GS on CityScapes (5.27 vs 6.41), and the authors acknowledge this. Either provide an experiment that isolates angular and magnitude conflicts to justify the premise, or explicitly limit the applicability claims of SAM-GS to settings in which magnitude conflicts dominate.
minor comments (5)
  1. [§5.2, Table 1] The row labeled 'SI' in Table 1 is never defined or cited; please introduce the method or remove it.
  2. [§4, 'γ is a learnable hyperparameter'] The threshold γ is tuned on validation data in Section 5.2, so calling it 'learnable' is misleading; use 'tunable' or 'hyperparameter' instead.
  3. [§5.3, Table 4] The text says SAM-GS 'surpasses' the STL SAC baseline on MetaWorld MT10, but the reported values overlap (SAM-GS 0.91±0.018 vs STL SAC 0.90±0.032); please soften this claim or support it with a significance test.
  4. [§5.1, Figures 2 and 4] The synthetic-problem claims are based only on trajectory plots; please report quantitative final distances or loss values for each initialization so that 'reaches the global optimum' can be verified.
  5. [Appendix B, Tables 5–7] Standard errors are reported for SAM-GS only; for a fair comparison, report errors for the closest baselines or perform a significance test on the Δm% differences.

Circularity Check

1 steps flagged · score 4.0 of 10

The low-similarity equalization branch of Algorithm 1 is the identity map by construction, so the claimed conservative-equalization mechanism is self-equivalent; aside from this, no fitting-to-target or self-citation circularity is present.

  1. self definitional [Algorithm 1 (low-similarity branch) and Section 4, paragraph after Fig. 1]
    "In cases where task gradients exhibit significantly different magnitudes, our approach equalises their magnitudes to compute a balanced direction not dominated by one task. ... if Ψ < γ then wk = (∥gk∥2 / ∥gk∥2) gk, ∀k ... Update: θ_t = θ_{t−1} − α Σ_{k=1}^K w_k ⊙ g_k"

    In the Ψ<γ branch the coefficient ∥gk∥2/∥gk∥2 equals 1 identically, so wk = gk for every k and the update reduces to θ ← θ − α Σ_k g_k, the ordinary task-gradient sum. This is exactly the regime the text says triggers equalization of magnitudes and conservative learning when gradients are dissimilar; the printed mechanism therefore applies no equalization and no conservative scaling. The claimed low-similarity behavior is the input gradient itself by construction, so the central 'gradient surgery' mechanism collapses into the unsurgered aggregation it was supposed to replace. This is a self-definitional identity, not a fitted parameter being renamed as a prediction.

full rationale

Aside from the identity branch, I find no classic circularity. The hyperparameters (β1, β2, γ) are selected on validation data and reported per benchmark; they are not fitted to the reported Δm% test targets, and the similarity measure ψ is defined independently of benchmark outcomes. The single overlapping-author citation ([7], an encyclopedia chapter by two co-authors) is used only to frame MTL as multi-objective optimization and is not load-bearing; no uniqueness theorem is invoked and no prior-work ansatz is imported. The paper's own admission that Aligned-MTL outperforms SAM-GS on CityScapes is a stated limitation about angle-based conflicts, not a circular derivation. The printed algorithm's low-similarity branch is a serious correctness/reproducibility defect and a self-equivalence by construction in the method's core mechanism, so it raises the circularity score above zero; however, the benchmark comparisons are external evidence not derived from the equations, so the paper is not wholly circular.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

The method introduces no new entities. The free parameters are standard optimizer hyperparameters and one threshold that is tuned per dataset. The core domain assumption, that angle conflicts are unimportant, is load-bearing and is contradicted by the authors' own CityScapes result.

free parameters (3)
  • gamma (similarity threshold) = 0.9 in most experiments; ablation over {0, 0.1, 0.3, 0.5, 0.7, 0.9, 1}
    Tuned per benchmark on validation data. The method's behavior changes strongly with gamma, as shown in Figure 5.
  • beta1 (momentum decay) = 0.9
    Set to 0.9 in all experiments; standard value.
  • beta2 (similarity momentum decay) = 0.9 or 0.99 depending on benchmark
    Tuned per benchmark; values 0.9 and 0.99 are used.
assumptions (2)
  • domain assumption Angle-based gradient conflicts only affect convergence speed and can be ignored; magnitude conflicts are the primary cause of MTL suboptimality.
    Section 4 states 'Angle-based gradient conflicts are intentionally disregarded, as they only impact convergence speed.' This underpins the design that uses only magnitude similarity.
  • domain assumption The harmonic-mean-like magnitude similarity psi is an adequate measure of gradient conflict and a valid basis for modulating the update.
    Equation (3) defines psi; the method uses it as the detector and modulator. No formal justification is given beyond intuition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gradient Similarity Surgery in Multi-Task Deep Learning." pith.science (2026). https://pith.science/paper/DQ5GTTDC

@misc{pith2026250606130,
  author       = {Pith},
  title        = {Pith review of: Gradient Similarity Surgery in Multi-Task Deep Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DQ5GTTDC}},
  note         = {Machine review of arXiv:2506.06130}
}
abstract

The multi-task learning ($MTL$) paradigm aims to simultaneously learn multiple tasks within a single model capturing higher-level, more general hidden patterns that are shared by the tasks. In deep learning, a significant challenge in the backpropagation training process is the design of advanced optimisers to improve the convergence speed and stability of the gradient descent learning rule. In particular, in multi-task deep learning ($MTDL$) the multitude of tasks may generate potentially conflicting gradients that would hinder the concurrent convergence of the diverse loss functions. This challenge arises when the gradients of the task objectives have either different magnitudes or opposite directions, causing one or a few to dominate or to interfere with each other, thus degrading the training process. Gradient surgery methods address the problem explicitly dealing with conflicting gradients by adjusting the overall gradient trajectory. This work introduces a novel gradient surgery method, the Similarity-Aware Momentum Gradient Surgery (SAM-GS), which provides an effective and scalable approach based on a gradient magnitude similarity measure to guide the optimisation process. The SAM-GS surgery adopts gradient equalisation and modulation of the first-order momentum. A series of experimental tests have shown the effectiveness of SAM-GS on synthetic problems and $MTL$ benchmarks. Gradient magnitude similarity plays a crucial role in regularising gradient aggregation in $MTDL$ for the optimisation of the learning process.

Figures

Figures reproduced from arXiv: 2506.06130 by the authors.

Figure 1
Figure 1. Illustration of four scenarios for two task gradients, [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Trajectories for different methods starting from 7 different initial points: [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the multi-task optimisation problem ( [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Trajectories for different methods in the second synthetic problem: Linear [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Ablation study over γ. The plot shows the performance, in terms of ∆m%, of SAM-GS across three supervised learning settings with γ values of {0, 0.1, 0.3, 0.5, 0.7, 0.9, 1}, including standard deviation (SD) [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: SAM-GS trajectories with different values of the loss weighting parameter. [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: trajectories with different values of the loss weighting parameter on the [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: MTL optimisation methods trajectories with different values of the loss [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 38 canonical work pages

  1. [1]

    Machine Learning28, 41–75 (1997)

    Caruana, R.: Multitask learning. Machine Learning28, 41–75 (1997)

    1997

  2. [2]

    In: Proceed- ingsofthe35thInternationalConferenceonMachineLearning.vol.80,pp.794–803 (2018)

    Chen, Z., Badrinarayanan, V., Lee, C.Y., Rabinovich, A.: GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In: Proceed- ingsofthe35thInternationalConferenceonMachineLearning.vol.80,pp.794–803 (2018)

    work page 2018

  3. [3]

    In: Advances in Neural Information Processing Systems

    Chen, Z., Ngiam, J., Huang, Y., Luong, T., Kretzschmar, H., Chai, Y., Anguelov, D.: Just pick a sign: Optimizing deep multitask models with gradient sign dropout. In: Advances in Neural Information Processing Systems. vol. 33, pp. 2039–2050 (2020)

    work page 2020

  4. [4]

    In: IEEE International Conference on Robotics and Automation (ICRA)

    Choi, W., Shin, M., Lee, H., Cho, J., Park, J., Im, S.: Multi-task learning for real- time autonomous driving leveraging task-adaptive attention generator. In: IEEE International Conference on Robotics and Automation (ICRA). pp. 14732–14739 (2024)

    work page 2024

  5. [5]

    In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR)

    Cipolla, R., Gal, Y., Kendall, A.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR). pp. 7482–7491 (2018)

    work page 2018

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    work page 2016

  7. [7]

    Multi-Task Deep Learning as Multi-Objective Optimization (2020)

    Di Fatta, G., Nicosia, G., Ojha, V., Pardalos, P.: Encyclopedia of Optimization, chap. Multi-Task Deep Learning as Multi-Objective Optimization (2020)

    work page 2020

  8. [8]

    Dong, D., Wu, H., He, W., Yu, D., Wang, H.: Multi-task learning for multiple language translation. In: Proceedings of the 53rd Annual Meeting of the Associa- tion for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 1723–1732 (2015)

    work page 2015

Show all 41 references
  1. [9]

    In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

    Dong, X., Wu, R., Xiong, C., Li, H., Cheng, L., He, Y., Qian, S., Cao, J., Mo, L.: Gdod: Effective gradient descent using orthogonal decomposition for multi-task learning. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. pp. 386–3...

    2022

  2. [10]

    Comptes Rendus Mathematique350(5), 313–318 (2012)

    Désidéri, J.A.: Multiple-gradient descent algorithm (mgda) for multiobjective op- timization. Comptes Rendus Mathematique350(5), 313–318 (2012)

    2012

  3. [11]

    In: Pattern Recognition

    Elich, C., Kirchdorfer, L., Köhler, J.M., Schott, L.: Examining common paradigms in multi-task learning. In: Pattern Recognition. pp. 131–147 (2025)

    2025

  4. [12]

    In: Advances in Neural Information Processing Systems

    Fifty, C., Amid, E., Zhao, Z., Yu, T., Anil, R., Finn, C.: Efficiently identifying task groupings for multi-task learning. In: Advances in Neural Information Processing Systems. vol. 34, pp. 27503–27516 (2021)

    2021

  5. [13]

    In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

    Guo, M., Haque, A., Huang, D.A., Yeung, S., Fei-Fei, L.: Dynamic task priori- tization for multitask learning. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

    2018

  6. [14]

    IEEE Transactions on Medical Imaging41(12), 3969–3980 (2022)

    Hao, J., Shen, T., Zhu, X., Liu, Y., Behera, A., Zhang, D., Chen, B., Liu, J., Zhang, J., Zhao, Y.: Retinal structure detection in octa image via voting-based multitask learning. IEEE Transactions on Medical Imaging41(12), 3969–3980 (2022)

    2022

  7. [15]

    In: Medical Image Comput- ing and Computer Assisted Intervention – MICCAI 2023 Workshops

    Kim, S., Purdie, T.G., McIntosh, C.: Cross-task attention network: Improving multi-task learning for medical imaging applications. In: Medical Image Comput- ing and Computer Assisted Intervention – MICCAI 2023 Workshops. pp. 119–128 (2023)

    2023

  8. [16]

    The third Interna- tional Conference on Learning Representations (2015)

    Kingma, D.P.: Adam: A method for stochastic optimization. The third Interna- tional Conference on Learning Representations (2015)

    2015

  9. [17]

    In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability (1951)

    Kuhn, H.W., Tucker, A.W.: Nonlinear programming. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability (1951)

    1951

  10. [18]

    Transactions on Machine Learning Re- search pp

    Lin, B., Ye, F., Zhang, Y., Tsang, I.W.: Reasonable effectiveness of random weight- ing: A litmus test for multi-task learning. Transactions on Machine Learning Re- search pp. 2835–8856 (2022)

    2022

  11. [19]

    In: Advances in Neural Information Processing Systems

    Liu, B., Feng, Y., Stone, P., Liu, Q.: Famo: Fast adaptive multitask optimization. In: Advances in Neural Information Processing Systems. vol. 36, pp. 57226–57243 (2023)

    2023

  12. [20]

    In: Advances in Neural Information Processing Systems

    Liu, B., Liu, X., Jin, X., Stone, P., Liu, Q.: Conflict-averse gradient descent for multi-task learning. In: Advances in Neural Information Processing Systems. vol. 34, pp. 18878–18890 (2021)

    2021

  13. [21]

    In: International Conference on Learning Representations (2021)

    Liu, L., Li, Y., Kuang, Z., Xue, J.H., Chen, Y., Yang, W., Liao, Q., Zhang, W.: Towards impartial multi-task learning. In: International Conference on Learning Representations (2021)

    2021

  14. [22]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Liu, S., Johns, E., Davison, A.J.: End-to-end multi-task learning with attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1871–1880 (2019)

    2019

  15. [23]

    In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV)

    Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). pp. 3730–3738 (2015)

    2015

  16. [24]

    In: Proceedings of the 39th International Conference on Machine Learning

    Navon, A., Shamsian, A., Achituve, I., Maron, H., Kawaguchi, K., Chechik, G., Fetaya, E.: Multi-task learning as a bargaining game. In: Proceedings of the 39th International Conference on Machine Learning. vol. 162, pp. 16428–16446 (2022)

    2022

  17. [25]

    Expert Systems with Applications230, 120587 (2023)

    Ong, J., Herremans, D.: Constructing time-series momentum portfolios with deep multi-task learning. Expert Systems with Applications230, 120587 (2023)

    2023

  18. [26]

    Ruder, S.: An overview of multi-task learning in deep neural networks (2017), http://arxiv.org/abs/1706.05098

    2017 arXiv

  19. [27]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Senushkin, D., Patakin, N., Kuznetsov, A., Konushin, A.: Independent component alignment for multi-task learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 20083–20093 (2023) Multi-Task Optimisation Similarity Based 17

    2023

  20. [28]

    In: Advances in Neural Information Processing Systems

    Shen, J., Zhen, X., Worring, M., Shao, L.: Variational multi-task learning with gumbel-softmax priors. In: Advances in Neural Information Processing Systems. vol. 34, pp. 21031–21042 (2021)

    2021

  21. [29]

    In: The Eleventh International Conference on Learning Representations (2023)

    SHI, G., Li, Q., Zhang, W., Chen, J., Wu, X.M.: Recon: Reducing conflicting gradients from the root for multi-task learning. In: The Eleventh International Conference on Learning Representations (2023)

    2023

  22. [30]

    In: Proceedings of the 12th European Conference on Computer Vision (ECCV)

    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: Proceedings of the 12th European Conference on Computer Vision (ECCV). pp. 746–760 (2012)

    2012

  23. [31]

    In: Proceedings of the 38th International Conference on Machine Learning

    Sodhani, S., Zhang, A., Pineau, J.: Multi-task reinforcement learning with context- based representations. In: Proceedings of the 38th International Conference on Machine Learning. vol. 139, pp. 9767–9779 (2021)

    2021

  24. [32]

    Standley, T., Zamir, A., Chen, D., Guibas, L., Malik, J., Savarese, S.: Which tasks should be learned together in multi-task learning? In: Proceedings of the 37th International Conference on Machine Learning. vol. 119, pp. 9120–9132 (2020)

    2020

  25. [33]

    IEEE Transactions on Neural Networks and Learning Systems35(7), 9579–9590 (2024)

    Tian, Y., Bai, K.: End-to-end multitask learning with vision transformer. IEEE Transactions on Neural Networks and Learning Systems35(7), 9579–9590 (2024)

    2024

  26. [34]

    In: Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Inter- preting Neural Networks for NLP

    Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S.: GLUE: A multi- task benchmark and analysis platform for natural language understanding. In: Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Inter- preting Neural Networks for NLP. pp. 353–355 (2018)

    2018

  27. [35]

    In: Proceedings of Robotics: Sci- ence and Systems (2020)

    Wulfmeier, M., Abdolmaleki, A., Hafner, R., Springenberg, J.T., Neunert, M., Siegel, N., Hertweck, T., Lampe, T., Heess, N., Riedmiller, M.: Compositional transfer in hierarchical reinforcement learning. In: Proceedings of Robotics: Sci- ence and Systems (2020)

    2020

  28. [36]

    In: Advances in Neural Information Processing Systems

    Yang, R., Xu, H., WU, Y., Wang, X.: Multi-task reinforcement learning with soft modularization. In: Advances in Neural Information Processing Systems. vol. 33, pp. 4767–4777 (2020)

    2020

  29. [37]

    In: Advances in Neural Information Processing Systems

    Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., Finn, C.: Gradient surgery for multi-task learning. In: Advances in Neural Information Processing Systems. vol. 33, pp. 5824–5836 (2020)

    2020

  30. [38]

    In: Proceedings of the Conference on Robot Learning (2020)

    Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., Levine, S.: Meta- world:Abenchmarkandevaluationformulti-taskandmetareinforcementlearning. In: Proceedings of the Conference on Robot Learning (2020)

    2020

  31. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Zamir, A.R., Sax, A., Shen, W., Guibas, L.J., Malik, J., Savarese, S.: Taskonomy: Disentangling task transfer learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    2018

  32. [40]

    In: Advances in Neural Information Processing Systems

    Zhuang, J., Tang, T., Ding, Y., Tatikonda, S.C., Dvornek, N., Papademetris, X., Duncan, J.: Adabelief optimizer: Adapting stepsizes by the belief in observed gra- dients. In: Advances in Neural Information Processing Systems. vol. 33, pp. 18795– 18806 (2020) 18 T. Borsani et a...

    2020

  33. [41]

    where the two loss functions are equally weighted. In order to allow the replication the results presented in the main paper, here we provide the detailed configuration for the seven initial points: θinit ∈ {(−8,5.0),(−3,7.5),(0,10.0),(3,7.5),(8,5.0),(−10,−2.5),(10,−2.5)}. We ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.