Pith. sign in

REVIEW 3 major objections 6 minor 69 references

Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Representation-level task saliency regularizers—entropy penalization for task-specific patterns plus contrastive cross-task alignment—let multi-task learning surpass previous optimizers even with plain equal weighting.

desk verdict Interesting TSR, but the CSA loss in Eq. 9 is mis-specified and cannot do what the paper claims; worth a referee, not acceptance. read the letter →

arxiv 2507.21049 v1 pith:IJYGXC3F submitted 2025-07-28 cs.LG cs.CV

Zedong Wang , Siyuan Li , Dan Xu This is my paper · ORCID
classification cs.LGcs.CV
keywords multi-tasklearningtasksaliencynegativetransfercontrastivealignmententropyregularizationrepresentationoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that multi-task optimization should operate on the shared representation space rather than only on loss weights or gradients. It introduces Rep-MTL, which computes a task saliency—the gradient of each task's loss with respect to the shared representation—and adds two regularizers: an entropy penalty that keeps task-specific spatial patterns distinct, and a contrastive alignment that pulls together saliency affinities across tasks for the same sample. The claim is that these representation-level manipulations mitigate negative transfer and explicitly encourage inter-task complementarity without changing the optimizer or architecture. The paper reports that Rep-MTL with plain equal weighting outperforms previous leading methods on four benchmarks, including a roughly 48% improvement in task-level gain over DB-MTL on NYUv2, while running faster than most gradient-manipulation baselines.

What carries the argument

The central object is representation-level task saliency, defined as $S_t = \nabla_Z \mathcal{L}_t(\theta_s, \theta_t)$, the gradient of task $t$'s loss with respect to the shared representation $Z$. Two modules are built on it: TSR aggregates channel-wise saliencies into per-region distributions across tasks and applies entropy penalization $\mathcal{L}_{tsr}$ to keep task-specific spatial patterns distinctive; CSA builds channel affinity matrices $M_t = S_t S_t^\top$, computes per-sample anchors, and applies a contrastive loss $\mathcal{L}_{csa}$ that aligns same-sample task affinities while repelling different-sample anchors, thereby promoting inter-task complementarity while preserving task distinctiveness.

What would settle it

Train a multi-task model with Rep-MTL and repeat the same training run after randomly shuffling the channel order of the saliency maps used in TSR and CSA; if the gains over equal weighting persist under shuffled saliencies, the regularizers are acting on noise rather than on meaningful task-interaction structure.

Watch

Extended reading notes

Core claim

The central claim is that task saliency—the gradient of each task loss with respect to the shared backbone representation—carries enough signal about where tasks interact that regularizing it improves multi-task training. Rep-MTL combines Task-specific Saliency Regulation, which computes per-region saliency distributions across tasks and penalizes their entropy so that task-specific spatial patterns remain distinct, with Cross-task Saliency Alignment, which builds channel affinity matrices from the saliencies, normalizes anchors per sample, and applies a contrastive loss that treats same-sample task affinities as positives and other-sample anchors as negatives. Together these terms steer shared representation learning toward preserving individual-task information while actively sharing complementary patterns. Evaluated on NYUv2, Cityscapes, Office-31, and Office-Home with a hard-parameter-sharing architecture, Rep-MTL with equal weighting yields positive task-level gains where most baselines show negative transfer, and Power Law exponent analysis is used to argue that the backbone and task heads are better trained and more balanced.

Load-bearing premise

The load-bearing premise is that the aggregated saliency distributions and their affinity matrices faithfully separate task-specific from task-generic patterns, so that entropy minimization preserves task-specific learning without suppressing shared features and contrastive alignment of noisy saliencies reliably extracts complementarity.

Editorial extensions

If this is right

  • Rep-MTL with equal weighting achieves positive average task gains on benchmarks where most MTO baselines show negative transfer, suggesting representation-level regularization is a viable alternative to optimizer-centric conflict solving.
  • The two modules are complementary: CSA alone improves backbone sharing (lower PL exponent), while TSR alone yields lower and more balanced decoder PL exponents indicative of reduced negative transfer.
  • Since Rep-MTL requires no changes to the optimizer or architecture, it can be combined with existing loss-scaling or gradient-manipulation policies.
  • On NYUv2 Rep-MTL improves the task-level gain from +1.15 (DB-MTL) to +1.70, a roughly 48% relative improvement, and on Cityscapes from +0.20 to +0.62.
  • Rep-MTL is computationally cheaper than typical gradient-manipulation methods, about 26% faster per epoch than Nash-MTL and 12% faster than FairGrad.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable prediction of the entropy-minimization premise is that TSR should help most when tasks have spatially overlapping saliency patterns; on task pairs with already distinct saliencies, TSR's benefit should shrink, which could be probed by measuring the entropy of saliency distributions under equal-weighting training.
  • If the saliency maps are dominated by a few channels or noisy regions, TSR could over-regularize toward spurious patterns; a stress test would be to corrupt the saliency maps, for example by shuffling channels, and check whether the performance gain collapses—an experiment the paper does not report.
  • The contrastive alignment relies on batch-level negatives, so its effectiveness likely depends on batch size; scaling to larger batches or using a memory bank could strengthen or destabilize CSA, an untested extension.
  • The PL exponent analysis is proxy evidence; a direct test would be to measure per-task generalization on held-out data after ablating TSR and CSA, which the paper only reports through aggregate metrics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Rep-MTL, a representation-level task-saliency regularization method for multi-task learning. Task saliency is defined as the gradient of each task loss with respect to the shared representation. Two complementary regularizers are introduced: Task-specific Saliency Regulation (TSR), which minimizes the entropy of per-location task saliency distributions, and Cross-task Saliency Alignment (CSA), which aims to align sample-wise saliency affinity maps across tasks in a contrastive manner. The method is evaluated on NYUv2, Cityscapes, Office-Home, and Office-31 against 23 MTO baselines, reporting gains over equal weighting and, on several benchmarks, over the prior DB-MTL method. The paper also uses Power Law exponent analysis to argue that Rep-MTL improves both cross-task sharing and task-specific training.

Significance. If the method works as described, the paper makes a useful conceptual contribution: it shifts MTO attention from optimizer-centric conflict resolution to regularizing the shared representation space, and it explicitly targets inter-task complementarity rather than only negative transfer. The empirical scope is a strength: four benchmarks, 23 baselines, efficiency comparisons, and a component-level PL-exponent ablation. The claim is falsifiable and the method is architecturally simple. However, the correctness of the central mechanism, especially CSA, hinges on the precise definition of the loss, and the current formulation is not clear enough to support the advertised behavior.

major comments (3)
  1. [Sec. 3.3, Eqs. (7)-(9)] The CSA loss is not actually defined for multiple tasks. Eq. (7) defines per-task affinity maps M_t, but the anchor in Eq. (8) drops the task index, and Eq. (9) contains no sum over t; the positive pair (z^a_b, z^t_b) can refer to at most one task, and the negatives are anchors of other samples rather than saliency maps of other tasks. Under either natural reading, the loss does not implement the advertised cross-task alignment or task-distinctiveness, so the central claim that CSA explicitly promotes inter-task complementarity is not supported by the equations as written. Please rewrite the objective with explicit per-task notation, for example a sum over t and/or an anchor that averages over tasks, and state exactly which pairs are positive and which are negative.
  2. [Tables 2-3 and Secs. 4.1-4.2] The headline improvements over DB-MTL are within one standard deviation: Cityscapes +0.62±0.53 vs. +0.20±0.40 and Office-Home +0.41±0.58 vs. +0.17±0.44. The text says Rep-MTL 'slightly exceeds' and 'advances the previous SOTA by approximately 140%', but these differences are not statistically significant at conventional levels. Please report paired comparisons or additional seeds, and temper the wording of the SOTA claims.
  3. [Sec. 4.5, Appendix D.1] The two regularization weights are selected by grid search over {0.7, 0.9, 1.1, 1.3, 1.5} on the same benchmarks whose test metrics are reported, and the main tables do not list the selected values. This leaves open the possibility that the reported gains are partly due to tuning on the evaluation set. Please specify the selection protocol, report the chosen lambda values per dataset, and ideally include a fixed-weight experiment.
minor comments (6)
  1. [Sec. 3.4, Eq. (10)] The text says 'where λ1 and λ2 balance' but the equation uses λ_tsr and λ_csa; unify the notation.
  2. [Abstract and Sec. 1] The project page is listed as 'HERE' in both the abstract and the introduction; this placeholder should be replaced before submission.
  3. [Tables 3 and 5] The MoCo row is missing standard deviations in both Office-Home and Office-31 tables; complete these entries.
  4. [Figure 2] The negative-pair annotation in the figure, B×(B−1), does not match the text's claim that task distinctiveness is maintained through cross-task negatives; once Eq. (9) is clarified, update the figure accordingly.
  5. [Appendix A, Office-Home/Office-31 settings] The sentence 'The Avg. reported ... represents the mean performance gains across three independent tasks' appears to describe average accuracy rather than performance gains; rephrase to avoid confusion.
  6. [Sec. 4.3 and Appendix C] The PL-exponent results are reported as point values or as visual differences without error bars or a table of values; adding numerical values with uncertainties would make the analysis reproducible.

Circularity Check

0 steps flagged · score 2.0 of 10

No material circularity: benchmark-based gains are independent, with only minor self-citations in the PL-exponent analysis.

full rationale

The paper's central claim is empirical: adding the TSR and CSA regularizers to a hard-parameter-sharing MTL model produces better task-level performance (Tables 1-3). The gains are measured directly against STL baselines and external MTO methods, so they cannot be a renaming of the method's own inputs. The saliency-based objectives (Eqs. 4-6 and 9-10) are not derived from, nor do they define, the evaluation metrics Delta_ptask and Delta_pmetric (Eqs. 11-12). Hyperparameters lambda_tsr and lambda_csa are tuned with a grid search, not fitted to reproduce a target quantity, so no prediction is forced by construction. The only self-citations are Refs. [25,26] in the Power-Law exponent discussion; the underlying PL theory is external (Refs. [41,44,45,46]), and the PL analysis is a supplementary diagnostic, so this is a minor, non-load-bearing self-citation. One non-circular correctness concern should be flagged: Eq. (9) is underspecified. There is no summation over tasks and no task index on the positive task affinity z^t_b, while the negatives are anchors of other samples: L_csa = (1/B) sum_b [ -log exp(sim(z^a_b, z^t_b)/tau) / sum_{k!=b} exp(sim(z^a_b, z^a_k)/tau) ]. As written, the loss reduces to a sample-anchor contrastive objective and cannot by itself enforce the advertised cross-task alignment; this undermines the mechanism claim in Sec. 3.3, but it does not make the benchmark results circular. Score 2 reflects the minor self-citation only.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities; it relies on existing notions of gradient-based saliency, entropy regularization, and contrastive learning. The main unstated costs are the two regularization weights and the contrastive temperature, none of which are reported with their chosen values.

free parameters (3)
  • lambda_tsr = Not reported per dataset; grid searched over {0.7, 0.9, 1.1, 1.3, 1.5}
    Weight of the Task-specific Saliency Regulation loss (Eq. 10). Selected by grid search for each benchmark, but the chosen values are not listed.
  • lambda_csa = Not reported per dataset; grid searched over {0.7, 0.9, 1.1, 1.3, 1.5}
    Weight of the Cross-task Saliency Alignment loss (Eq. 10). Selected by grid search for each benchmark, but the chosen values are not listed.
  • tau (contrastive temperature) = Not reported
    Temperature in the contrastive loss (Eq. 9). No value or sensitivity analysis is given in the paper.
assumptions (5)
  • domain assumption Task saliency defined as the gradient of the task loss w.r.t. the shared representation (Eq. 3) is a valid indicator of task-important regions.
    This is the foundation of the method, stated in Sec. 3.1 and supported only by analogy to Grad-CAM and prior saliency work, not by a formal justification in the MTL setting.
  • domain assumption Penalizing the entropy of the normalized saliency distribution (Eq. 6) preserves task-specific patterns and mitigates negative transfer.
    The paper assumes low-entropy saliency corresponds to task-salient patterns and that keeping them helps individual tasks; this is plausible but not proven.
  • domain assumption Aligning sample-wise saliency affinity maps across tasks (Eq. 9) promotes beneficial inter-task complementarity.
    The contrastive objective encourages positive pairs (same sample, different tasks) to be similar, but the link to improved multi-task generalization is assumed.
  • domain assumption Power Law exponent alpha in the range [2,4] (or [2,6] in places) indicates well-trained models, and lower balanced alpha across task heads indicates effective task-specific learning.
    Used in Sec. 4.3 and Appendix C to interpret results; the paper cites prior works (including the authors' own) for this, but the connection to multi-task transfer quality is not independently validated.
  • ad hoc to paper The grid search range {0.7,0.9,1.1,1.3,1.5} for lambda_tsr and lambda_csa yields stable, positive gains (Delta_ptask > 0) across benchmarks.
    Appendix D.1 reports this empirical observation, but it is used to justify the choice of hyperparameters, which is a form of tuning on the evaluation benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning." pith.science (2026). https://pith.science/paper/IJYGXC3F

@misc{pith2026250721049,
  author       = {Pith},
  title        = {Pith review of: Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IJYGXC3F}},
  note         = {Machine review of arXiv:2507.21049}
}
read the original abstract

Despite the promise of Multi-Task Learning in leveraging complementary knowledge across tasks, existing multi-task optimization (MTO) techniques remain fixated on resolving conflicts via optimizer-centric loss scaling and gradient manipulation strategies, yet fail to deliver consistent gains. In this paper, we argue that the shared representation space, where task interactions naturally occur, offers rich information and potential for operations complementary to existing optimizers, especially for facilitating the inter-task complementarity, which is rarely explored in MTO. This intuition leads to Rep-MTL, which exploits the representation-level task saliency to quantify interactions between task-specific optimization and shared representation learning. By steering these saliencies through entropy-based penalization and sample-wise cross-task alignment, Rep-MTL aims to mitigate negative transfer by maintaining the effective training of individual tasks instead pure conflict-solving, while explicitly promoting complementary information sharing. Experiments are conducted on four challenging MTL benchmarks covering both task-shift and domain-shift scenarios. The results show that Rep-MTL, even paired with the basic equal weighting policy, achieves competitive performance gains with favorable efficiency. Beyond standard performance metrics, Power Law exponent analysis demonstrates Rep-MTL's efficacy in balancing task-specific learning and cross-task sharing. The project page is available at HERE.

Figures

Figures reproduced from arXiv: 2507.21049 by the authors.

Figure 1
Figure 1. Overview of Rep-MTL and existing MTO methods. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The Rep-MTL framework. It comprises two complementary task saliency driven modules: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of PL exponent alpha [46] for backbone pa￾rameters trained with diverse MTO methods on NYUv2 [59]. It validates how well the backbone adapts to MTL objectives, where lower values indicate more effective training. Values outside [2, 4] suggest potential over- or under-training. We leverage this to show how methods affect model updates, as well-trained backbones sug￾gest beneficial cross-task sharing to the… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Visualization of PL exponent alpha [44, 46] for tasks-specific heads (3 tasks) trained with different MTO methods on NYUv2 [59]. PL exponent quantifies how well each decoder adapts to its task-specific objective, where lower values practically indicate more effective m…
Figure 5
Figure 5. Figure 5: Ablation studies through PL exponent metrics [ [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Ablation studies through PL exponent metrics [ [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 8
Figure 8. Figure 8: Training time per epoch comparison across different [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Learning rate sensitivity analysis of our proposed Rep [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 63 canonical work pages

  1. [1]

    Saliency-regularized deep multi-task learning

    Guangji Bai and Liang Zhao. Saliency-regularized deep multi-task learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 15–25, 2022. 1, 2

    work page 2022

  2. [2]

    Fair resource allocation in multi-task learning

    Hao Ban and Kaiyi Ji. Fair resource allocation in multi-task learning. In Forty-first International Conference on Machine Learning, 2024. 1, 2, 5, 6, 8

    work page 2024

  3. [3]

    Sparks of artificial general intelligence: Early experiments with gpt-4

    S ´ebastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Jo- hannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023. 1

    arXiv 2023

  4. [4]

    Multitask learning

    Rich Caruana. Multitask learning. Machine Learning, 28(1): 41–75, 1997. 1, 3

    work page 1997

  5. [5]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818, 2018. 5, 6, 12

    work page 2018

  6. [6]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 4

    work page 2020

  7. [7]

    Big self-supervised mod- els are strong semi-supervised learners

    Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised mod- els are strong semi-supervised learners. Advances in neural information processing systems, 33:22243–22255, 2020. 4

    work page 2020

  8. [8]

    GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks

    Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and An- drew Rabinovich. GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In Inter- national Conference on Machine Learning, 2018. 2, 5, 6, 7, 13

    work page 2018

Show all 69 references
  1. [9]

    Just pick a sign: Optimizing deep multitask models with gra- dient sign dropout

    Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gra- dient sign dropout. In Neural Information Processing Sys- tems, 2020. 1, 2, 5, 6, 7, 13

    2020

  2. [10]

    MultiNet++: Multi-stream feature ag- gregation and geometric loss strategy for multi-task learning

    Sumanth Chennupati, Ganesh Sistu, Senthil Yogamani, and Samir A Rawashdeh. MultiNet++: Multi-stream feature ag- gregation and geometric loss strategy for multi-task learning. In IEEE Conference on Computer Vision and Pattern Recog- nition Workshops, 2019. 2, 5, 6, 7, 13

    2019

  3. [11]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In IEEE Conference on Computer Vision and Pattern Recognition ,

  4. [12]

    Improvable gap bal- ancing for multi-task learning

    Yanqi Dai, Nanyi Fei, and Zhiwu Lu. Improvable gap bal- ancing for multi-task learning. In Uncertainty in Artificial Intelligence, pages 496–506. PMLR, 2023. 2, 5, 6, 7, 13

    2023

  5. [13]

    Multiple-gradient descent algorithm (MGDA) for multiobjective optimization

    Jean-Antoine D ´esid´eri. Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. Comptes Rendus Mathematique, 350(5):313–318, 2012. 2, 5, 6, 7, 13

    2012

  6. [14]

    Multi-task self- supervised visual learning

    Carl Doersch and Andrew Zisserman. Multi-task self- supervised visual learning. In Proceedings of the IEEE inter- national conference on computer vision , pages 2051–2060,

  7. [15]

    Miti- gating gradient bias in multi-objective learning: A provably convergent approach

    Heshan Devaka Fernando, Han Shen, Miao Liu, Subhajit Chaudhury, Keerthiram Murugesan, and Tianyi Chen. Miti- gating gradient bias in multi-objective learning: A provably convergent approach. In International Conference on Learn- ing Representations, 2023. 2, 5, 6, 7, 13

    2023

  8. [16]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Ghesh- laghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neur...

    2020

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6, 12

    2016

  10. [18]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020. 4

    2020

  11. [19]

    MetaBalance: improving multi-task recommendations via adapting gradient magnitudes of aux- iliary tasks

    Yun He, Xue Feng, Cheng Cheng, Geng Ji, Yunsong Guo, and James Caverlee. MetaBalance: improving multi-task recommendations via adapting gradient magnitudes of aux- iliary tasks. In ACM Web Conference, 2022. 5, 6, 7, 13

    2022

  12. [20]

    Position: The platonic representation hypothesis

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The platonic representation hypothesis. In Forty-first International Conference on Machine Learning ,

  13. [21]

    Rotograd: Gradient ho- mogenization in multitask learning

    Adri ´an Javaloy and Isabel Valera. Rotograd: Gradient ho- mogenization in multitask learning. In International Confer- ence on Learning Representations, 2021. 1, 2

    2021

  14. [22]

    Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics

    Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics. In IEEE Conference on Computer Vision and Pattern Recognition, 2018. 1, 2, 5, 6, 7, 13

    2018

  15. [23]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 1

    2023

  16. [24]

    In defense of the uni- tary scalarization for deep multi-task learning

    Vitaly Kurin, Alessandro De Palma, Ilya Kostrikov, Shimon Whiteson, and Pawan K Mudigonda. In defense of the uni- tary scalarization for deep multi-task learning. Advances in 9 Neural Information Processing Systems , 35:12169–12183,

  17. [25]

    Unveiling the backbone-optimizer coupling bias in visual representation learning

    Siyuan Li, Juanxi Tian, Zedong Wang, Luyuan Zhang, Zicheng Liu, Weiyang Jin, Yang Liu, Baigui Sun, and Stan Z Li. Unveiling the backbone-optimizer coupling bias in visual representation learning. arXiv preprint arXiv:2410.06373 ,

    arXiv

  18. [26]

    Siyuan Li, Zedong Wang, Zicheng Liu, Juanxi Tian, Di Wu, Cheng Tan, Weiyang Jin, and Stan Z. Li. Openmixup: Open mixup toolbox and benchmark for visual representa- tion learning, 2024. 7

    2024

  19. [27]

    Libmtl: A python library for deep multi-task learning

    Baijiong Lin and Yu Zhang. Libmtl: A python library for deep multi-task learning. The Journal of Machine Learning Research, 24(1):9999–10005, 2023. 1, 5, 6

    2023

  20. [28]

    Rea- sonable effectiveness of random weighting: A litmus test for multi-task learning

    Baijiong Lin, Feiyang Ye, Yu Zhang, and Ivor Tsang. Rea- sonable effectiveness of random weighting: A litmus test for multi-task learning. Transactions on Machine Learning Re- search, 2022. 2, 5, 6, 7, 12, 13

    2022

  21. [29]

    Rea- sonable effectiveness of random weighting: A litmus test for multi-task learning

    Baijiong Lin, Feiyang Ye, Yu Zhang, and Ivor Tsang. Rea- sonable effectiveness of random weighting: A litmus test for multi-task learning. Transactions on Machine Learning Re- search, 2022. 2

    2022

  22. [30]

    Dual-balancing for multi-task learning, 2024

    Baijiong Lin, Weisen Jiang, Feiyang Ye, Yu Zhang, Peng- guang Chen, Ying-Cong Chen, Shu Liu, and James Kwok. Dual-balancing for multi-task learning, 2024. 2, 5, 6, 7, 12, 13

    2024

  23. [31]

    Smooth tchebycheff scalarization for multi-objective optimization

    Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang, Fei Liu, Zhenkun Wang, and Qingfu Zhang. Smooth tchebycheff scalarization for multi-objective optimization. In Forty-first International Conference on Machine Learning, 2024. 2, 5, 6

    2024

  24. [32]

    Conflict-averse gradient descent for multi-task learning

    Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. In Neural Information Processing Systems, 2021. 2, 5, 6, 7, 13

    2021

  25. [33]

    Famo: Fast adaptive multitask optimization

    Bo Liu, Yihao Feng, Peter Stone, and Qiang Liu. Famo: Fast adaptive multitask optimization. Advances in Neural Information Processing Systems, 36, 2024. 1, 2

    2024

  26. [34]

    Towards impartial multi-task learning

    Liyang Liu, Yi Li, Zhanghui Kuang, Jing-Hao Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. Towards impartial multi-task learning. InInternational Con- ference on Learning Representations , 2021. 2, 5, 6, 7, 12, 13

    2021

  27. [35]

    Shikun Liu, Edward Johns, and Andrew J. Davison. End- to-end multi-task learning with attention. In CVPR, pages 1871–1880, 2019. 1, 2, 3, 4, 5, 6, 7, 13

    2019

  28. [36]

    Auto-lambda: Disentangling dynamic task relation- ships

    Shikun Liu, Stephen James, Andrew Davison, and Edward Johns. Auto-lambda: Disentangling dynamic task relation- ships. Transactions on Machine Learning Research , 2022. 2

    2022

  29. [37]

    Unified-io: A unified model for vision, language, and multi-modal tasks

    Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mot- taghi, and Aniruddha Kembhavi. Unified-io: A unified model for vision, language, and multi-modal tasks. In The Eleventh International Conference on Learning Representa- tions, 2022. 1

    2022

  30. [38]

    Unified-io 2: Scaling autoregressive multimodal models with vision language audio and action

    Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. Unified-io 2: Scaling autoregressive multimodal models with vision language audio and action. In Proceed- ings of the IEEE/CVF Conference on Computer Vision ...

    2024

  31. [39]

    Modeling task relationships in multi-task learning with multi-gate mixture-of-experts

    Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceed- ings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining , pages 1930–1939,

    1930

  32. [40]

    Entire space multi-task model: An effective approach for estimating post-click conversion rate

    Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xi- aoqiang Zhu, and Kun Gai. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval ...

    2018

  33. [41]

    Traditional and heavy tailed self regularization in neural network models

    Michael Mahoney and Charles Martin. Traditional and heavy tailed self regularization in neural network models. In In- ternational Conference on Machine Learning , pages 4284–

  34. [42]

    MTAdam: Automatic balancing of multiple training loss terms

    Itzik Malkiel and Lior Wolf. MTAdam: Automatic balancing of multiple training loss terms. In Conference on Empirical Methods in Natural Language Processing, 2021. 5, 6, 7, 13

    2021

  35. [43]

    Robust analysis of multi-task learning on a complex vision system

    Dayou Mao, Yuhao Chen, Yifan Wu, Maximilian Gilles, and Alexander Wong. Robust analysis of multi-task learning on a complex vision system. arXiv preprint arXiv:2402.03557,

    arXiv

  36. [44]

    Martin and Michael W

    Charles H. Martin and Michael W. Mahoney. Implicit self- regularization in deep neural networks: Evidence from ran- dom matrix theory and implications for learning. Journal of Machine Learning Research , 22(165):1–73, 2021. 2, 7, 8, 12, 14

    2021

  37. [45]

    Implicit self- regularization in deep neural networks: Evidence from ran- dom matrix theory and implications for learning

    Charles H Martin and Michael W Mahoney. Implicit self- regularization in deep neural networks: Evidence from ran- dom matrix theory and implications for learning. Journal of Machine Learning Research, 22(165):1–73, 2021. 7, 14

    2021

  38. [46]

    Predicting trends in the quality of state-of-the-art neural net- works without access to training or testing data.Nature Com- munications, 12(1):4122, 2021

    Charles H Martin, Tongsu Peng, and Michael W Mahoney. Predicting trends in the quality of state-of-the-art neural net- works without access to training or testing data.Nature Com- munications, 12(1):4122, 2021. 2, 7, 8, 12, 13, 14

    2021

  39. [47]

    Cross-stitch networks for multi-task learning

    Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Mar- tial Hebert. Cross-stitch networks for multi-task learning. In IEEE Conference on Computer Vision and Pattern Recogni- tion, 2016. 3, 4

    2016

  40. [48]

    Can optimization trajectories explain multi-task transfer? arXiv preprint arXiv:2408.14677, 2024

    David Mueller, Mark Dredze, and Nicholas Andrews. Can optimization trajectories explain multi-task transfer? arXiv preprint arXiv:2408.14677, 2024. 1

    2024 arXiv

  41. [49]

    Multi- task learning as a bargaining game

    Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi- task learning as a bargaining game. In International Con- ference on Machine Learning, pages 16428–16446. PMLR,

  42. [50]

    Language models are unsu- pervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners. OpenAI blog, 1(8):9, 2019. 1

    2019

  43. [51]

    Latent multi-task architecture learning

    Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, and Anders Søgaard. Latent multi-task architecture learning. In Proceedings of the AAAI conference on artificial intelli- gence, pages 4822–4829, 2019. 3 10

    2019

  44. [52]

    Adapting visual category models to new domains

    Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Euro- pean Conference on Computer Vision, 2010. 5, 7, 12, 13

    2010

  45. [53]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,

  46. [54]

    Multi-task learning as multi-objective optimization

    Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Advances in neural informa- tion processing systems, 31, 2018. 2

    2018

  47. [55]

    Independent component alignment for multi-task learning

    Dmitry Senushkin, Nikolay Patakin, Arseny Kuznetsov, and Anton Konushin. Independent component alignment for multi-task learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2, 5, 6, 7, 13

    2023

  48. [56]

    Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer. arXiv preprint arXiv:1701.06538, 2017. 3

    2017 arXiv

  49. [57]

    Go4align: Group optimization for multi-task alignment

    Jiayi Shen, Cheems Wang, Zehao Xiao, Nanne Van No- ord, and Marcel Worring. Go4align: Group optimization for multi-task alignment. arXiv preprint arXiv:2404.06486,

    arXiv

  50. [58]

    Recon: Reducing conflicting gradients from the root for multi-task learning.ArXiv, abs/2302.11289,

    Guangyuan Shi, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao-Ming Wu. Recon: Reducing conflicting gradients from the root for multi-task learning.ArXiv, abs/2302.11289,

    arXiv

  51. [59]

    Indoor segmentation and support inference from RGBD images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from RGBD images. In European Conference on Computer Vi- sion, 2012. 5, 6, 7, 8, 12, 13, 14, 15, 16

    2012

  52. [60]

    Which tasks should be learned together in multi-task learning? In International conference on machine learning, pages 9120–9132

    Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In International conference on machine learning, pages 9120–9132. PMLR,

  53. [61]

    Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations

    Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations. In Proceedings of the 14th ACM Conference on Recommender Systems, pages 269–278, 2020. 3

    2020

  54. [62]

    Deep hashing network for unsupervised domain adaptation

    Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, 2017. 5, 7, 12

    2017

  55. [63]

    Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models

    Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In Interna- tional Conference on Learning Representations, 2021. 2, 5, 6, 7, 13

    2021

  56. [64]

    Do current multi-task optimization methods in deep learning even help? Advances in neural information processing systems, 35:13597–13609, 2022

    Derrick Xin, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, and Orhan Firat. Do current multi-task optimization methods in deep learning even help? Advances in neural information processing systems, 35:13597–13609, 2022. 1, 15

    2022

  57. [65]

    Cross-task knowledge distil- lation in multi-task recommendation

    Chenxiao Yang, Junwei Pan, Xiaofeng Gao, Tingyu Jiang, Dapeng Liu, and Guihai Chen. Cross-task knowledge distil- lation in multi-task recommendation. In Proceedings of the AAAI conference on artificial intelligence, pages 4318–4326,

  58. [66]

    Gradient surgery for multi-task learning

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Pro- cessing Systems, 33:5824–5836, 2020. 1, 2, 5, 6, 7, 13

    2020

  59. [67]

    A survey on negative transfer

    Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu. A survey on negative transfer. IEEE/CAA Journal of Automat- ica Sinica, 10(2):305–329, 2022. 1

    2022

  60. [68]

    Rep-MTL w/o CA

    Yu Zhang and Qiang Yang. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering, 34 (12):5586–5609, 2022. 2 11 Appendix This appendix offers additional empirical analyses, exper- imental results, and further discussions of our work. The appendix s...

    2022

  61. [2022]

    1, 2, 5, 6, 7, 8, 12, 13

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.