REVIEW 4 major objections 6 minor 72 references
Resolving Token-Space Gradient Conflicts: Token Space Manipulation for Transformer-Based Multi-Task Learning
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Token-space fix lifts multi-task transformers with 0.2% parameters
desk verdict Plausible lightweight MTL plug-in with solid empirical gains, but the dynamic mechanism is oversold and the theory is shaky; worth refereeing for the taxonomy and controls. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is an SVD-based token-space decomposition. For each layer, the uncentered covariance of the shared tokens is formed as $\tilde{T}^d_s = \frac{1}{n}\sum_l (T^{l,d}_s)(T^{l,d}_s)^T$, then decomposed as $\tilde{T}^d_s = U\Lambda V^T$, and the eigenvectors are split into a range part $U_R$ and a null part $U_N$ using the proportion of total variance $r$. Each task gradient is projected as $g_{R,i} = (U_R U_R^T)\nabla_{T_{s,k}} L_i$ and $g_{N,i} = (U_N U_N^T)\nabla_{T_{s,k}} L_i$; a conflict is declared when the relevant projected gradients have non-positive dot product. Range-space conflicts trigger token modulators that apply $W \odot T_{s,i} + b$, while null-space conflicts trigger task-specific tokens added to the token sequence. Propositions 1 and 2 use first-order Taylor expansions to show each operation lowers the multi-task loss component associated with its subspace, under the assumption that the input token spans the corresponding space. The covariance, SVD, and allocation are computed once early in fine-tuning and then held fixed, as described in Section B.3 of the supplementary material.
What would settle it
Run the same DTME-MTL recipe but periodically recompute the SVD and conflict measurements during training, comparing against the fixed-snapshot version. If the token space drifts as the backbone is updated, the fixed version should degrade relative to the recomputed one over a longer schedule. A second check is to measure per-subspace gradient cosine similarity before and after each remedy: the paper predicts TM reduces range-space conflicts more than null-space ones, and TE does the reverse, which is directly testable on any task pair.
Extended reading notes
Core claim
The paper establishes a taxonomy of token-space gradient conflicts and shows that the right remedy depends on which subspace the conflict occupies. Using a singular value decomposition of the uncentered covariance of the shared tokens at each layer, it partitions the token space into range and null subspaces by a proportion-of-variance criterion r. Task gradients are projected onto each subspace; if two tasks' projected gradients have non-positive dot product in the range space, the paper inserts an affine modulator (per-task weight and bias) that scales and rotates existing tokens, while conflicts in the null space are handled by concatenating small task-specific tokens before the transformer block. Propositions 1 and 2 argue through Taylor expansions of the multi-task loss that each remedy reduces the loss contributed by its own subspace. Empirically, TM+TE improves over the multi-task baseline by 4.14 mIoU on NYUD-v2 semantic segmentation, by up to 11.71 mIoU on PASCAL-Context, and by 4.67% average Δm on Taskonomy, with parameter increases of 0.046% to 0.46%; it also improves InvPT and TaskPrompter with roughly 0.05% more parameters. The paper further claims that parameter-level conflict resolution, as in Recon, overfits on transformers, whereas token-level resolution does not.
Load-bearing premise
The SVD of the token covariance is computed once early in fine-tuning and then held fixed, so the entire conflict taxonomy and the placement of modulators and tokens depend on that single snapshot remaining representative of the token space for the rest of training.
Editorial extensions
If this is right
- Pretrained transformer backbones can be adapted to new multi-task sets with negligible parameter growth and a single batched forward pass per step, unlike per-task LoRA modules or duplicated network branches.
- The conflict taxonomy transfers across backbone sizes from ViT-T to ViT-L and across existing MTL architectures, suggesting that layer-wise token-space inspection is a usable diagnostic for where negative transfer concentrates.
- Because only a moderate subset of high-conflict layers is expanded, the method implicitly locates the layers where task interference is strongest, and over-expanding beyond roughly 50% of layers hurts performance.
- The claim that parameter-level conflict resolution overfits while token-level resolution does not, if true, shifts attention in multi-task optimization from weight-space gradient surgery to representation-space adaptation.
- Gains disappear when the backbone is frozen after expansion, so continued fine-tuning of the token space is required for the method to work.
Reading between the lines
- The paper does not explore whether periodic recomputation of the SVD during training would improve results; if the token space drifts, a fixed snapshot could become stale, and an adaptive version might be a natural extension.
- The same range/null decomposition could be applied to other token-based architectures such as language models or multimodal transformers, where task conflicts also arise in shared token embeddings.
- A direct head-to-head against per-task LoRA on the same MTL benchmarks would clarify whether the claimed efficiency advantage holds when LoRA allows shared inference through weight merging.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DTME-MTL, a token-space manipulation framework for transformer-based multi-task learning. It computes the uncentered covariance of shared tokens at each layer, performs an SVD, and divides the token space into range and null subspaces using a variance ratio r. Task-specific gradients are projected onto these subspaces; conflicts in the range space are addressed by affine token modulators, while conflicts in the null space are addressed by adding task-specific tokens. The method is evaluated on NYUD-v2, PASCAL-Context, and Taskonomy, and is also applied on top of InvPT and TaskPrompter. The reported results show consistent multi-task improvements with small parameter overheads, e.g., a 4.14 mIoU gain on NYUD-v2 semseg and a 4.67% average improvement on Taskonomy with a 0.118% parameter increase. The paper also includes ablations on layer selection, conflict-type swapping, expansion timing, and modulator configuration.
Significance. If the mechanism claimed in the paper holds, DTME-MTL offers a lightweight, off-the-shelf way to reduce negative transfer in transformer MTL without duplicating network parameters, which would be practically valuable. The empirical work is a clear strength: results are reported across three benchmarks and several backbone sizes, the parameter overheads are consistently small, and the paper includes useful control experiments such as reversing the TM/TE assignment (Table 13), random versus conflict-based layer selection (Table 2), and varying the number of expanded layers (Fig. 6). The code is made available. However, the theoretical support is not rigorous, and the implementation described in the supplementary material does not match the dynamic procedure in Algorithm 1, so the central claim that performance gains come specifically from resolving range- versus null-space conflicts is not yet established. The contribution is therefore conditional: the empirical gains appear credible, but the mechanism and the supporting theory need substantial clarification and additional validation.
major comments (4)
- [Sec. 4.1, Eq. (3) and Fig. 2] There is a dimensional inconsistency in the definition of the range/null boundary. Eq. (3) defines r as the ratio of null-to-range eigenvalue sums, which is dimensionless, while Fig. 2 and the surrounding text say to select m such that the cumulative sum up to lambda_m is 'smaller than r' and the sum up to lambda_{m+1} is 'larger than r'; this compares a ratio with dimensioned cumulative sums. With the reported r=100, the two readings select very different subspaces, so the conflict taxonomy is under-specified. The paper must state precisely which comparison is implemented and report the resulting range/null dimensions for each dataset and layer.
- [Supple B.3 and Algorithm 1] The implementation described in Supple B.3 does not match the dynamic procedure in Algorithm 1. The SVD is computed only once early in training, the six task-specific tokens are statically allocated and 'fixed during training', and a manually chosen activation ratio decides where TM and TE are applied. Under this implementation the claimed 'adaptive, conflict-type-based' mechanism reduces to a single early snapshot plus manual layer selection. The paper should quantify how much the token covariance changes during fine-tuning (e.g., by measuring principal angles or subspace distances between early and late SVD bases) and should show whether the reported gains survive when the decomposition is refreshed periodically. Without this, the improvements in Tables 1-5 could be attributed to added capacity in hand-selected layers rather than to the range/null conflict decomposition.
- [Supple E.1, Proposition 1] The proof of Proposition 1 is circular as written: it assumes 'By optimizing the modulator M_i^t so that |nabla_{M_i^t} L_i| approaches zero for each task', which is essentially the reduction in task-gradient conflict that the proposition is supposed to establish. The proof also models M_i as a p-by-p matrix, while Section 4.3 defines the modulator as W elementwise-multiplied with T_{s,i} plus b, with W and b in R^p, so the analyzed object does not match the implemented mechanism. The Taylor expansion omits first-order terms in the modulator update and does not justify why optimizing the modulators reduces the gradient-conflict term in Eq. (7). This step needs a rigorous derivation, or the theoretical claim should be downgraded to a heuristic motivation.
- [Supple E.2, Proposition 2] The proof of Proposition 2 is not rigorous. Eq. (21) drops cross terms without comment, and the conclusion that the null-space term 'cannot be reduced' is asserted rather than derived, especially because shared tokens are treated as learnable parameters elsewhere in the paper (Section 4.2 states 'We treat T_s as network parameters'). In addition, the matrix blocks in Eq. (14) have unclear dimensions for K task tokens, and the projection U_{R,i} is never defined. Please either provide a complete proof or explicitly present Propositions 1 and 2 as empirical design rationales rather than as formal guarantees.
minor comments (6)
- [Tables 1 and 2] Table 1 reports the TM+TE saliency score on PASCAL-Context as 83.41, while Table 2 reports 83.21 for the same configuration; please correct this inconsistency.
- [Throughout] The paper switches between 'range space' and 'row space' (e.g., Proposition 1 says 'row space' while Section 4.2 defines 'range space'). Please unify the terminology, since the covariance matrix here is symmetric and the two notions coincide, but the inconsistent usage is confusing.
- [Main text, Sec. 5.2] The phrase 'As as shown in Tab. 7' contains a typo and should read 'As shown in Tab. 7'.
- [Table 3 and Table 10] Several methods (e.g., MGDA in Table 3 and Align-MTL in Table 10) are marked with dashes. The text should state explicitly whether these runs failed to converge or were omitted, because the comparison is otherwise incomplete.
- [Sec. 4.1] The term 'null space' is used for eigenvectors with small but nonzero eigenvalues. It would be clearer to state explicitly that this is a low-variance subspace approximation rather than the exact null space of the uncentered covariance matrix.
- [Supple E.1, Eq. (7)] Eq. (7) is dimensionally unclear: the expression (U_R M_i^t)(U_R M_i^t)^T requires a definition of M_i^t and of how it acts on the token, and the chain-rule factor nabla_{T_in} M_i^t is not defined. Please clarify the notation.
Circularity Check
Empirical gains are benchmark-validated and self-contained, but the theoretical support for the range/null remedy assignment is circular: Proposition 1's proof assumes the modulator reaches a zero-gradient (zero-conflict) endpoint and reads off 'conflicts alleviated,' while Proposition 2's proof is generic to any added parameter.
-
other
[Supplementary E.1, Proof of Proposition 1; invoked at Sec. 4.3 'Token Modulation']
"By optimizing the modulator Mt_i so that |∇Mt_i Li(Θt_s, Mt_i, Tin)| approaches zero for each task i = 1, 2, . . . ,K, we can alleviate gradient conflicts in the row space of ˜Ts (as Eq. (7) also approaches zero) and reduce the overall multi-task loss, since Eq. (11) is always greater than or equal to zero."
The proposition is presented as the demonstration that modulation resolves range-space conflicts, but the proof derives 'conflicts alleviated' directly from the assumed endpoint |∇Mt_i Li| → 0 for every task. At that joint stationary point the token gradients in Eq. (7) vanish for all tasks, so the pairwise dot products of Definition 1 are trivially at their minimum, and the loss decrease comes solely from the generic term −η Σ|∇Mt_i Li|² (Eq. 11), which is non-negative for any learnable parameter of any form. The argument never uses the modulator's affine structure or the range-space projection, so the same equations would 'prove' the same conclusion for arbitrary added parameters.
-
other
[Supplementary E.2, Proof of Proposition 2; invoked at Sec. 4.3 'Token Expansion']
"The increase in multi-task loss caused by gradient conflicts in the null space (as described in Eq. (22)) cannot be reduced since the shared token T t in is not a learnable parameter. Instead, task-specific tokens T t i can be added to mitigate the increase in multi-task loss due to null space gradient conflicts by optimizing the learnable parameters{Ti}K i=1 as described in Eq. (23)."
The proof explicitly declines to reduce the null-space conflict term (Eq. 22) and instead offsets it with the standard, always-negative gradient-descent decrease −η(UR,iU^T R,i)∇Ti Li · (UR,iU^T R,i)∇Ti Li (Eq. 23), a term produced by ANY newly added learnable parameter in any subspace, regardless of token structure. Nothing in the argument uses the fact that the added parameters are tokens or that the conflict lies in the null space of the token covariance, so the proposition cannot support the null-conflict-to-token-expansion mapping for which Sec. 4.3 invokes it.
full rationale
The empirical core of the paper is self-contained: Tables 1–5 and 8–10 compare DTME-MTL directly against baselines, MTL optimizers (GradDrop, PCGrad, CAGrad, IMTL, Nash-MTL, Aligned-MTL, FAMO), and Recon across NYUD-v2, PASCAL-Context, and Taskonomy with ViT-T/S/B/L, and these gains do not reduce to any fitted parameter or to the propositions. The TM/TE assignment also has independent empirical support from the layer-placement ablation (Tab. 2), the conflict-swap test (Tab. 13), and the conflict-count measurements (Tab. 14). The circularity is in the theoretical framing: Sec. 4.3 presents Propositions 1–2 as demonstrations of the range-conflict-to-modulation and null-conflict-to-expansion mapping, but the proofs reduce to generic gradient-descent statements. Proposition 1's proof reaches its conclusion by assuming the modulator reaches a joint zero-gradient state, the endpoint at which the conflict-defining quantities (Eq. 7) trivially vanish, so the conclusion is assumed by construction, and the loss-decrease term (Eq. 11) is the standard descent term of any parameter. Proposition 2's proof explicitly leaves the null-space conflict term (Eq. 22) unreduced and offsets it with the generic negative step (Eq. 23) that any added learnable parameter would produce, so the mechanism-specific claim is not derived. Because the headline performance claims stand on empirical footing, this is partial circularity rather than total, hence 4 rather than 6+. No load-bearing self-citation was found: refs [29–30] are background, and [46] is an external, plausible basis for the range/null treatment. Separately, for correctness risk rather than circularity: Eq. (3) defines r as a ratio of eigenvalue sums while Fig. 2 compares r to raw cumulative sums (dimensionally inconsistent), and Supple B.3 fixes the SVD from one early snapshot with statically allocated tokens and a manually chosen activation ratio, so the 'dynamic/adaptive' mechanism is a single snapshot plus manual selection; these concerns affect whether the gains are attributable to the stated conflict taxonomy, not whether the empirical claims reduce to their inputs.
Assumptions & free parameters
free parameters (5)
- variance ratio r =
100
- number of task-specific tokens =
6
- activation ratio for TM/TE assignment =
not disclosed (manually chosen)
- expansion layer ratio =
25-50% of layers
- SVD timing =
once early in training (iteration not specified)
assumptions (5)
- domain assumption The token space of the pretrained transformer remains in the same loss basin during fine-tuning, so the range/null space decomposition computed once early in training stays representative.
- standard math Gradients with respect to token activations can be projected onto the range and null spaces of the uncentered token covariance and used to classify conflicts.
- ad hoc to paper Range-space conflicts are resolvable by affine modulation and null-space conflicts require token expansion.
- ad hoc to paper The variance ratio r separates range and null spaces meaningfully.
- domain assumption Adding task-specific tokens and modulators does not break the pretrained backbone's feature space.
Cite this review
Pith. "Pith review of Resolving Token-Space Gradient Conflicts: Token Space Manipulation for Transformer-Based Multi-Task Learning." pith.science (2026). https://pith.science/paper/OUIENIMN
@misc{pith2026250707485,
author = {Pith},
title = {Pith review of: Resolving Token-Space Gradient Conflicts: Token Space Manipulation for Transformer-Based Multi-Task Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/OUIENIMN}},
note = {Machine review of arXiv:2507.07485}
}
read the original abstract
Multi-Task Learning (MTL) enables multiple tasks to be learned within a shared network, but differences in objectives across tasks can cause negative transfer, where the learning of one task degrades another task's performance. While pre-trained transformers significantly improve MTL performance, their fixed network capacity and rigid structure limit adaptability. Previous dynamic network architectures attempt to address this but are inefficient as they directly convert shared parameters into task-specific ones. We propose Dynamic Token Modulation and Expansion (DTME-MTL), a framework applicable to any transformer-based MTL architecture. DTME-MTL enhances adaptability and reduces overfitting by identifying gradient conflicts in token space and applying adaptive solutions based on conflict type. Unlike prior methods that mitigate negative transfer by duplicating network parameters, DTME-MTL operates entirely in token space, enabling efficient adaptation without excessive parameter growth. Extensive experiments demonstrate that DTME-MTL consistently improves multi-task performance with minimal computational overhead, offering a scalable and effective solution for enhancing transformer-based MTL models.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Mult: an end-to-end multitask learning transformer
Deblina Bhattacharjee, Tong Zhang, Sabine S ¨usstrunk, and Mathieu Salzmann. Mult: an end-to-end multitask learning transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12031– 12041, 2022. 2
work page 2022
-
[2]
Felix JS Bragman, Ryutaro Tanno, Sebastien Ourselin, Daniel C Alexander, and Jorge Cardoso. Stochastic filter groups for multi-task cnns: Learning specialist and gener- alist convolution kernels. In Proceedings of the IEEE/CVF international conference on computer vision , pages 1385– 1394, 2019. 2
work page 2019
-
[4]
Automated search for resource- efficient branched multi-task networks
David Bruggemann, Menelaos Kanakis, Stamatios Geor- goulis, and Luc Van Gool. Automated search for resource- efficient branched multi-task networks. arXiv preprint arXiv:2008.10292, 2020. 12
arXiv 2008
-
[5]
Multi-task learning for multi-objective evolutionary neural architecture search
Ronghong Cai and Jianping Luo. Multi-task learning for multi-objective evolutionary neural architecture search. In 2021 IEEE Congress on Evolutionary Computation (CEC) , pages 1680–1687. IEEE, 2021. 2
work page 2021
-
[6]
Rich Caruana. Multitask learning. Machine learning, 28: 41–75, 1997. 1
work page 1997
-
[7]
Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks
Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and An- drew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In In- ternational conference on machine learning, pages 794–803. PMLR, 2018. 3
work page 2018
-
[8]
Just pick a sign: Optimizing deep multitask models with gra- dient sign dropout
Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gra- dient sign dropout. Advances in Neural Information Process- ing Systems, 33:2039–2050, 2020. 3, 6, 8
work page 2020
-
[9]
Mod-squad: Designing mixtures of experts as modular multi-task learners
Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik G Learned-Miller, and Chuang Gan. Mod-squad: Designing mixtures of experts as modular multi-task learners. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 11828–11837, 2023. 1, 2
work page 2023
Show all 72 references
-
[10]
Multi-task learning with deep neural networks: A survey
Michael Crawshaw. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796, 2020. 1
2009 arXiv
-
[11]
Instance-aware se- mantic segmentation via multi-task network cascades
Jifeng Dai, Kaiming He, and Jian Sun. Instance-aware se- mantic segmentation via multi-task network cascades. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3150–3158, 2016. 1, 12
2016
-
[12]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 12
2009
-
[13]
Multiple-gradient descent algorithm (mgda) for multiobjective optimization
Jean-Antoine D ´esid´eri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350(5-6):313–318, 2012. 2, 3
2012
-
[14]
An image is worth 16x16 words: Trans- formers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...
2010 arXiv
-
[15]
Representation similar- ity analysis for efficient task taxonomy & transfer learning
Kshitij Dwivedi and Gemma Roig. Representation similar- ity analysis for efficient task taxonomy & transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12387–12396, 2019. 1
2019
-
[16]
Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture
David Eigen and Rob Fergus. Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture. In Proceedings of the IEEE inter- national conference on computer vision , pages 2650–2658,
-
[17]
The pascal visual object classes challenge 2012 (voc2012) development kit
Mark Everingham and John Winn. The pascal visual object classes challenge 2012 (voc2012) development kit. Pattern Anal. Stat. Model. Comput. Learn., Tech. Rep , 2007:1–45,
2012
-
[18]
M 3vit: Mixture-of-experts vision transformer for efficient multi- task learning with model-accelerator co-design
Zhiwen Fan, Rishov Sarkar, Ziyu Jiang, Tianlong Chen, Kai Zou, Yu Cheng, Cong Hao, Zhangyang Wang, et al. M 3vit: Mixture-of-experts vision transformer for efficient multi- task learning with model-accelerator co-design. Advances in Neural Information Processing Systems, 35:28...
-
[19]
Pathnet: Evolution channels gradient descent in super neural networks
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734,
-
[20]
Nddr-cnn: Layerwise feature fusing in multi-task cnns by neural discriminative dimensionality reduction
Yuan Gao, Jiayi Ma, Mingbo Zhao, Wei Liu, and Alan L Yuille. Nddr-cnn: Layerwise feature fusing in multi-task cnns by neural discriminative dimensionality reduction. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 3205–3214, 2019. 12
2019
-
[21]
Mtl-nas: Task-agnostic neural architecture search towards general-purpose multi-task learning
Yuan Gao, Haoping Bai, Zequn Jie, Jiayi Ma, Kui Jia, and Wei Liu. Mtl-nas: Task-agnostic neural architecture search towards general-purpose multi-task learning. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 11543–11552, 2020. 2
2020
-
[22]
Recon: Reducing conflicting gradients from the root for multi-task learning
SHI Guangyuan, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao-Ming Wu. Recon: Reducing conflicting gradients from the root for multi-task learning. In The Eleventh Inter- national Conference on Learning Representations, 2022. 2, 3, 6, 7, 8
2022
-
[23]
Dynamic task prioritization for multitask learning
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Li Fei-Fei. Dynamic task prioritization for multitask learning. In Proceedings of the European conference on com- puter vision (ECCV), pages 270–287, 2018. 2
2018
-
[24]
Learn- ing to branch for multi-task learning
Pengsheng Guo, Chen-Yu Lee, and Daniel Ulbricht. Learn- ing to branch for multi-task learning. InInternational confer- ence on machine learning, pages 3854–3863. PMLR, 2020. 2
2020
-
[25]
Learn- ing to branch for multi-task learning
Pengsheng Guo, Chen-Yu Lee, and Daniel Ulbricht. Learn- ing to branch for multi-task learning. In International Con- ference on Machine Learning , pages 3854–3863. PMLR,
-
[26]
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 15
2022
-
[27]
Going beyond multi-task dense prediction with synergy em- bedding models
Huimin Huang, Yawen Huang, Lanfen Lin, Ruofeng Tong, Yen-Wei Chen, Hao Zheng, Yuexiang Li, and Yefeng Zheng. Going beyond multi-task dense prediction with synergy em- bedding models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages...
2024
-
[28]
Rotograd: Gradi- ent homogenization in multitask learning
Adri ´an Javaloy and Isabel Valera. Rotograd: Gradi- ent homogenization in multitask learning. arXiv preprint arXiv:2103.02631, 2021. 3
2021 arXiv
-
[29]
Quantifying task pri- ority for multi-task optimization
Wooseong Jeong and Kuk-Jin Yoon. Quantifying task pri- ority for multi-task optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 363–372, 2024. 3
2024
-
[30]
Selective task group updates for multi-task optimization
Wooseong Jeong and Kuk-Jin Yoon. Selective task group updates for multi-task optimization. arXiv preprint arXiv:2502.11986, 2025. 3
2025 arXiv
-
[31]
Forkmerge: Mitigating negative transfer in auxiliary-task learning
Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Dapeng Liu, Jie Jiang, and Mingsheng Long. Forkmerge: Mitigating negative transfer in auxiliary-task learning. Ad- vances in Neural Information Processing Systems, 36, 2024. 2, 3
2024
-
[32]
Principal component anal- ysis: A review and recent developments
Ian T Jollife and Jorge Cadima. Principal component anal- ysis: A review and recent developments. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci, 374(2065):20150202, 2016. 4
2016
-
[33]
Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482–7491,
-
[34]
Evo- lutionary architecture search for deep multitask networks
Jason Liang, Elliot Meyerson, and Risto Miikkulainen. Evo- lutionary architecture search for deep multitask networks. In Proceedings of the genetic and evolutionary computation conference, pages 466–473, 2018. 2
2018
-
[35]
Conflict-averse gradient descent for multi-task learn- ing
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learn- ing. Advances in Neural Information Processing Systems , 34:18878–18890, 2021. 2, 3, 6, 8
2021
-
[36]
Famo: Fast adaptive multitask optimization
Bo Liu, Yihao Feng, Peter Stone, and Qiang Liu. Famo: Fast adaptive multitask optimization. Advances in Neural Information Processing Systems, 36, 2024. 2, 6, 8
2024
-
[37]
Towards impartial multi-task learning
Liyang Liu, Yi Li, Zhanghui Kuang, J Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. Towards impartial multi-task learning. iclr, 2021. 2, 3, 6, 8
2021
-
[38]
End- to-end multi-task learning with attention
Shikun Liu, Edward Johns, and Andrew J Davison. End- to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1871–1880, 2019. 2, 6, 8
2019
-
[39]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 2
2021
-
[40]
Fully-adaptive feature shar- ing in multi-task networks with applications in person at- tribute classification
Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris. Fully-adaptive feature shar- ing in multi-task networks with applications in person at- tribute classification. In Proceedings of the IEEE conference on computer vision and pattern recognition...
2017
-
[41]
Modeling task relationships in multi-task learning with multi-gate mixture-of-experts
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceed- ings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining , pages 1930–1939,
1930
-
[42]
Attentive single-tasking of multiple tasks
Kevis-Kokitsi Maninis, Ilija Radosavovic, and Iasonas Kokkinos. Attentive single-tasking of multiple tasks. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1851–1860, 2019. 6, 12
2019
-
[43]
The role of context for object detection and semantic segmentation in the wild
Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. The role of context for object detection and semantic segmentation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recogni...
2014
-
[44]
Multimodal contrastive learn- ing with limoe: the language-image mixture of experts
Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, and Neil Houlsby. Multimodal contrastive learn- ing with limoe: the language-image mixture of experts. Ad- vances in Neural Information Processing Systems, 35:9564– 9576, 2022. 1, 2
2022
-
[45]
Multi- task learning as a bargaining game
Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi- task learning as a bargaining game. arXiv preprint arXiv:2202.01017, 2022. 2, 3, 6, 8
2022 arXiv
-
[46]
What is being transferred in transfer learning? Advances in neural information processing systems , 33:512–523, 2020
Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. What is being transferred in transfer learning? Advances in neural information processing systems , 33:512–523, 2020. 5, 15
2020
-
[47]
Continual and multi-task architecture search
Ramakanth Pasunuru and Mohit Bansal. Continual and multi-task architecture search. arXiv preprint arXiv:1906.05226, 2019. 2
1906 arXiv
-
[48]
Scaling vision with sparse mix- ture of experts
Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, Andr ´e Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mix- ture of experts. Advances in Neural Information Processing Systems, 34:8583–8595, 2021. 1, 2
2021
-
[49]
Multi-task learning as multi-objective optimization
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Advances in neural informa- tion processing systems, 31, 2018. 2, 3, 6, 8
2018
-
[50]
Independent component alignment for multi-task learning
Dmitry Senushkin, Nikolay Patakin, Arseny Kuznetsov, and Anton Konushin. Independent component alignment for multi-task learning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 20083–20093, 2023. 2, 3, 6, 8
2023
-
[51]
Indoor segmentation and support inference from rgbd images
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision–ECCV 2012: 12th Eu- ropean Conference on Computer Vision, Florence, Italy, Oc- tober 7-13, 2012, Proceedings, Part V 12 , pages 746–760...
2012
-
[52]
One-shot neural architecture search for deep multi-task learning in computer vision, 2020
Gianluigi Silvestri. One-shot neural architecture search for deep multi-task learning in computer vision, 2020. 2
2020
-
[53]
Very deep convo- lutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 1, 12
2014 arXiv
-
[54]
Gradient adversarial training of neural networks
Ayan Sinha, Zhao Chen, Vijay Badrinarayanan, and An- drew Rabinovich. Gradient adversarial training of neural networks. 2018. 12
2018
-
[55]
Task switching network for multi-task learn- ing
Guolei Sun, Thomas Probst, Danda Pani Paudel, Nikola Popovi´c, Menelaos Kanakis, Jagruti Patel, Dengxin Dai, and Luc Van Gool. Task switching network for multi-task learn- ing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 8291–8300, 2021. 12
2021
-
[56]
Branched multi-task net- works: deciding what layers to share
Simon Vandenhende, Stamatios Georgoulis, Bert De Bra- bandere, and Luc Van Gool. Branched multi-task net- works: deciding what layers to share. arXiv preprint arXiv:1904.02920, 2019. 12
1904 arXiv
-
[57]
Mti-net: Multi-scale task interaction networks for multi-task learning
Simon Vandenhende, Stamatios Georgoulis, and Luc Van Gool. Mti-net: Multi-scale task interaction networks for multi-task learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pages 527–543. Springer, 2020. 1, 12
2020
-
[58]
Pyramid vision transformer: A versatile backbone for dense prediction without convolutions
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vision , p...
2021
-
[59]
Crossformer: A versatile vision transformer hinging on cross-scale attention
Wenxiao Wang, Lu Yao, Long Chen, Binbin Lin, Deng Cai, Xiaofei He, and Wei Liu. Crossformer: A versatile vision transformer hinging on cross-scale attention. arXiv preprint arXiv:2108.00154, 2021
2021 arXiv
-
[60]
Segformer: Simple and efficient design for semantic segmentation with transform- ers
Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transform- ers. Advances in Neural Information Processing Systems , 34:12077–12090, 2021. 2
2021
-
[61]
Pad-net: Multi-tasks guided prediction-and-distillation net- work for simultaneous depth estimation and scene parsing
Dan Xu, Wanli Ouyang, Xiaogang Wang, and Nicu Sebe. Pad-net: Multi-tasks guided prediction-and-distillation net- work for simultaneous depth estimation and scene parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 675–684, 2018. 1, 12
2018
-
[62]
Mtformer: Multi-task learn- ing via transformer and cross-task reasoning
Xiaogang Xu, Hengshuang Zhao, Vibhav Vineet, Ser-Nam Lim, and Antonio Torralba. Mtformer: Multi-task learn- ing via transformer and cross-task reasoning. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Is- rael, October 23–27, 2022, Proceedings, Part XXVII , ...
2022
-
[63]
Multi-task learning with multi-query trans- former for dense prediction
Yangyang Xu, Xiangtai Li, Haobo Yuan, Yibo Yang, and Lefei Zhang. Multi-task learning with multi-query trans- former for dense prediction. IEEE Transactions on Circuits and Systems for Video Technology, 2023. 1, 2
2023
-
[64]
Demt: De- formable mixer transformer for multi-task learning of dense prediction
Yangyang Xu, Yibo Yang, and Lefei Zhang. Demt: De- formable mixer transformer for multi-task learning of dense prediction. In Proceedings of the AAAI conference on artifi- cial intelligence, pages 3072–3080, 2023. 1, 2
2023
-
[65]
Focal self-attention for local-global interactions in vision transformers
Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, and Jianfeng Gao. Focal self-attention for local-global interactions in vision transformers. arXiv preprint arXiv:2107.00641, 2021. 2
2021 arXiv
-
[66]
Inverted pyramid multi-task trans- former for dense scene understanding
Hanrong Ye and Dan Xu. Inverted pyramid multi-task trans- former for dense scene understanding. In Computer Vision– ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVII, pages 514–
2022
-
[67]
Taskprompter: Spatial-channel multi-task prompting for dense scene understanding
Hanrong Ye and Dan Xu. Taskprompter: Spatial-channel multi-task prompting for dense scene understanding. In The Eleventh International Conference on Learning Representa- tions, 2022. 1, 2, 8, 12
2022
-
[68]
Gradient surgery for multi-task learning
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Pro- cessing Systems, 33:5824–5836, 2020. 2, 3, 6, 8
2020
-
[69]
Taskonomy: Disentangling task transfer learning
Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 3712–3722, 2018. 6
2018
-
[70]
Mixture of attention heads: Selecting attention heads per token
Xiaofeng Zhang, Yikang Shen, Zeyu Huang, Jie Zhou, Wenge Rong, and Zhang Xiong. Mixture of attention heads: Selecting attention heads per token. arXiv preprint arXiv:2210.05144, 2022. 1, 2
2022 arXiv
-
[71]
Facial landmark detection by deep multi-task learning
Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial landmark detection by deep multi-task learning. In Computer Vision–ECCV 2014: 13th European Confer- ence, Zurich, Switzerland, September 6-12, 2014, Proceed- ings, Part VI 13, pages 94–108. Springer, 2014. 1, 12
2014
-
[72]
Pattern-affinitive propagation across depth, surface normal and semantic segmentation
Zhenyu Zhang, Zhen Cui, Chunyan Xu, Yan Yan, Nicu Sebe, and Jian Yang. Pattern-affinitive propagation across depth, surface normal and semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4106–4115, 2019. 1, 12 Reso...
2019
-
[530]
2, 8, 12
Springer, 2022. 2, 8, 12
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.