Pith. sign in

REVIEW 4 major objections 6 minor 39 references

Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces MTIF, a per-training-example influence score for multitask models, and shows that deleting the examples it flags as harmful consistently raises average test accuracy on image benchmarks.

desk verdict MTIF's influence-function derivation for MTL is clean and well-validated on convex problems, but the data-selection claim needs a random-removal baseline. read the letter →

arxiv 2505.21438 v1 pith:SEB6WWBG submitted 2025-05-27 cs.LG

Yiwen Tu , Ziqi Liu , Jiaqi W. Ma , Weijing Tang This is my paper
classification cs.LG
keywords multitasklearningnegativetransfertaskrelatednessdataattributioninfluencefunctionleave-one-outselectioninstance-level
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multitask learning fails when tasks fight each other, and the usual fixes treat each task as one indivisible block. This paper tries to get inside the block: it adapts influence functions, a data-attribution tool that estimates how much each training point matters, to multitask models with hard or soft parameter sharing. The proposed MultiTask Influence Function (MTIF) gives a per-example score for how removing one training point from one task changes validation loss on another task. The paper argues that these scores are accurate first-order approximations of leave-one-out retraining, and that using them to delete the worst-scoring examples reliably improves multitask test accuracy. If true, task relatedness becomes a fine-grained, instance-level quantity instead of a task-to-task aggregate, and negative transfer can be reduced by data curation rather than by changing the optimizer or the architecture.

What carries the argument

The load-bearing object is the block-structured Hessian of the per-sample weighted multitask objective $L(w,\sigma)=\sum_{k}\,[\frac{1}{n_k}\sum_i \sigma_{ki}\ell_{ki}(\theta_k,\gamma)+\Omega_k(\theta_k,\gamma)]$. Because the cross-task blocks $H_{kl}$ are zero, the full Hessian inverse can be written in closed form using the Schur complement $N=H_{K+1,K+1}-\sum_k H_{K+1,k}H_{kk}^{-1}H_{k,K+1}$, which yields the three analytical influence formulas in Proposition 1. MTIF itself is the first-order change in a target task's validation loss when the weight $\sigma_{li}$ of one training point is perturbed away from 1. For deep networks the Hessian inverse is replaced by computational tricks of random projection, ensembling over checkpoints, and sparsification, so that scores stay computable at scale.

What would settle it

On a small strictly convex multitask regression problem, compute exact leave-one-out validation loss changes for every training point in every task and compare them with MTIF scores for the same points; if the rank correlation is near zero or negative for any task pair, the approximation claim is refuted. On a deep multitask benchmark, if deleting the examples with the lowest summed MTIF scores does not outperform deleting an equal-sized random subset, the data-selection claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that MTIF, defined as a first-order Taylor expansion of validation loss with respect to per-sample weights in the multitask objective, accurately recovers leave-one-out effects for single training points across tasks. Proposition 1 splits the influence into within-task, between-task, and shared-parameter components, exploiting the block-diagonal-plus-coupling structure of the multitask Hessian to write its inverse through a Schur complement. The between-task component is purely mediated by the shared parameters, so a sample from one task reaches another task's head only through the common trunk. Empirically, MTIF scores correlate strongly with brute-force leave-one-out retraining on synthetic and HAR data, and the task-level aggregate MTIFtask correlates with leave-one-task-out retraining better than gradient cosine similarity or gradient-based task grouping. On CelebA, Office-31, and Office-Home, removing the training examples with the most negative summed MTIF scores improves average test accuracy over state-of-the-art multitask baselines, with larger gains when 10% or 20% of labels are corrupted.

Load-bearing premise

The derivation of the influence formulas assumes the multitask objective is twice-differentiable and strictly convex, but the large-scale experiments that demonstrate the accuracy gains train non-convex deep models, where the paper relies on heuristic approximations rather than the proven formulas.

Editorial extensions

If this is right

  • Instance-level task relatedness can be estimated without retraining, giving a cheap substitute for leave-one-out experiments in multitask models.
  • Deleting the training examples flagged as most harmful by MTIF mitigates negative transfer and improves average test accuracy on CelebA, Office-31, and Office-Home.
  • Aggregating MTIF over the data of a source task yields a task-level relatedness score that tracks leave-one-task-out retraining more closely than gradient cosine similarity or gradient-based task grouping.
  • Because MTIF acts on the training data rather than on gradients or loss weights, the resulting data-selection step is orthogonal to existing multitask optimization and architecture methods and can be stacked with them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the MTIF approximation is trustworthy, the same per-example scoring can be applied to transfer learning: a source-domain sample that scores negatively against a target validation set is a candidate for removal or downweighting before fine-tuning.
  • Equation (8) makes a sharp testable prediction: between-task influence flows only through shared parameters, so a model variant with no shared trunk should show near-zero MTIF cross-task scores; an experiment that fails to show this would undercut the mechanism.
  • The paper aggregates influence by summing over target tasks before deleting, which assumes a harmful sample hurts every target similarly; a per-target deletion rule is a natural extension that could help when a sample helps one task while hurting another.
  • The paper's own caution that MTIFtask weakens for tasks with many data points suggests a continuum from instance removal to task removal, with fine-grained scores carrying the most signal precisely where leave-one-task-out is least reliable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes the MultiTask Influence Function (MTIF), an influence-function-based method for multitask learning that approximates the leave-one-out (LOO) effect of removing a single training point from one task on the validation loss of another task, thereby providing an instance-level measure of task relatedness. The authors also derive a task-level variant (MTIFtask) that approximates leave-one-task-out (LOTO) effects. They validate the approximation quality against brute-force LOO and LOTO on a synthetic linear-regression dataset and the HAR dataset, where the underlying convexity assumptions hold. They then apply MTIF-guided data selection—removing the training samples with the most negative summed influence—on CelebA, Office-31, and Office-Home with ResNet-18 models, reporting consistent improvements in average test accuracy over several MTL optimization baselines. The paper claims that MTIF efficiently and accurately approximates the performance of models trained on data subsets and that the resulting data-selection strategy mitigates negative transfer.

Significance. If the claims are sustained, the paper makes a useful connection between data attribution and multitask learning, offering a fine-grained, instance-level relatedness measure that is more informative than existing task-level measures and enables a data-curation approach to negative transfer. The analytical block-inverse derivation of the MTL Hessian is a solid contribution, and the LOO/LOTO validation on convex models is a strength that gives some confidence in the method's core approximation. The data-selection experiments span three standard benchmarks and show consistent gains over strong MTL baselines. However, the practical claim is currently underdetermined by the missing random-removal control, and the technical normalization inconsistency and the convexity gap between theory and the deep-network experiments need to be addressed before the results can be fully accepted.

major comments (4)
  1. [§4.2, Table 2] The data selection experiments never compare MTIF-guided removal against removing the same number of uniformly random training points with the same retraining protocol. Because the removal fraction is tuned on a held-out subset, the reported accuracy gains cannot be attributed to the MTIF ranking; a smaller training set or the tuned fraction alone could produce the same effect. Adding a random-removal control (with the same tuned fraction, seeds, and retraining setup) is necessary to support the headline claim that MTIF-guided data selection consistently improves MTL performance.
  2. [§3.3, Eq. (3), Lemma A.2, and Proposition 1] The objective in Eq. (3) includes the 1/n_k normalization in the data term, but the Hessian blocks in Lemma A.1 and the derivative expressions in Lemma A.2 and Proposition 1 omit this factor. Consequently, MTIF(i,l;k) is scaled by 1/n_l for each source task l, which changes the cross-task ranking used in the data selection step of Section 4.2 and also affects task-level relatedness comparisons on HAR, where n_l varies across volunteers. The authors should either add the missing 1/n_k factors throughout the derivation or redefine the objective without the normalization and state that choice explicitly.
  3. [§4.2 and Appendix C.1] Proposition 1 assumes the MTL objective is strictly convex, but the data selection experiments use non-convex ResNet-18 models on CelebA, Office-31, and Office-Home. The TRAK-inspired heuristics in Appendix C.1 are described as computational tricks without evidence that the convex influence formulas remain accurate in the non-convex regime. A small-scale validation of MTIF against brute-force LOO on a deep network, or a principled argument for why the approximation transfers, is needed to support the claim that MTIF guides data selection beyond the convex setting.
  4. [§4.1 vs. §4.2] The approximation quality experiments validate MTIF only against single-point LOO and whole-task LOTO, whereas the data selection method removes a tuned fraction of points, which is neither a single point nor the entire task. There is no validation that the first-order approximation holds for the multi-point removals actually performed, nor a comparison of the predicted subset performance against the actual retrained subset performance. This weakens the link between the demonstrated approximation quality and the data-selection utility results.
minor comments (6)
  1. [§4.2] The removal ratio hyperparameter and the resulting number of removed samples are not reported; please provide these values for all datasets to allow reproducibility.
  2. [Appendix C.3.1] The same validation set is used both to compute MTIF scores and to tune the removal ratio; this should be stated in the main text as a potential limitation, since it could inflate the apparent benefit of MTIF-guided selection.
  3. [Table 2] The paper reports standard errors over five seeds but does not provide significance tests or confidence intervals for the difference between MTIF and the best baseline; additionally, the averaging scheme over tasks should be specified.
  4. [Table 3] The caption contains a typo ('teh' should be 'the'), and the hardware/software environment for the runtime measurements is not specified.
  5. [Appendix C.1] The TRAK integration details—such as the random projection dimension, ensemble size, and sparsification threshold—are not described; please include these details for reproducibility.
  6. [Figure 1] The caption's description of the subplot order is confusing ('first two plots from the left show within-task and between-task results (in order)'), and the axis labels should clearly state the actual loss difference and the MTIF-predicted loss difference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; MTIF is a Taylor approximation validated against brute-force retraining.

full rationale

The paper's central derivation is not circular. MTIF is defined in Eq. (4) as the first-order Taylor coefficient of the target-task validation loss with respect to an infinitesimal perturbation of a source-task training-point weight in the MTL objective (3). This is the standard influence-function construction of Koh and Liang, specialized to the block-structured MTL Hessian; Proposition 1 follows by implicit differentiation of the first-order optimality condition and does not assume the leave-one-out or leave-one-task-out quantities it is intended to approximate. The approximation claims are then tested against brute-force LOO retraining (Eq. (2)) and LOTO retraining (Eq. (9)) on synthetic and HAR data, which are independent ground truths rather than outputs of the MTIF formula. In the data-selection experiments, samples are ranked by summed MTIF scores, a fraction is removed, and the model is retrained; reported accuracies are on held-out test partitions, so the improvement is not forced by the construction. The same validation split is used to compute influence scores and to tune the removal ratio, but this is a hyperparameter-selection choice shared with the baselines and does not make the test-accuracy claim definitionally circular. The one self-citation (Ma et al. 2018, MMoE) appears only as background and architectural related work and is not load-bearing for any derivation or experimental conclusion.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central method rests on standard influence-function machinery plus a convexity assumption that is violated in the main experiments. The only hand-tuned quantities are the removal ratio and TRAK integration hyperparameters, neither of which is reported. No new physical or conceptual entities are introduced.

free parameters (3)
  • removal ratio = not reported
    Fraction of worst-scoring training samples removed in data selection; tuned on a held-out validation subset (Section 4.2).
  • TRAK integration hyperparameters = not reported
    Random projection dimension, ensemble size, and sparsification threshold used to approximate Hessian-vector products in the neural-network experiments (Appendix C.1).
  • regularization strengths lambda_k and c = not reported
    Ridge penalty and weight-decay constants in the MTL objectives (Examples 1 and 2); values are chosen but not stated.
assumptions (3)
  • domain assumption Twice-differentiable, strictly convex MTL objective (Proposition 1)
    The influence formulas rely on convexity and Hessian invertibility; the CelebA, Office-31, and Office-Home experiments use non-convex deep networks where this is not satisfied.
  • domain assumption First-order Taylor approximation of finite LOO and LOTO removals
    The paper approximates removing a data point or a whole task by an infinitesimal weight perturbation; validity for large removals is assumed and only partially flagged as a limitation in Section 5.
  • standard math Hessian invertibility (H_kk and N invertible)
    Lemma A.3 gives sufficient conditions; assumed without verification in the experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution." pith.science (2026). https://pith.science/paper/SEB6WWBG

@misc{pith2026250521438,
  author       = {Pith},
  title        = {Pith review of: Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEB6WWBG}},
  note         = {Machine review of arXiv:2505.21438}
}
read the original abstract

Measuring task relatedness and mitigating negative transfer remain a critical open challenge in Multitask Learning (MTL). This work extends data attribution -- which quantifies the influence of individual training data points on model predictions -- to MTL setting for measuring task relatedness. We propose the MultiTask Influence Function (MTIF), a method that adapts influence functions to MTL models with hard or soft parameter sharing. Compared to conventional task relatedness measurements, MTIF provides a fine-grained, instance-level relatedness measure beyond the entire-task level. This fine-grained relatedness measure enables a data selection strategy to effectively mitigate negative transfer in MTL. Through extensive experiments, we demonstrate that the proposed MTIF efficiently and accurately approximates the performance of models trained on data subsets. Moreover, the data selection strategy enabled by MTIF consistently improves model performance in MTL. Our work establishes a novel connection between data attribution and MTL, offering an efficient and fine-grained solution for measuring task relatedness and enhancing MTL models.

Figures

Figures reproduced from arXiv: 2505.21438 by the authors.

Figure 1
Figure 1. Instance-level MTIF approximation quality on the synthetic and HAR datasets. The x-axis is the actual loss [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. LOO experiments on linear regression. The x-axis is the actual loss difference obtained by LOO retraining, [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. LOO experiments on linear regression. The x-axis is the actual loss difference obtained by LOO retraining, [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: LOO experiments on linear regression. The x-axis is the actual loss difference obtained by LOO retraining, [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: LOO experiments on linear regression. The x-axis is the actual loss difference obtained by LOO retraining, [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 20 canonical work pages

  1. [1]

    nlX i=1 ∂ℓli ∂θl + ∂Ωl ∂θl # − N −1

    Proposition 2 (Task-Level Between-task Influence). Under the assumptions of Proposition 1, for any two tasks k ̸= l where 1 ≤ k, l≤ K, the influence of data from task l on the task-specific parameters of task k, ˆθk, is given by ∂ ˆθk ∂σl = −H −1 kk Hk,K+1 · ∂ˆγ ∂σl , (B.4) where ∂ ˆγ ∂σl is the influence of data from task l on the shared parameters, ˆγ, ...

    work page 2023

  2. [6]

    URL https://doi.org/10.1214/23-AOS2319

    doi: 10.1214/23-AOS2319. URL https://doi.org/10.1214/23-AOS2319. Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. Low resource dependency parsing: Cross-lingual parameter sharing in a neural network parser. In Chengqing Zong and Michael Strube, editors, Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th...

    work page doi:10.1214/23-aos2319

  3. [9]

    Amirata Ghorbani and James Zou

    URL https://proceedings.neurips.cc/paper_files/paper/2021/file/ e77910ebb93b511588557806310f78f1-Paper.pdf. Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings o...

    work page 2021

  4. [10]

    Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong

    URL https://arxiv.org/abs/2308.03296. Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. FastIF: Scalable influence functions for efficient model interpretation and debugging. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Proceedings of the 2021 Conference on Empirical Methods in Natural Language ...

    arXiv 2021

  5. [11]

    doi: 10.18653/v1/2021.emnlp-main.808

    Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.808. URL https://aclanthology.org/2021.emnlp-main

    doi:10.18653/v1/2021.emnlp-main.808 2021

  6. [14]

    Rapid adaptation for deep neural networks through multi-task learning

    Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Ji Wu, and Chin-Hui Lee. Rapid adaptation for deep neural networks through multi-task learning. InInterspeech 2015, pages 3625–3629,

    work page 2015

  7. [15]

    2015-719

    doi: 10.21437/Interspeech. 2015-719. Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, and Aleksander Madry. Datamodels: Understanding predictions with data and data with predictions. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Ma...

    doi:10.21437/interspeech 2015

  8. [16]

    10 Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution A PREPRINT Pang Wei Koh and Percy Liang

    URL https://arxiv.org/ abs/1412.6980. 10 Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution A PREPRINT Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceeding...

    arXiv

Show all 39 references
  1. [17]

    Baijiong Lin and Yu Zhang

    URL https://arxiv.org/abs/2410.12774. Baijiong Lin and Yu Zhang. Libmtl: A python library for deep multi-task learning. The Journal of Machine Learning Research, 24(1):9999–10005,

    arXiv

  2. [18]

    Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang, Fei Liu, Zhenkun Wang, and Qingfu Zhang

    URL https://arxiv.org/abs/2308.12029. Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang, Fei Liu, Zhenkun Wang, and Qingfu Zhang. Smooth tchebycheff scalarization for multi-objective optimization. arXiv preprint arXiv:2402.19078,

  3. [19]

    Conflict-averse gradient descent for multi-task learning

    Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, page...

    2021

  4. [21]

    doi: https://doi.org/10.1016/j.eswa.2024.123739

    ISSN 0957-4174. doi: https://doi.org/10.1016/j.eswa.2024.123739. URL https://www.sciencedirect.com/science/article/pii/ S0957417424006055. Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H. Chi. Modeling task relationships in multi-task learning with multi-gate...

    2024

  5. [22]

    ISBN 9781450355520

    Association for Computing Machinery. ISBN 9781450355520. doi: 10.1145/3219819.3220007. URL https://doi.org/10. 1145/3219819.3220007. Ze Meng, Xin Yao, and Lifeng Sun. Multi-task distillation: Towards mitigating the negative transfer in multi- task learning. In 2021 IEEE Intern...

  6. [23]

    11 Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution A PREPRINT Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry

    doi: 10.1109/ICIP42928.2021.9506618. 11 Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution A PREPRINT Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. TRAK: Attributing model behavior at scale. In Andreas Kraus...

    2021

  7. [24]

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan

    doi: 10.1109/TSMC.2023.3315541. Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Proces...

    2023

  8. [25]

    Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell

    URL https://arxiv.org/abs/ 1706.05098. Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Kostas Daniilidis, Petros Maragos, and Nikos Paragios, editors, Computer Vision – ECCV 2010, pages 213–226, Berlin, Heidelberg,

    2010 arXiv

  9. [27]

    Jiachen T

    URL https://arxiv.org/abs/2402.15328. Jiachen T. Wang and Ruoxi Jia. Data banzhaf: A robust data valuation framework for machine learning. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent, editors, Proceedings of The 26th International Conference on Artificial Intel...

    arXiv

  10. [28]

    doi: https://doi.org/10.1016/j.patrec.2020.05.031

    ISSN 0167-8655. doi: https://doi.org/10.1016/j.patrec.2020.05.031. URL https://www.sciencedirect.com/science/article/pii/S0167865520302087. Ya Xue, Xuejun Liao, Lawrence Carin, and Balaji Krishnapuram. Multi-task learning for classification with dirichlet process priors. Journ...

    2020 doi

  11. [29]

    doi: 12 Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution A PREPRINT https://doi.org/10.1016/j.knosys.2020.106132

    ISSN 0950-7051. doi: 12 Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution A PREPRINT https://doi.org/10.1016/j.knosys.2020.106132. URL https://www.sciencedirect.com/science/article/ pii/S0950705120303877. Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Se...

    2020

  12. [30]

    URL https://proceedings.neurips.cc/paper_files/paper/2020/file/ 3fe78a8acf5fda99de95303940a2420c-Paper.pdf. Amir R. Zamir, Alexander Sax, William Shen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of t...

    2020

  13. [31]

    A survey on negative transfer

    Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu. A survey on negative transfer. IEEE/CAA Journal of Automatica Sinica, 10(2):305–329, 2023a. doi: 10.1109/JAS.2022.106004. Yu Zhang and Dit-Yan Yeung. A convex formulation for learning task relationships in multi-task learning...

    2022

  14. [33]

    2020.3004555

    doi: 10.1109/JPROC. 2020.3004555. 13 Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution A PREPRINT A Lemmas and Proofs The first lemma describe the structure of the Hessian matrices for instance-level inference. Lemma A.1 (Hessian Matrix Structure fo...

    2020

  15. [34]

    nkX i=1 ∂2ℓki(θk, γ) ∂θk∂θ ⊤ k + ∂2Ωk(θk, γ) ∂θk∂θ ⊤ k # ∈ Rdk×dk for 1 ≤ k ≤ K, Hkl = 0 ∈ Rdk×dl for 1 ≤ k, l≤ K and k ̸= l, H ⊤ K+1,k = Hk,K+1 = σk

    Moreover, we have: ∂ ˆw(σ) ∂σki = −H( ˆw(σ), σ)−1  0, · · ·, 0, ∂ℓki ∂θ ⊤ k k-th block , 0, · · ·, 0, ∂ℓki ∂γ ⊤ (K+1)-th block   ⊤ , where H(w, σ) ∈ R(PK k=1 dk+p)×(PK k=1 dk+p) is the Hessian matrix of L(w, σ) with respect to w. Proof. The result is obtained by applying...

    2017

  16. [35]

    Furthermore, we have: ∂ ˆw(σ) ∂σk = −H( ˆw(σ), σ)−1  0, · · ·, 0, nkX i=1 ∂ℓki ∂θk + ∂Ωk ∂θk k-th block , 0, · · ·, 0, nkX i=1 ∂ℓki ∂γ + ∂Ωk ∂γ (K+1)-th block   ⊤ , where H(w, σ) ∈ R(PK k=1 dk+p)×(PK k=1 dk+p) is the Hessian matrix of L(w, σ) with respect to w. Proof. ...

    2017

  17. [37]

    is independent noise. The coefficient vectors θ⋆ j for task j are generated by starting with a common vector β⋆ = 2e1 (where e1 is a unit vector) and adding random perturbations δj, sampled from a sphere with norm δ. For a fraction αm of the tasks, θ⋆ j 16 Measuring Fine-Grain...

    2023

  18. [39]

    Task 1 Task 2 Task 3 Task 4 Task 5 0.67 ± 0.08 0.52 ± 0.10 0.56 ± 0.09 0.64 ± 0.06 0.54 ± 0.15 Task 6 Task 7 Task 8 Task 9 Task 10 0.42 ± 0.16 0.52 ± 0.08 0.65 ± 0.05 0.56 ± 0.04 0.38 ± 0.12 Table 10: The average Spearman correlation coefficients over 5 random seeds on the syn...

    2010

  19. [409]

    Each sample corresponds to one of six activities: walking, walking upstairs, walking downstairs, sitting, standing, or lying. The feature vector for each sample is 561-dimensional, capturing information from both the time and frequency domains, and are reduced to 100 dimension...

    2023

  20. [1997]

    URL https://doi.org/10.1023/A:1007379606734

    doi: 10.1023/A:1007379606734. URL https://doi.org/10.1023/A:1007379606734. Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In Jennifer Dy and Andreas Krause, editors, ...

    doi

  21. [2004]

    ISBN 1581138881

    Association for Computing Machinery. ISBN 1581138881. doi: 10.1145/1014052.1014067. URL https://doi.org/10.1145/1014052.1014067. Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. In M. Ra...

  22. [2010]

    ISBN 9780974903965

    AUAI Press. ISBN 9780974903965. Zhihan Zhang, Wenhao Yu, Mengxia Yu, Zhichun Guo, and Meng Jiang. A survey of multi-task learning in natural language processing: Regarding task relatedness and training methods. In Andreas Vlachos and Isabelle Augenstein, editors, Proceedings o...

    arXiv

  23. [2015]

    doi: 10.3115/v1/P15-2139

    Association for Computational Linguistics. doi: 10.3115/v1/P15-2139. URL https://aclanthology.org/P15-2139. Kshitij Dwivedi and Gemma Roig. Representation similarity analysis for efficient task taxonomy & transfer learning. In Proceedings of the IEEE/CVF Conference on Computer...

    doi

  24. [2017]

    doi: 10.18653/v1/D17-1206

    Association for Computational Linguistics. doi: 10.18653/v1/D17-1206. URL https: //aclanthology.org/D17-1206. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and patter...

    doi

  25. [2018]

    Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov

    URL https://proceedings.mlr.press/v80/chen18a.html. Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. In H. Larochelle, M. Ranzato, R. Hads...

  26. [2019]

    Deep learning in sheet metal bending with a novel theory-guided deep neural network

    Shiming Liu, Yifan Xia, Zhusheng Shi, Hui Yu, Zhiqiang Li, and Jianguo Lin. Deep learning in sheet metal bending with a novel theory-guided deep neural network. IEEE/CAA Journal of Automatica Sinica, 8(3):565–581, 2021c. doi: 10.1109/JAS.2021.1003871. Ziwei Liu, Ping Luo, Xiao...

  27. [2020]

    Adaptive and robust multi-task learning

    Yaqi Duan and Kaizheng Wang. Adaptive and robust multi-task learning. The Annals of Statistics, 51(5):2015 – 2039,

    2015

  28. [2021]

    doi: 10.1093/imaiai/iaaa033

    ISSN 2049-8772. doi: 10.1093/imaiai/iaaa033. URL https://doi.org/10.1093/imaiai/iaaa033. Naman Agarwal, Brian Bullins, and Elad Hazan. Second-order stochastic optimization for machine learning in linear time. Journal of Machine Learning Research, 18(116):1–40,

    doi

  29. [2022]

    Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese

    URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ f50f282a3093d36471008b045bd478af-Paper-Conference.pdf . Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? ...

    2022

  30. [2023]

    Elnaz Barshan, Marc-Etienne Brunet, and Gintare Karolina Dziugaite

    URL https://arxiv.org/abs/2301.02873. Elnaz Barshan, Marc-Etienne Brunet, and Gintare Karolina Dziugaite. Relatif: Identifying explanatory training samples via relative influence. In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Co...

    1909 arXiv

  31. [2024]

    A joint many-task model: Growing a neural network for multiple NLP tasks

    Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. A joint many-task model: Growing a neural network for multiple NLP tasks. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors, Proceedings of the 2017 Conference on Empirical Methods in Natural L...

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.