Pith. sign in

REVIEW 4 major objections 4 minor 110 references

Understanding Knowledge Transferability for Transfer Learning: A Survey

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Transferability metrics can be unified by knowledge type and measurement granularity, and a metric's assumptions should drive selection across learning paradigms.

desk verdict A useful map of transferability metrics, but the theoretical grounding is too sloppy to trust as written; needs major revision before it can serve as a reference. read the letter →

arxiv 2507.03175 v1 pith:CPKCD4J4 submitted 2025-07-03 cs.LG cs.AI

Haohua Wang , Jingge Wang , Zijie Zhao , Yang Tan , Yanru Wu , Hanbing Liu , Jingyun Yang , Enming Zhang
show 4 more authors
Xiangyu Chen Zhengze Rong Shanxin Guo Yang Li
This is my paper
classification cs.LGcs.AI
keywords transferabilitymetricstransferlearningknowledgedatasetmodeltaxonomygeneralizationboundsnegative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey aims to show that the scattered collection of transferability metrics in transfer learning can be organized into one coherent framework, and that once organized, the right metric for a given application becomes a matter of matching its assumptions to the scenario. The paper proposes a unified definition of transferability as the effectiveness of transferring a source knowledge modality to a target evaluation set, then classifies metrics by what knowledge is transferred (dataset versus model), the granularity of evaluation (task-level versus instance-level), and when transferability is computed (before, during, or after training). It connects these categories to theoretical generalization bounds based on divergence, optimal transport, kernel embeddings, and information theory, and shows how representative metrics such as H-score, LEEP, LogME, and OTCE instantiate the framework. The value would be practical: researchers could select or design transferability metrics deliberately rather than by trial and error, and negative transfer could be anticipated before expensive fine-tuning.

What carries the argument

The machinery is the unified definition $\mathrm{Trf}(K_s, K_t, E)$ together with the two-axis taxonomy it supports: knowledge modality (dataset transferability versus model transferability, with prompt transferability as a special case) and measurement granularity (task-level, region-level, and instance-level), further indexed by the stage at which transferability is computed (pre-hoc, online, post-hoc). The generalization bounds in Section 3, namely the $H\Delta H$-divergence bound, the Wasserstein-distance bound, the MMD bound, and the information-theoretic bound, provide the theoretical justification for using distributional divergence as dataset transferability, while the equation equating model transferability with expected log-likelihood under a non-overfitting assumption grounds the model-side metrics. These pieces do the work of unifying otherwise disparate metrics such as OTDD, H-score, LEEP, LogME, NCE, and Task2Vec under a single framework.

What would settle it

One concrete test: take a fixed source model and a target dataset, compute the taxonomy's recommended metric for model transferability at task-level granularity, then fine-tune and measure actual target accuracy. If across many source-target pairs the metric and the actual accuracy disagree in ranking more often than chance, or if a metric explicitly matched to knowledge modality and granularity performs no better than a random source selector, the claim that the taxonomy guides metric selection would be refuted. A sharper version targets the log-likelihood proxy: find a trained model that achieves high training-set log-likelihood on the target data but poor held-out accuracy, which would break the non-overfitting assumption that the definition of model transferability rests on.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that transferability is not a single hidden quantity but a family of quantities indexed by what is transferred and how finely it is measured. The paper defines transferability as $\mathrm{Trf}(K_s, K_t, E)$, the effectiveness of transferring source knowledge $K_s$ toward target knowledge $K_t$ over evaluation target $E$, and shows that existing metrics fall out of this definition as special cases: dataset transferability estimates the divergence between source and target distributions, while model transferability estimates the expected performance of a source-trained model after adaptation to the target task. Under the assumption that training and test distributions coincide without overfitting, the expected log-likelihood of target labels given the adapted model serves as the operational proxy for model transferability. The taxonomy that results covers both the knowledge modality and the granularity of evaluation, and the paper argues that the empirically observed instability of transferability rankings across studies is explained by metrics being applied outside the settings their assumptions were designed for.

Load-bearing premise

The load-bearing premise is that a model's log-likelihood on its own training set, provided the model does not overfit, reliably measures how well it will transfer to a new task.

Editorial extensions

If this is right

  • Practitioners can choose a transferability metric by situating their scenario: a training-free pre-hoc filter needs a dataset-divergence or representation-analysis method, while an online controller can use gradient-based metrics like PGE.
  • The taxonomy predicts that dataset transferability and model transferability are not interchangeable: the former depends only on source and target distributions, the latter on the source model, so a model can transfer well even when raw dataset distance is large.
  • Instance-level transferability metrics double as out-of-distribution detectors, since a single sample's transferability score indicates how far it deviates from the training distribution.
  • Metrics that seem unrelated, such as NCE's negative conditional entropy and OTCE's entropy term, are connected through the same lower bound on target loss, unifying uncertainty-based and linear-loss-based estimators.
  • The three timing stages (pre-hoc, online, post-hoc) determine what information a metric may use, so computational cost and signal quality trade off systematically rather than arbitrarily.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the survey's framework would be a decision tree that maps scenario features (labeled target data, source data availability, training stage) to a shortlist of metrics, then validate the shortlist on the stability benchmark from the literature the survey cites.
  • If the unified definition holds, new knowledge modalities such as prompts, policies in reinforcement learning, or model edits can be slotted into the same $\mathrm{Trf}(K_s, K_t, E)$ form, giving prompt transferability and adversarial transferability a shared formal skeleton with classic model transfer.
  • The non-overfitting proxy suggests that transferability metrics may need a complexity penalty when the source model overfits the target training set, an empirical prediction the paper does not make.
  • The survey's taxonomy implies that reports of conflicting metric evaluations should be re-read not as contradictions but as boundary conditions: each metric excels inside its own assumption region, so the field's next step is to map those regions explicitly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This survey proposes a unified definition and taxonomy of transferability metrics for transfer learning, organizing metrics by knowledge modality (dataset vs. model) and evaluation granularity (task-level vs. instance-level), and by whether transferability is computed pre-hoc, online, or post-hoc. It provides a theoretical motivation through generalization bounds (HΔH-divergence, Wasserstein distance, MMD, information-theoretic bounds), then catalogs dataset-level and model-level transferability metrics, and finally discusses applications across eight learning paradigms including domain adaptation, meta-learning, domain generalization, knowledge distillation, continual learning, and curriculum learning. The central claim is that understanding the assumptions behind each metric guides practitioners in selecting appropriate metrics for specific transfer scenarios.

Significance. If the technical statements are corrected, this survey could serve as a useful organizing reference for a growing and fragmented literature. Its taxonomy based on knowledge type and measurement granularity is reasonable and covers recent metrics (OTCE, TMI, CLUE, PGE) not fully treated in earlier surveys. The paper also connects transferability to broader topics such as OOD detection, model editing, and adversarial transferability. However, the survey's value depends heavily on the correctness of its theoretical motivation and of the formulas it reports; several of these are currently unreliable. The paper ships no new results, no empirical comparisons, and no code, so its contribution is conceptual and organizational rather than evidential.

major comments (4)
  1. [Section 3.2, Eq. (16)] The transition from Eq. (15), which defines transferability as expected test accuracy, to Eq. (16), which replaces it with expected training log-likelihood, is the load-bearing definitional step of the theoretical section, but it is merely asserted. Since the target model θ_t is obtained by minimizing loss on the target training set, the training log-likelihood ℓ_Yt(θ_t) is high by construction and can diverge arbitrarily from target generalization when the target set is small or shifted. The 'non-overfitting' caveat stated in the text is precisely the condition that transfer settings typically violate. This unsupported proxy undermines the survey's claim that likelihood-based metrics such as LEEP, NCE, and LogME rest on a unified, understood theoretical foundation. Please either prove Eq. (16) under explicit conditions or clearly reframe it as a modeling assumption with a discussion of when it fails and with empirical references.
  2. [Section 3.1.2, Eqs. (6) and (7)] The population Wasserstein bound in Eq. (6) is stated as ε_T(h,h') ≤ ε_S(h,h') + W_1(D_S,D_T), but the finite-sample bound in Eq. (7) additionally contains the ideal-joint-error term λ and a sample-complexity term. In the standard optimal-transport DA bounds from the literature, the population bound also includes a term depending on the disagreement of the labeling functions across domains; without such a term, Eq. (6) incorrectly suggests that target error is controlled by source error plus distribution distance alone. Please correct Eq. (6) and state explicitly the assumptions on the loss (convex, symmetric, bounded, triangle inequality) and on the ideal hypothesis.
  3. [Section 3.1.1, Eq. (1) and surrounding text] The definition of the HΔH hypothesis class is garbled: 'HΔH = {h | h = h1⊗h2, h1,h2∈H}' is not the standard symmetric-difference hypothesis class, and the notation ε_S(h,h') used in Eq. (1) is never defined (only ε_S(h,f) is defined earlier). As written, the HΔH-divergence and the bounds in Eqs. (2) and (3) cannot be checked. Please replace this with the standard definition in which HΔH contains disagreement functions of pairs of hypotheses, and define ε_S(h,h') as the probability that h and h' disagree on samples from the source distribution.
  4. [Section 4.2.1, Eq. (31)] The Fisher information matrix expression in Eq. (31) is not a recognizable Fisher information for any stated model: F = E_{x,y}[(y-p)^2·S⊗xx^T] with S = ww^T∘zz^T∘(1-z)(1-z)^T does not match the standard FIM of a probabilistic classifier, and the symbols p, S, and z are introduced without definition. This formula appears in the explanation of Task2Vec, a method whose correctness relies on the Fisher information construction. Please either provide the exact FIM formula with all terms defined, or cite the precise equation from the original Task2Vec paper and state any simplifications being made.
minor comments (4)
  1. [Section 4.1.1, Eq. (26)] The k-NN KL estimator is introduced with an empty citation ('presented in []' and 'defined in [] as:'), and the symbol n is overloaded: it denotes the number of dimensions in Eq. (26) while elsewhere it denotes sample size. Please fill in the citation and use distinct symbols for dimension and sample counts.
  2. [References] There are duplicate references: [16] and [17] both cite Devlin et al.'s BERT paper, and [68] and [69] are the same remote-sensing transferability paper. Please deduplicate and renumber.
  3. [Throughout] The manuscript contains many typographical errors that should be corrected in a revision, including 'transfered', 'knowlegde', 'practicle', 'catagorize', 'Tranferability', 'finetuning' as a verb, and inconsistent use of 'transferability' vs. 'transferrability'.
  4. [Section 4.2.3, H-score discussion] In the interpretation of H-score, the statement that 'a small trace of the feature covariance matrix' indicates low redundancy is not immediate from H(g) = tr(cov(g(X))^{-1} cov(E[g(X)|Y])), since the inverse covariance appears in the formula. Please clarify the intended relationship between the trace of the covariance and the H-score.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey's taxonomy is an organizing review framework, and its log-likelihood proxy is an unproven assumption rather than a derivation from its own inputs.

full rationale

The paper is a survey and taxonomy, not a derivation of new transferability results, so the principal circularity patterns (self-definitional reductions, fitted inputs called predictions, uniqueness theorems imported from authors) do not occur. The closest thing to a load-bearing step is Section 3.2, where Eq. (15) defines transferability as expected test accuracy and Eq. (16) then says that 'under the assumption of non-overfitting, the log-likelihood on the training set, ℓ_Yt(θ_t), serves as a reliable proxy for the transferability measure,' so that 'in subsequent sections, we employ the log-likelihood as an alternative indicator of task transferability.' This is an unsupported equivalence claim, not a circular reduction: the training log-likelihood is not fitted to the accuracy it is supposed to stand in for, nor is any quantity derived from Eq. (16) used to predict itself. It is a definitional proxy whose validity is assumed, and the weakness belongs in a correctness/rigor assessment rather than a circularity score. The generalization bounds in Section 3.1 are standard external results (e.g., Ben-David et al. HΔH-divergence, Wasserstein/MMD bounds) and provide independent support. Self-citations such as [7] (H-score), [83] (OTCE/F-OTCE), [49], [88], [94], and [22] are used only to describe metrics under review or applications, not to justify the taxonomy's validity or to rule out alternatives, so they are not load-bearing. The manuscript's editorial gaps — e.g., Section 4.1.1 says the KL approximation 'was presented in []' and 'is defined in [] as:' — are missing references, not circular dependencies. The taxonomy's re-labeling of existing metrics is a review-organizing choice, not a disguised empirical prediction. Thus the derivation chain is self-contained with respect to circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The survey does not introduce new entities. The central claim rests on standard learning-theoretic bounds and a non-overfitting proxy assumption, plus one underdetermined formula (OTCE).

free parameters (1)
  • OTCE coefficients λ1, λ2, b
    Eq (34) defines OTCE as a linear combination of domain and task differences but gives no values or fitting procedure for the coefficients, so the metric as described is underdetermined.
assumptions (4)
  • standard math Domain adaptation generalization bounds: target error ≤ source error + half HΔH-divergence + λ (Eq 2-3), and Wasserstein/MMD variants (Eq 6-10).
    Invoked in Section 3.1 to motivate divergence-based transferability metrics; these are standard results from Ben-David et al. [8] and others, but stated without proof and with potential transcription errors.
  • domain assumption Loss function is positive, symmetric, and obeys triangle inequality (Section 3.1).
    Assumed for the theoretical bounds in Section 3.1; not discussed for non-metric losses.
  • domain assumption Non-overfitting assumption: training-set log-likelihood is a reliable proxy for transferability (Section 3.2, Eq (16)).
    The paper states this assumption explicitly; it underlies the definition of model transferability as expected log-likelihood.
  • standard math RKHS kernel embedding assumptions for MMD bounds (Section 3.1.3).
    The MMD bound relies on the RKHS being rich enough, as in Fukumizu et al. [26].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Understanding Knowledge Transferability for Transfer Learning: A Survey." pith.science (2026). https://pith.science/paper/CPKCD4J4

@misc{pith2026250703175,
  author       = {Pith},
  title        = {Pith review of: Understanding Knowledge Transferability for Transfer Learning: A Survey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CPKCD4J4}},
  note         = {Machine review of arXiv:2507.03175}
}
read the original abstract

Transfer learning has become an essential paradigm in artificial intelligence, enabling the transfer of knowledge from a source task to improve performance on a target task. This approach, particularly through techniques such as pretraining and fine-tuning, has seen significant success in fields like computer vision and natural language processing. However, despite its widespread use, how to reliably assess the transferability of knowledge remains a challenge. Understanding the theoretical underpinnings of each transferability metric is critical for ensuring the success of transfer learning. In this survey, we provide a unified taxonomy of transferability metrics, categorizing them based on transferable knowledge types and measurement granularity. This work examines the various metrics developed to evaluate the potential of source knowledge for transfer learning and their applicability across different learning paradigms emphasizing the need for careful selection of these metrics. By offering insights into how different metrics work under varying conditions, this survey aims to guide researchers and practitioners in selecting the most appropriate metric for specific applications, contributing to more efficient, reliable, and trustworthy AI systems. Finally, we discuss some open challenges in this field and propose future research directions to further advance the application of transferability metrics in trustworthy transfer learning.

Figures

Figures reproduced from arXiv: 2507.03175 by the authors.

Figure 1
Figure 1. Different types of knowledge to be transferred. (a) Both model backbone and task head are tunable. (b) (c) The backbone of [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Transferability estimation between source and target model modality, where transferability is computed over different [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Three stages (pre-hoc, online, post-hoc) of when transferability is measured. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Taxonomy of transferability quantification methods. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Embedding of marginal distributions: each distribution is mapped into a reproducing kernel Hilbert space via an expectation [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: DDS compares model features to obtain model initialization rankings. Matrix [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Adds before the target model additional free layers (consist of only linear transformations), whose parameters are optimized [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Simple architecture of NN-based transfer learning, where transferability is the expected likelihood target with optimal decoder [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Domain adaptation problem. Another line of works offers a different perspective on how transferability can be achieved between domains. Incorporating with a domain discriminator, adversarial-based methods encourage domain confusion by training a domain￾invariant featur…
Figure 10
Figure 10. Figure 10: Multi-source transfer problem. Multi-source transfer learning (as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: Meta learning problem. Meta learning, often known as "learning to learn", aims to optimize the learning process itself across multiple tasks, enabling rapid adaptation to new, unseen tasks [81]. The incorporation of transferability metrics into meta-learning processes…
Figure 12
Figure 12. Figure 12: Domain generalization problem. Domain generalization seeks to train a model on multiple source domains, en￾abling it to perform well on a new target domain with a different data distribution, without requiring any target domain data during training. The primary object…
Figure 13
Figure 13. Figure 13: Knowledge distillation. Knowledge distillation is a form of model-to-model transfer learning, where a smaller, more efficient student model learns from a larger, more complex teacher model. A critical aspect of this process is the transferability between the teacher a…
Figure 14
Figure 14. Figure 14: Continual learning problem. Catastrophic forgetting is a critical issue in fine-tuning neural networks, espe￾cially with limited labeled data. This problem leads to representational collapse, where generalizable features degrade during fine-tuning [3]. To mitigate thi…
Figure 15
Figure 15. Figure 15: Task curriculum learning. Curriculum learning is the training strategy that trains a machine learning model in a meaningful order. This learning paradigm aims to determine the best order of tasks that should be learned to maximize performance on the target task [78]. …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

110 extracted references · 58 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless C Fowlkes, Stefano Soatto, and Pietro Perona

  3. [3]

    Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. 2021. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . 7319–7328

    2021

  4. [4]

    Andrea Agostinelli, Michal Pándy, Jasper Uijlings, Thomas Mensink, and Vittorio Ferrari. 2022. How stable are transferability metrics evaluations?. In European Conference on Computer Vision . Springer, 303–321

    2022

  5. [5]

    Andrea Agostinelli, Jasper Uijlings, Thomas Mensink, and Vittorio Ferrari. 2022. Transferability metrics for selecting source model ensembles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 7936–7946

    2022

  6. [6]

    David Alvarez-Melis and Nicolo Fusi. 2020. Geometric dataset distances via optimal transport. Advances in Neural Information Processing Systems 33 (2020), 21428–21439

    2020

  7. [7]

    Yajie Bao, Yang Li, Shao-Lun Huang, Lin Zhang, Lizhong Zheng, Amir Zamir, and Leonidas Guibas. 2019. An information-theoretic approach to transferability in task transfer learning. In 2019 IEEE international conference on image processing (ICIP) . IEEE, 2309–2313

    2019

  8. [8]

    Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine learning 79 (2010), 151–175. Manuscript submitted to ACM 32 Wang et al

    2010

Show all 110 references
  1. [9]

    Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2006. Analysis of Representations for Domain Adaptation. In Advances in Neural Information Processing Systems , B. Schölkopf, J. Platt, and T. Hoffman (Eds.), Vol. 19. MIT Press

    2006

  2. [10]

    Hong Chen, Xin Wang, Chaoyu Guan, Yue Liu, and Wenwu Zhu. 2022. Auxiliary learning with joint task and data scheduling. In International Conference on Machine Learning . PMLR, 3634–3647

    2022

  3. [11]

    Zhuohao Chen, Jangwon Kim, Ram Bhakta, and Mustafa Sir. 2022. Leveraging task transferability to meta-learning for clinical section classification with limited data. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape...

    2022

  4. [12]

    Stergios Christodoulidis, Marios Anthimopoulos, Lukas Ebner, Andreas Christe, and Stavroula Mougiakakou. 2016. Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE journal of biomedical and health informatics 21, 1 (2016), 76–84

    2016

  5. [13]

    Alexandra Chronopoulou, Christos Baziotis, and Alexandros Potamianos. 2019. An embarrassingly simple approach for transfer learning from pretrained language models. arXiv preprint arXiv:1902.10547 (2019)

    2019 arXiv

  6. [14]

    Koby Crammer, Michael Kearns, and Jennifer Wortman. 2008. Learning from Multiple Sources. Journal of Machine Learning Research 9, 8 (2008)

    2008

  7. [15]

    Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu. 2007. Boosting for transfer learning. In Proceedings of the 24th international conference on Machine learning. 193–200

    2007

  8. [16]

    Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, ...

    2019

  9. [17]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  10. [18]

    Yuhe Ding, Bo Jiang, Aijing Yu, Aihua Zheng, and Jian Liang. 2024. Which model to transfer? a survey on transferability estimation. arXiv preprint arXiv:2402.15231 (2024)

    2024 arXiv

  11. [19]

    Jiahua Dong, Zhen Fang, Anjin Liu, Gan Sun, and Tongliang Liu. 2021. Confident anchor-induced multi-source free domain adaptation. Advances in Neural Information Processing Systems 34 (2021), 2848–2860

    2021

  12. [20]

    Xiangtong Du, Zhidong Liu, Zunlei Feng, and Hai Deng. 2024. DataMap: Dataset transferability map for medical image classification. Pattern Recognition 146 (2024), 110044

    2024

  13. [21]

    Yunshu Du, Wojciech M Czarnecki, Siddhant M Jayakumar, Mehrdad Farajtabar, Razvan Pascanu, and Balaji Lakshminarayanan. 2018. Adapting auxiliary losses using gradient similarity. arXiv preprint arXiv:1812.02224 (2018)

    2018 arXiv

  14. [22]

    Shutong Duan, Jingyun Yang, Yang Tan, Guoqing Zhang, Yang Li, and Xiao-Ping Zhang. 2025. Transfer Risk Map: Mitigating Pixel-level Negative Transfer in Medical Segmentation. arXiv preprint arXiv:2502.02340 (2025)

    2025 arXiv

  15. [23]

    Kshitij Dwivedi, Jiahui Huang, Radoslaw Martin Cichy, and Gemma Roig. 2020. Duality diagram similarity: a generic framework for initialization selection in task transfer learning. In European Conference on Computer Vision . Springer, 497–513

    2020

  16. [24]

    Kshitij Dwivedi and Gemma Roig. 2019. Representation similarity analysis for efficient task taxonomy & transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 12387–12396

    2019

  17. [25]

    Filippelli, Karen Schleeweis, Mark D

    Steven K. Filippelli, Karen Schleeweis, Mark D. Nelson, Patrick A. Fekety, and Jody C. Vogeler. 2024. Testing temporal transferability of remote sensing models for large area monitoring. Science of Remote Sensing 9 (June 2024), 100119

    2024

  18. [26]

    Kenji Fukumizu, Arthur Gretton, Gert Lanckriet, Bernhard Schölkopf, and Bharath K Sriperumbudur. 2009. Kernel choice and classifiability for RKHS embeddings of probability distributions. Advances in neural information processing systems 22 (2009)

    2009

  19. [27]

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. Journal of machine learning research 17, 59 (2016), 1–35

    2016

  20. [28]

    Jindong Gu, Xiaojun Jia, Pau de Jorge, Wenqain Yu, Xinwei Liu, Avery Ma, Yuan Xun, Anjun Hu, Ashkan Khakzar, Zhijiang Li, et al. 2023. A survey on transferability of adversarial examples across deep neural networks. arXiv preprint arXiv:2310.17626 (2023)

    2023 arXiv

  21. [29]

    Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964 (2020)

    2020 arXiv

  22. [30]

    Dongchen Han, Xiaojun Jia, Yang Bai, Jindong Gu, Yang Liu, and Xiaochun Cao. 2023. Ot-attack: Enhancing adversarial transferability of vision-language models via optimal transport optimization. arXiv preprint arXiv:2312.04403 (2023)

    2023 arXiv

  23. [31]

    Zhongyi Han, Zhiyan Zhang, Fan Wang, Rundong He, Wan Su, Xiaoming Xi, and Yilong Yin. 2023. Discriminability and Transferability Estimation: A Bayesian Source Importance Estimation Approach for Multi-Source-Free Domain Adaptation. In Proceedings of the AAAI Conference on Artif...

    2023

  24. [32]

    Dan Hendrycks and Kevin Gimpel. 2017. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In International Conference on Learning Representations

    2017

  25. [33]

    Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146 (2018)

    2018 arXiv

  26. [34]

    Jingke Huang, Ni Xiao, and Lei Zhang. 2022. Balancing transferability and discriminability for unsupervised domain adaptation. IEEE Transactions on Neural Networks and Learning Systems 35, 4 (2022), 5807–5814

    2022

  27. [35]

    Junguang Jiang, Yang Shu, Jianmin Wang, and Mingsheng Long. 2022. Transferability in deep learning: A survey. arXiv preprint arXiv:2201.05867 (2022). Manuscript submitted to ACM Understanding Knowledge Transferability for Transfer Learning: A Survey 33

    2022 arXiv

  28. [36]

    Jing Jiang and ChengXiang Zhai. 2007. Instance Weighting for Domain Adaptation in NLP. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics . 264–271

    2007

  29. [37]

    Mostafa Kalhor, Ahmad Kalhor, and Mehdi Rahmani. 2020. Ranking and rejecting of pre-trained deep neural networks in transfer learning based on separation index. arXiv preprint arXiv:2012.13717 (2020)

    2020 arXiv

  30. [38]

    Leonid V Kantorovich. 2006. On the Translocation of Masses. Journal of mathematical sciences 133, 4 (2006)

    2006

  31. [39]

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of...

    2017

  32. [40]

    Joshua Lee, Prasanna Sattigeri, and Gregory Wornell. 2019. Learning new tricks from old dogs: Multi-source transfer learning from pre-trained networks. Advances in neural information processing systems 32 (2019)

    2019

  33. [41]

    Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. In Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and...

    2018

  34. [42]

    Jingyao Li, Shuai Lü, Wenbo Zhu, and Zhanshan Li. 2022. Enhancing transferability and discriminability simultaneously for unsupervised domain adaptation. Knowledge-Based Systems 247 (2022), 108705

    2022

  35. [43]

    Juan Li, Zhiqiang Xiao, Rui Sun, and Jinling Song. 2022. Retrieval of the Leaf Area Index from Visible Infrared Imaging Radiometer Suite (VIIRS) Surface Reflectance Based on Unsupervised Domain Adaptation. Remote Sensing 14, 8 (April 2022), 1826

    2022

  36. [44]

    Xinyao Li, Jingjing Li, Fengling Li, Lei Zhu, and Ke Lu. 2024. Agile Multi-Source-Free Domain Adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 13673–13681

    2024

  37. [45]

    Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40, 12 (2017), 2935–2947

    2017

  38. [46]

    Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. 2017. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690 (2017)

    2017 arXiv

  39. [47]

    Sihao Lin, Hongwei Xie, Bing Wang, Kaicheng Yu, Xiaojun Chang, Xiaodan Liang, and Gang Wang. 2022. Knowledge distillation via the target-aware transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10915–10924

    2022

  40. [48]

    Ziqian Lin, Sreya Dutta Roy, and Yixuan Li. 2021. Mood: Multi-level out-of-distribution detection. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition . 15313–15323

    2021

  41. [49]

    Hanbing Liu, Jingge Wang, Xuan Zhang, Ye Guo, and Yang Li. 2024. Enhancing Continuous Domain Adaptation with Multi-path Transfer Curriculum. In Pacific-Asia Conference on Knowledge Discovery and Data Mining . Springer, 286–298

    2024

  42. [50]

    Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. 2020. Energy-based out-of-distribution detection. Advances in neural information processing systems 33 (2020), 21464–21475

    2020

  43. [51]

    Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. InInternational conference on machine learning . PMLR, 97–105

    2015

  44. [52]

    Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. Advances in neural information processing systems 31 (2018)

    2018

  45. [53]

    Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2016. Unsupervised domain adaptation with residual transfer networks. Advances in neural information processing systems 29 (2016)

    2016

  46. [54]

    Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2017. Deep transfer learning with joint adaptation networks. In International conference on machine learning . PMLR, 2208–2217

    2017

  47. [55]

    Yuchi Ma and Zhou Zhang. 2022. A Bayesian Domain Adversarial Neural Network for Corn Yield Prediction. IEEE Geoscience and Remote Sensing Letters 19 (2022), 1–5

    2022

  48. [56]

    Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. 2008. Domain adaptation with multiple sources. Advances in neural information processing systems 21 (2008)

    2008

  49. [57]

    Mauro Martini, Vittorio Mazzia, Aleem Khaliq, and Marcello Chiaberge. 2021. Domain-Adversarial Training of Self-Attention-Based Networks for Land Cover Classification Using Multi-Temporal Sentinel-2 Satellite Imagery. Remote Sensing 13, 13 (June 2021), 2564

    2021

  50. [58]

    Sören Mindermann, Jan M Brauner, Muhammed T Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen, Aidan N Gomez, Adrien Morisot, Sebastian Farquhar, et al. 2022. Prioritized training on points that are learnable, worth learning, and not yet learnt. In Internatio...

    2022

  51. [59]

    Peyman Morteza and Yixuan Li. 2022. Provable guarantees for understanding out-of-distribution detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7831–7840

    2022

  52. [60]

    Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, and Bernhard Schölkopf. 2017. Kernel Mean Embedding of Distributions: A Review and Beyond. Foundations and Trends® in Machine Learning 10, 1–2 (2017), 1–141

    2017

  53. [61]

    Cuong Nguyen, Tal Hassner, Matthias Seeger, and Cedric Archambeau. 2020. LEEP: A New Measure to Evaluate Transferability of Learned Representations. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119) , H...

    2020

  54. [62]

    Cuong N Nguyen, Phong Tran, Lam Si Tung Ho, Vu Dinh, Anh T Tran, Tal Hassner, and Cuong V Nguyen. 2023. Simple transferability estimation for regression tasks. In Uncertainty in Artificial Intelligence. PMLR, 1510–1521

    2023

  55. [63]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  56. [64]

    Sinno Jialin Pan and Qiang Yang. 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (2010), 1345–1359

    2010

  57. [65]

    Viraj Prabhu, Arjun Chandrasekaran, Kate Saenko, and Judy Hoffman. 2021. Active domain adaptation via clustering uncertainty-weighted embeddings. In Proceedings of the IEEE/CVF international conference on computer vision . 8505–8514

    2021

  58. [66]

    Rhorom Priyatikanto, Yang Lu, Jadu Dash, and Justin Sheffield. 2023. Improving generalisability and transferability of machine-learning-based maize yield prediction model through domain adaptation. Agricultural and Forest Meteorology 341 (2023), 109652

    2023

  59. [67]

    Huiyan Qi, Lechao Cheng, Jingjing Chen, Yue Yu, Xue Song, Zunlei Feng, and Yu-Gang Jiang. 2022. Transferability estimation based on principal gradient expectation. arXiv preprint arXiv:2211.16299 (2022)

    2022 arXiv

  60. [69]

    Rongjun Qin, Guixiang Zhang, and Yang Tang. 2023. On the transferability of learning models for semantic segmentation for remote sensing data. arXiv preprint arXiv:2310.10490 (2023)

    2023 arXiv

  61. [70]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  62. [71]

    M Rosenblat. 1956. Remarks on some nonparametric estimates of a density function. Ann. Math. Stat 27 (1956), 832–837

    1956

  63. [72]

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. [n. d.]. Distributionally Robust Neural Networks. InInternational Conference on Learning Representations

  64. [73]

    Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018. Wasserstein distance guided representation learning for domain adaptation. InProceedings of the AAAI conference on artificial intelligence , Vol. 32

    2018

  65. [74]

    Baifeng Shi, Judy Hoffman, Kate Saenko, Trevor Darrell, and Huijuan Xu. 2020. Auxiliary task reweighting for minimum-data learning. Advances in Neural Information Processing Systems 33 (2020), 7148–7160

    2020

  66. [75]

    Yaowei Shi, Aidong Deng, Minqiang Deng, Jing Li, Meng Xu, Shun Zhang, Xue Ding, and Shuo Xu. 2022. Domain transferability-based deep domain generalization method towards actual fault diagnosis scenarios. IEEE Transactions on Industrial Informatics 19, 6 (2022), 7355–7366

    2022

  67. [76]

    Aman Sinha, Hongseok Namkoong, and John C Duchi. 2017. Certifiable Distributional Robustness with Principled Adversarial Training. CoRR, abs/1710.10571. arXiv preprint arXiv:1710.10571 (2017)

    2017 arXiv

  68. [77]

    Jie Song, Yixin Chen, Xinchao Wang, Chengchao Shen, and Mingli Song. 2019. Deep model transferability from attribution maps. Advances in Neural Information Processing Systems 32 (2019)

    2019

  69. [78]

    Petru Soviany, Radu Tudor Ionescu, Paolo Rota, and Nicu Sebe. 2022. Curriculum learning: A survey. International Journal of Computer Vision 130, 6 (2022), 1526–1565

    2022

  70. [79]

    Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, and Jie Zhou. 2022. On Transferability of Prompt Tuning for Natural Language Processing. In Proceedings of the 2022 Conference of the...

    2022

  71. [80]

    Baochen Sun, Jiashi Feng, and Kate Saenko. 2017. Correlation alignment for unsupervised domain adaptation. Domain adaptation in computer vision applications (2017), 153–171

    2017

  72. [81]

    Qianru Sun, Yaoyao Liu, Zhaozheng Chen, Tat-Seng Chua, and Bernt Schiele. 2020. Meta-transfer learning through hard tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 3 (2020), 1443–1456

    2020

  73. [82]

    Shiliang Sun, Honglei Shi, and Yuanbin Wu. 2015. A survey of multi-source domain adaptation. Information Fusion 24 (2015), 84–92

    2015

  74. [83]

    Yang Tan, Enming Zhang, Yang Li, Shao-Lun Huang, and Xiao-Ping Zhang. 2024. Transferability-guided cross-domain cross-task transfer learning. IEEE Transactions on Neural Networks and Learning Systems (2024)

    2024

  75. [84]

    Xinyi Tong, Xiangxiang Xu, Shao-Lun Huang, and Lizhong Zheng. 2021. A mathematical framework for quantifying transferability in multi-source transfer learning. Advances in Neural Information Processing Systems 34 (2021), 26103–26116

    2021

  76. [85]

    Anh T Tran, Cuong V Nguyen, and Tal Hassner. 2019. Transferability and hardness of supervised classification tasks. InProceedings of the IEEE/CVF international conference on computer vision . 1395–1405

    2019

  77. [86]

    Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition . 7167–7176

    2017

  78. [87]

    Liang Wan, Weijun Zhou, Yong He, Thomas Cherico Wanger, and Haiyan Cen. 2022. Combining transfer learning and hyperspectral reflectance analysis to assess leaf nitrogen concentration across different plant species datasets. Remote Sensing of Environment 269 (Feb. 2022), 112826

    2022

  79. [88]

    Jingge Wang, Liyan Xie, Yao Xie, Shao-Lun Huang, and Yang Li. 2024. Generalizing to unseen domains with Wasserstein distributional robustness under limited source knowledge. IEEE Journal of Selected Topics in Signal Processing (2024). Manuscript submitted to ACM Understanding ...

    2024

  80. [89]

    Lin Wang and Kuk-Jin Yoon. 2021. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE transactions on pattern analysis and machine intelligence 44, 6 (2021), 3048–3068

    2021

  81. [90]

    Yumiao Wang, Luwei Feng, Weiwei Sun, Zhou Zhang, Hanyu Zhang, Gang Yang, and Xiangchao Meng. 2022. Exploring the potential of multi-source unsupervised domain adaptation in crop mapping using Sentinel-2 images. GIScience & Remote Sensing 59, 1 (Dec. 2022), 2247–2265

    2022

  82. [91]

    Xing Wei, Shaofan Liu, Yaoci Xiang, Zhangling Duan, Chong Zhao, and Yang Lu. 2020. Incremental learning based multi-domain adaptation for object detection. Knowledge-Based Systems 210 (2020), 106420

    2020

  83. [92]

    Jun Wu and Jingrui He. 2024. Trustworthy Transfer Learning: A Survey. arXiv preprint arXiv:2412.14116 (2024)

    2024

  84. [93]

    Xuetong Wu, Jonathan H Manton, Uwe Aickelin, and Jingge Zhu. 2024. On the generalization for transfer learning: An information-theoretic analysis. IEEE Transactions on Information Theory (2024)

    2024

  85. [94]

    Yanru Wu, Jianning Wang, Weida Wang, and Yang Li. 2024. H-ensemble: An Information Theoretic Approach to Reliable Few-Shot Multi-Source-Free Transfer. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 15970–15978

    2024

  86. [95]

    Huiwen Xu and U Kang. 2023. Fast and accurate transferability measurement by evaluating intra-class feature variance. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 11474–11482

    2023

  87. [96]

    Yihao Xue, Rui Yang, Xiaohan Chen, Weibo Liu, Zidong Wang, and Xiaohui Liu. 2024. A Review on Transferability Estimation in Deep Transfer Learning. IEEE Transactions on Artificial Intelligence 5, 12 (2024), 5894–5914

    2024

  88. [97]

    Chen Yang, Yifan Liu, and Yixuan Yuan. 2023. Transferability-Guided Multi-source Model Adaptation for Medical Image Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 703–712

    2023

  89. [98]

    Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. 2024. Generalized out-of-distribution detection: A survey. International Journal of Computer Vision (2024), 1–28

    2024

  90. [99]

    Yuncheng Yang et al. 2023. Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation. arXiv preprint arXiv:2307.11958 (2023)

    2023 arXiv

  91. [100]

    Yuncheng Yang, Meng Wei, Junjun He, Jie Yang, Jin Ye, and Yun Gu. 2023. Pick the best pre-trained model: Towards transferability estimation for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 674–683

    2023

  92. [101]

    Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language models: Problems, methods, and opportunities. arXiv preprint arXiv:2305.13172 (2023)

    2023 arXiv

  93. [102]

    Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks?. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc

    2014

  94. [103]

    Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks? Advances in neural information processing systems 27 (2014)

    2014

  95. [104]

    Kaichao You, Yong Liu, Jianmin Wang, and Mingsheng Long. 2021. Logme: Practical assessment of pre-trained models for transfer learning. In International Conference on Machine Learning . PMLR, 12133–12143

    2021

  96. [105]

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. 2020. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems 33 (2020), 5824–5836

    2020

  97. [106]

    Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Zi-Hang Jiang, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. 2021. Tokens-to-token vit: Training vision transformers from scratch on imagenet. In Proceedings of the IEEE/CVF international conference on computer vision . 558–567

    2021

  98. [107]

    Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. 2018. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition . 3712–3722

    2018

  99. [108]

    Guojun Zhang, Han Zhao, Yaoliang Yu, and Pascal Poupart. 2021. Quantifying and improving transferability in domain generalization. Advances in Neural Information Processing Systems 34 (2021), 10957–10970

    2021

  100. [109]

    Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. 2024. Panda: Prompt transfer meets knowledge distillation for efficient model adaptation. IEEE Transactions on Knowledge and Data Engineering (2024)

    2024

  101. [110]

    Fuzhen Zhuang, Xiaohu Cheng, Ping Luo, Sinno Jialin Pan, and Qing He. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In Twenty-fourth international joint conference on artificial intelligence . Received 23 June 2025; revised XX XXXX 202X; a...

    2015

  102. [2019]

    In Proceedings of the IEEE/CVF international conference on computer vision

    Task2vec: Task embedding for meta-learning. In Proceedings of the IEEE/CVF international conference on computer vision . 6430–6439

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.