Pith. sign in

REVIEW 3 major objections 6 minor 75 references

Transferability and Hardness of Supervised Classification Tasks

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Label entropy bounds transferability between tasks

desk verdict A label-only conditional entropy measure for task transferability is a genuinely useful idea, but the experiments need to control for target hardness before the core claim is proven. read the letter →

arxiv 1908.08142 v1 pith:5Z2ZNT5P submitted 2019-08-21 cs.LG cs.CVstat.ML

Anh T. Tran , Cuong V. Nguyen , Tal Hassner This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords conditionalentropytransferabilitytaskhardnessinformationtheorytransferlearninglabelstatisticsclassificationmodel-freeestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that how well a model trained on one task transfers to another can be predicted in advance from the training labels alone, without training any model. The key quantity is the conditional entropy $H(Y|Z)$ of the target labels given the source labels, which lower-bounds the target log-likelihood of a transferred model (Theorem 1). The same idea estimates task hardness: a task is hard to the extent that its labels have high conditional entropy given a trivial constant label. If correct, this gives a cheap, model-free way to rank tasks and plan transfer learning. The claim is tested on 437 attribute-classification tasks across three image benchmarks, where the entropy estimates correlate strongly with measured transfer errors and hardness.

What carries the argument

The central object is the empirical conditional entropy $H(Y|Z)$ of two label sequences, computed by counting how often each source label $z$ co-occurs with each target label $y$ in the training set. It measures, in bits, how much uncertainty about the target label remains after the source label is known. The proof of the bound uses a specially constructed classifier $\bar{k}$ that converts the source softmax output into a target distribution by taking the conditional expectation $\sum_z \hat{P}(y|z) p_Z(z)$; because this classifier is available to the optimization, the target log-likelihood of the actual best classifier can only be higher, yielding the inequality.

What would settle it

Take two source-target task pairs with identical empirical joint label distributions (the same counts of each $(z,y)$ combination) but different assignments of those label pairs to individual images—for example, swap which images carry the $(z=0,y=1)$ label between two tasks. The conditional entropy estimate is identical, but actual transferability can diverge because the representation learned from $Z$ may separate the input space along the wrong boundary for one target task. An experiment showing large transfer-error differences between such paired tasks, despite equal $H(Y|Z)$, would refute the claim that the entropy alone orders transferability.

Watch

Extended reading notes

Core claim

The paper's central claim is Theorem 1: for two classification tasks $T_Z$ and $T_Y$ sharing the same training inputs, the transferability of $T_Z$ to $T_Y$—measured as the target training log-likelihood after transferring the representation trained on $Z$ and retraining a classifier on $Y$—is at least $l_Z(w_Z, h_Z) - H(Y|Z)$, where $l_Z$ is the source task's own log-likelihood and $H(Y|Z)$ is the conditional entropy of the label sequences. Because $l_Z$ is fixed when the source is fixed, the conditional entropy alone orders target tasks by transferability. The paper also claims that task hardness is estimated by $H(Z|C)$, the conditional entropy of the task's labels given a constant label sequence, which upper-bounds the negative log-likelihood of the optimal model. Empirically, these label-only estimates correlate strongly with measured test errors on 437 attribute tasks, and a face-recognition representation transferred to attributes ranked as most transferable matches or beats dedicated classifiers.

Load-bearing premise

The bound is proven for training log-likelihood, while the experiments measure test error, and the paper assumes that a well-trained model's training log-likelihood tracks its test accuracy, and that the linear SVM used in practice behaves like the specially constructed classifier in the proof.

Editorial extensions

If this is right

  • Transferability between any two discrete-label classification tasks sharing an input domain can be estimated by counting label co-occurrences alone, in milliseconds, without training any model or maintaining a zoo of trained networks.
  • Task hardness can be ranked before training, so budgets for model training and data collection can be allocated to tasks predicted to be hardest.
  • With a fixed source task, the conditional entropy orders all candidate target tasks before any transfer, enabling selection of targets worth transferring to and early stopping of unpromising transfers.
  • The estimates cover multi-class and multi-label settings, since they are computed directly on the label sequences, not on model outputs.
  • Following the case study, a single representation trained for a rich source task (such as face recognition) can be transferred to many target attributes, with accuracy comparable to dedicated models on targets ranked highly transferable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same conditional-entropy quantity could serve as a task-ordering criterion in continual learning, choosing the next task to minimize information loss from previously learned labels; the paper mentions continual learning only as a potential application.
  • Editorial inference: the bound implies that mutual information $I(Y;Z) = H(Y) - H(Y|Z)$ is the model-free component of task relatedness; this connects the entropy measure to a broader information-geometric view of task similarity.
  • Editorial inference: a natural stress test is to apply the measure to structured or continuous labels (segmentation maps, regression outputs) by discretizing them; the paper notes the extension but does not test it.
  • Editorial inference: the theory may also predict when fine-tuning the full network is necessary: targets with high $H(Y|Z)$ require changing the representation itself, not just retraining the last layer.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an information-theoretic, solution-agnostic way to estimate the transferability and hardness of supervised classification tasks defined over the same input instances. Transferability from a source task T_Z to a target task T_Y is estimated by the conditional entropy H(Y|Z) of the two training label sequences, and task hardness is estimated by H(Z|C), the conditional entropy from a constant-label trivial task. The main theoretical result, Theorem 1, lower-bounds the target training log-likelihood of a transferred representation by the source training log-likelihood minus H(Y|Z). The paper validates these estimates on 437 attribute classification tasks from CelebA, AwA2, and CUB, reporting strong Pearson correlations between the proposed estimates and empirically measured test errors, plus a case study transferring a face recognition network to CelebA attribute prediction.

Significance. If the central claim holds, the paper offers a remarkably cheap and model-agnostic predictor of transferability and hardness, computable from label counts alone, in contrast to model-zoo approaches such as Taskonomy or Task2Vec. The proof of Theorem 1 in Appendix A is elementary and correct, and the empirical scale is substantial: all 437 transferability scatter plots in the appendix show high correlations, and the identity-to-attribute case study is a practically relevant demonstration. The main risk is interpretive: the proposed transferability measure H(Y|Z) contains the target marginal entropy H(Y), which is the paper's own hardness measure, and the theoretical bound concerns training log-likelihood of an augmented classifier space, while the experiments measure test error of lSVMs. The paper therefore needs additional controls and a clearer statement of what is proven versus what is empirically suggested.

major comments (3)
  1. [§3.1, Eq. (5), §5.1] Theorem 1 is a correct lower bound on the training log-likelihood lY(wZ,kY) of the transferred model, but the experiments in §5.1 measure target test error. The only bridge is the informal statement in §3.1 that 'if the trained model does not overfit, the log-likelihood on the training set ... provides a good indicator of Eq. (4)', and Discussion 3 extends this to test log-likelihood only under an unproven bounded-difference assumption. Moreover, the theorem's bound holds for kY chosen from K after augmenting with the constructed classifier \bar{k}, whereas the experiments use a linear SVM; no argument is given that the lSVM's log-likelihood attains at least the bound. Consequently, the analytical result as stated does not cover the evaluation protocol used to support the paper's main empirical claim. Please prove a test-error analogue under explicit assumptions, or present the empirical results as heuristic validation and add a direct check, such as comparing lSVM training log-likelihood with the right-hand side of Eq. (10), to show that the missing link holds in practice.
  2. [§5.1, Fig. 2, Eq. (7)] The quantity being correlated with transferred test error is H(Y|Z) = H(Y) - I(Y;Z). For the binary attributes used in these experiments, I(Y;Z) is often small relative to H(Y), so H(Y|Z) is dominated by the target marginal entropy H(Y), which is exactly the paper's own hardness estimate (for a constant C, H(Z|C) = H(Z) in Eq. (14); Fig. 6 shows that H(Y) already strongly predicts dedicated test error). The high correlations in Fig. 2 may therefore reflect target hardness rather than the source-target relationship. No source-agnostic baseline, such as predicting transferred error from H(Y) alone, and no partial correlation controlling for H(Y), are reported. The identity-to-attribute case study in Sec. 5.2 provides one example where CE and H(Y) diverge (male has high hardness but low CE from identity), but it does not quantify the fraction of the overall correlation that is source-specific. Please add the baseline and partial-correlation analysis; if CE does not add predictive value over H(Y), the central transferability claim is not supported.
  3. [§4, Eq. (14), Fig. 6] Eq. (14) is an upper bound on hardness, not an equality or a proof of monotonicity: Hard(T_Z) ≤ H(Z|C) does not by itself imply that a task with larger H(Z|C) is harder. The empirical correlation in Fig. 6 is therefore the only support for using H(Z|C) as a hardness ordering, and the paper should state this explicitly. Additionally, because H(Z|C) equals the marginal label entropy H(Z) for a constant C, the measure is a label-balance statistic that ignores the input domain entirely. The text should acknowledge this and compare against a simple majority-class or label-entropy baseline for the dedicated errors, to separate the contribution of the information-theoretic framing from this well-known cue.
minor comments (6)
  1. [Appendix B] The derivation of Eq. (14) is only sketched; please add the short argument that optimality of (wZ,hZ) for T_Z gives Hard(T_Z) ≤ -lZ(wC,kZ), which combined with Theorem 1 and zero training loss of the trivial source yields the bound.
  2. [§5.1] The paper reports correlations for selected source tasks in Fig. 2 and says full results are in the appendix, but it does not give a summary statistic across all source tasks. Please report the mean, median, minimum, and maximum correlation over the 437 transferability conditions.
  3. [§5.1, Fig. 2] The p-values treat the target tasks as independent observations, but the 39 or 84 target tasks share the same images and are not statistically independent. A permutation test or a mixed-effects model would be more appropriate for assessing significance.
  4. [§5.1] The sentence 'We then compare these transferability scores to the corresponding CE estimates ... using an existing correlation analysis [44]' cites a continual-learning preprint; the standard Pearson correlation analysis should be described directly rather than through that reference.
  5. [Table 1 and Appendix D] The notation 'CE↑(decreasing transferability)' and 'Conditional Entropy↑' is unclear; please state that attributes are sorted by ascending CE and explain the arrow convention in the caption.
  6. [General] There are minor typos and formatting issues, including 'tranferability' in Sec. 3.2 and inconsistent capitalization of Softmax; these should be corrected in a revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the CE estimates are computed directly from label counts, the main theorem is an explicit bound proven from the model assumptions, and the empirical validation is external to the estimate.

full rationale

The derivation chain is self-contained and non-circular. Theorem 1 bounds the transferred model's training log-likelihood by lZ(wZ,hZ) - H(Y|Z). The proof constructs a particular classifier \bar{k} whose log-likelihood equals exactly the terms that become -H(Y|Z) plus lZ(wZ,hZ) (Eqs. 11-12 and appendix Eq. 15-17). This is a real analytic reduction, not a restatement: H(Y|Z) appears via the identity (1/n)Σ log P(y_i|z_i) = -H(Y|Z), which follows from counting the empirical joint distribution, and is not fitted to transfer outcomes. The empirical transferability experiments compute CE from label counts (Eq. 6) and compare it to test errors of separately trained lSVM classifiers; no parameter in the CE estimate is tuned to those errors, so the measured correlations are not forced by construction. The hardness estimate H(Z|C) reduces mathematically to the marginal label entropy H(Z), but the paper explicitly presents Eq. (14) as an upper bound on optimal loss and treats the agreement with dedicated-network test errors as an empirical finding, acknowledging that it ignores the input domain. The skeptic's concern that H(Y|Z) = H(Y) - I(Y;Z) may be dominated by target marginal entropy is a real interpretational confound about what drives the correlation, but it is not circularity: the CE estimate and the measured accuracy are different quantities, and a confound is not an equation-by-construction identification. The only self-citation used as a method, reference [44] for 'an existing correlation analysis', is not load-bearing for the central claims. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' own work, and no ansatz is smuggled in via citation. The paper is therefore not circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The method itself is parameter-free, with no constants fitted to transfer outcomes. It rests on the shared-input-domain assumption, the train-to-test approximation, and the constructed classifier used only for the proof. No new physical or model entities are introduced.

assumptions (4)
  • domain assumption Tasks share identical input instances (X) and differ only in label sequences Y and Z.
    The CE H(Y|Z) is defined via the empirical joint distribution over labels assigned to the same inputs. This excludes transfer between different domains, a limitation acknowledged in Sec. 6.
  • domain assumption Training-set log-likelihood is a valid proxy for test accuracy (non-overfitting).
    Invoked in Sec. 3.1 to replace expected test accuracy with training log-likelihood (Eq. 5). The theorem bounds training loss; the experiments report test errors, so this bridge is load-bearing.
  • ad hoc to paper The classifier space K can be augmented with the constructed \bar{k}, and in practice the lSVM classifier achieves or exceeds the bound.
    Theorem 1 relies on \bar{k} being in K (Discussion 2); the experiments use lSVM and never check that the bound holds for it.
  • domain assumption Models are trained to global optimality in Eq. (1) and (3).
    The proof assumes argmin over W,H and K; the paper acknowledges local optima in Discussion 2 and claims the difference is small, but the experimental protocol does not verify global optimality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transferability and Hardness of Supervised Classification Tasks." pith.science (2026). https://pith.science/paper/5Z2ZNT5P

@misc{pith2026190808142,
  author       = {Pith},
  title        = {Pith review of: Transferability and Hardness of Supervised Classification Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5Z2ZNT5P}},
  note         = {Machine review of arXiv:1908.08142}
}
read the original abstract

We propose a novel approach for estimating the difficulty and transferability of supervised classification tasks. Unlike previous work, our approach is solution agnostic and does not require or assume trained models. Instead, we estimate these values using an information theoretic approach: treating training labels as random variables and exploring their statistics. When transferring from a source to a target task, we consider the conditional entropy between two such variables (i.e., label assignments of the two tasks). We show analytically and empirically that this value is related to the loss of the transferred model. We further show how to use this value to estimate task hardness. We test our claims extensively on three large scale data sets -- CelebA (40 tasks), Animals with Attributes 2 (85 tasks), and Caltech-UCSD Birds 200 (312 tasks) -- together representing 437 classification tasks. We provide results showing that our hardness and transferability estimates are strongly correlated with empirical hardness and transferability. As a case study, we transfer a learned face recognition model to CelebA attribute classification tasks, showing state of the art accuracy for tasks estimated to be highly transferable.

Figures

Figures reproduced from arXiv: 1908.08142 by the authors.

Figure 1
Figure 1. Visualizing toy examples. The transferability between two tasks, represented as sequences (X, Y ) and (X, Z). The horizontal axis represent instances and the values for Z (in red) and Y (cyan). In which of these examples would it be easiest to transfer a model trained for task T Z to task T Y ? See discussion and details in Sec. 3.3. Theorem 1 Under the training procedure described in Sec. 3.1, we have: Trf( f T Z →… view at source ↗
Figure 2
Figure 2. Attribute prediction; CE vs. test errors on target tasks. Examples from CelebA (a-d), AwA2 (e-h), and CUB (i-l). Plot titles name the source tasks T Z ; points represent different target tasks T Y . Corr is the Pearson correlation coefficient between the two variables and p is the statistical significance of the correlation. In all cases, the correlation is statistically significant. See Sec. 5.1 for details. Attrib… view at source ↗
Figure 3
Figure 3. Identity to attribute; CE vs. test errors on target tasks. Predicting 40 CelebA attributes using a face recognition network. Corr is the Pearson correlation coefficient between the two variables, and p is the statistical significance of the correlation [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Classification accuracy for varying training set sizes. Top: male; bottom: double chin. Dedicated classification net￾works trained from scratch (blue) vs. face recognition network transferred to the attributes with an lSVM (red). Because recogni￾tion transfers well to …
Figure 6
Figure 6. Figure 6: Estimated task hardness vs. empirical errors on the three benchmarks. Estimated hardness is well correlated with empirical hardness with significance p < 0.001. expressions (e.g., smiling and mouth open, right columns). Although this relationship has been noted by othe…
Figure 7
Figure 7. Figure 7: Attribute prediction; CE vs. test errors on CelebA (Extended from [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Attribute prediction; CE vs. test errors on AwA2 (Extended from [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Attribute prediction; CE vs. test errors on AwA2 (Extended from [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Attribute prediction; CE vs. test errors on AwA2 (Extended from [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 71 canonical work pages

  1. [1]

    Task2Vec: Task embedding for meta-learning

    Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless Fowlkes, Stefano Soatto, and Pietro Perona. Task2Vec: Task embedding for meta-learning. arXiv preprint arXiv:1902.03545, 2019

    arXiv 1902

  2. [2]

    The Information Complexity of Learning Tasks, their Structure and their Distance

    Alessandro Achille, Giovanni Paolini, Glen Mbeng, and Stefano Soatto. The information complexity of learning tasks, their structure and their distance. arXiv preprint arXiv:1904.03292, 2019

    work page Pith review arXiv 1904

  3. [3]

    Provable bounds for learning some deep representations

    Sanjeev Arora, Aditya Bhaskara, Rong Ge, and Tengyu Ma. Provable bounds for learning some deep representations. In Int. Conf. Mach. Learning, pages 584–592, 2014

    work page 2014

  4. [4]

    Factors of transferability for a generic convnet representation

    Hossein Azizpour, Ali Sharif Razavian, Josephine Sullivan, Atsuto Maki, and Stefan Carlsson. Factors of transferability for a generic convnet representation. Trans. Pattern Anal. Mach. Intell., 38(9):1790–1802, 2015

    work page 2015

  5. [5]

    Regularized learning for domain adaptation under label shifts

    Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang, and An- imashree Anandkumar. Regularized learning for domain adaptation under label shifts. In Int. Conf. on Learning Rep- resentations, 2019

    work page 2019

  6. [6]

    A theory of learning from different domains

    Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Mach. Learn., 79(1-2):151–175, 2010

    work page 2010

  7. [7]

    Exploiting task relat- edness for multiple task learning

    Shai Ben-David and Reba Schuller. Exploiting task relat- edness for multiple task learning. In Learning Theory and Kernel Machines, pages 567–580. Springer, 2003

    work page 2003

  8. [8]

    Learning bounds for domain adaptation

    John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman. Learning bounds for domain adaptation. In Neural Inform. Process. Syst., pages 129–136, 2008

    work page 2008

Show all 75 references
  1. [9]

    Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman. VGGFace2: A dataset for recognising faces across pose and age. In Automatic Face and Gesture Recognition, 2018

    2018

  2. [10]

    Faceposenet: Making a case for landmark-free face alignment

    Feng-Ju Chang, Anh Tran, Tal Hassner, Iacopo Masi, Ram Nevatia, and G ´erard Medioni. Faceposenet: Making a case for landmark-free face alignment. In Proc. Int. Conf. Com- put. Vision Workshops, 2017

    2017

  3. [11]

    Deep, landmark-free fame: Face alignment, modeling, and expression estimation

    Feng-Ju Chang, Anh Tuan Tran, Tal Hassner, Iacopo Masi, Ram Nevatia, and G ´erard Medioni. Deep, landmark-free fame: Face alignment, modeling, and expression estimation. Int. J. Comput. Vision, 127(6-7):930–956, 2019

    2019

  4. [12]

    Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems.arXiv preprint arXiv:1512.01274, 2015

    Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems.arXiv preprint arXiv:1512.01274, 2015

    2015 arXiv

  5. [13]

    The loss surfaces of multilayer networks

    Anna Choromanska, Mikael Henaff, Michael Mathieu, G´erard Ben Arous, and Yann LeCun. The loss surfaces of multilayer networks. In Artificial Intelligence and Statistics, pages 192–204, 2015

    2015

  6. [14]

    Elements of informa- tion theory

    Thomas M Cover and Joy A Thomas. Elements of informa- tion theory. John Wiley & Sons, 2012

    2012

  7. [15]

    On measuring the iconicity of a face

    Prithviraj Dhar, Carlos Castillo, and Rama Chellappa. On measuring the iconicity of a face. In Winter Conf. on App. of Comput. Vision, pages 2137–2145. IEEE, 2019

    2019

  8. [16]

    Class rectifi- cation hard mining for imbalanced deep learning

    Qi Dong, Shaogang Gong, and Xiatian Zhu. Class rectifi- cation hard mining for imbalanced deep learning. In Proc. Conf. Comput. Vision Pattern Recognition , pages 1851– 1860, 2017

    2017

  9. [17]

    Pattern classification

    Richard O Duda, Peter E Hart, and David G Stork. Pattern classification. John Wiley & Sons, 2012

    2012

  10. [18]

    Learning disentangled joint continuous and discrete representations

    Emilien Dupont. Learning disentangled joint continuous and discrete representations. In Neural Inform. Process. Syst. , pages 708–718, 2018

    2018

  11. [19]

    Towards a neural statistician

    Harrison Edwards and Amos Storkey. Towards a neural statistician. arXiv preprint arXiv:1606.02185, 2016

    2016 arXiv

  12. [20]

    Large- scale weakly-supervised pre-training for video action recog- nition

    Deepti Ghadiyaram, Du Tran, and Dhruv Mahajan. Large- scale weakly-supervised pre-training for video action recog- nition. In Proc. Conf. Comput. Vision Pattern Recognition, pages 12046–12055, 2019

    2019

  13. [21]

    MS-Celeb-1M: A dataset and benchmark for large scale face recognition

    Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. MS-Celeb-1M: A dataset and benchmark for large scale face recognition. In European Conf. Comput. Vi- sion. Springer, 2016

    2016

  14. [22]

    Heterogeneous face attribute estimation: A deep multi-task learning approach

    Hu Han, Anil K Jain, Fang Wang, Shiguang Shan, and Xilin Chen. Heterogeneous face attribute estimation: A deep multi-task learning approach. Trans. Pattern Anal. Mach. Intell., 40(11):2597–2609, 2018

    2018

  15. [23]

    Attributes for improved attributes: A multi-task network utilizing implicit and ex- plicit relationships for facial attribute classification

    Emily M Hand and Rama Chellappa. Attributes for improved attributes: A multi-task network utilizing implicit and ex- plicit relationships for facial attribute classification. In AAAI Conf. on Artificial Intelligence, 2017

    2017

  16. [24]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proc. Int. Conf. Comput. Vision, pages 2961–2969, 2017

    2017

  17. [25]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. Conf. Comput. Vision Pattern Recognition, June 2016

    2016

  18. [26]

    Learning deep representation for imbalanced classi- fication

    Chen Huang, Yining Li, Chen Change Loy, and Xiaoou Tang. Learning deep representation for imbalanced classi- fication. In Proc. Conf. Comput. Vision Pattern Recognition, pages 5375–5384, 2016

    2016

  19. [27]

    Registration-free face-ssd: Single shot analysis of smiles, fa- cial attributes, and affect in the wild

    Youngkyoon Jang, Hatice Gunes, and Ioannis Patras. Registration-free face-ssd: Single shot analysis of smiles, fa- cial attributes, and affect in the wild. Comput. Vision Image Understanding, 2019

    2019

  20. [28]

    Deep cross residual learn- ing for multitask visual recognition

    Brendan Jou and Shih-Fu Chang. Deep cross residual learn- ing for multitask visual recognition. In Int. Conf. Multime- dia, pages 998–1007. ACM, 2016

    2016

  21. [29]

    Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics

    Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 7482– 7491, 2018

    2018

  22. [30]

    Ubernet: Training a universal convo- lutional neural network for low-, mid-, and high-level vi- sion using diverse datasets and limited memory

    Iasonas Kokkinos. Ubernet: Training a universal convo- lutional neural network for low-, mid-, and high-level vi- sion using diverse datasets and limited memory. In Proc. Conf. Comput. Vision Pattern Recognition , pages 6129– 6138, 2017

    2017

  23. [31]

    Asymmetric multi-task learning based on task relatedness and loss

    Giwoong Lee, Eunho Yang, and Sung Hwang. Asymmetric multi-task learning based on task relatedness and loss. InInt. Conf. Mach. Learning, pages 230–238, 2016. 9

    2016

  24. [32]

    Age and gender classification us- ing convolutional neural networks

    Gil Levi and Tal Hassner. Age and gender classification us- ing convolutional neural networks. In Proc. Conf. Comput. Vision Pattern Recognition Workshops, June 2015

    2015

  25. [33]

    Exploring disentangled feature represen- tation beyond face identification

    Yu Liu, Fangyin Wei, Jing Shao, Lu Sheng, Junjie Yan, and Xiaogang Wang. Exploring disentangled feature represen- tation beyond face identification. In Proc. Conf. Comput. Vision Pattern Recognition, pages 2080–2089, 2018

    2018

  26. [34]

    Detach and adapt: Learning cross-domain disentangled deep repre- sentation

    Yen-Cheng Liu, Yu-Ying Yeh, Tzu-Chien Fu, Sheng-De Wang, Wei-Chen Chiu, and Yu-Chiang Frank Wang. Detach and adapt: Learning cross-domain disentangled deep repre- sentation. In Proc. Conf. Comput. Vision Pattern Recogni- tion, pages 8867–8876, 2018

    2018

  27. [35]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proc. Int. Conf. Comput. Vision, 2015

    2015

  28. [36]

    Object recognition from local scale-invariant features

    David G Lowe. Object recognition from local scale-invariant features. In Proc. Int. Conf. Comput. Vision , page 1150, 1999

    1999

  29. [37]

    Fully-adaptive feature shar- ing in multi-task networks with applications in person at- tribute classification

    Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris. Fully-adaptive feature shar- ing in multi-task networks with applications in person at- tribute classification. In Proc. Conf. Comput. Vision Pattern Recognition, pages 5334–5343, 2017

    2017

  30. [38]

    Exploring the limits of weakly supervised pretraining

    Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. Exploring the limits of weakly supervised pretraining. In European Conf. Comput. Vision, pages 181–196, 2018

    2018

  31. [39]

    Domain adaptation: Learning bounds and algorithms

    Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In Conference on Learning Theory, 2009

    2009

  32. [40]

    I. Masi, F. J. Chang, J. Choi, S. Harel, J. Kim, K. Kim, J. Leksut, S. Rawls, Y . Wu, T. Hassner, W. AbdAlmageed, G. Medioni, L. P. Morency, P. Natarajan, and R. Nevatia. Learn- ing pose-aware models for pose-invariant face recognition in the wild. Trans. Pattern Anal. Mach. I...

    2018

  33. [41]

    Face-specific data augmentation for un- constrained face recognition

    Iacopo Masi, Anh Tuan Tran, Tal Hassner, Gozde Sahin, and G´erard Medioni. Face-specific data augmentation for un- constrained face recognition. Int. J. Comput. Vision, 127(6- 7):642–667, 2019

    2019

  34. [42]

    On the method of bounded differences

    Colin McDiarmid. On the method of bounded differences. Surveys in combinatorics, 141(1):148–188, 1989

    1989

  35. [43]

    Cross-stitch networks for multi-task learning

    Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Mar- tial Hebert. Cross-stitch networks for multi-task learning. In Proc. Conf. Comput. Vision Pattern Recognition , pages 3994–4003, 2016

    2016

  36. [44]

    Toward understanding catastrophic forgetting in continual learning

    Cuong V Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, and Stefano Soatto. Toward understanding catastrophic forgetting in continual learning. arXiv:1908.01091, 2019

    1908 arXiv

  37. [45]

    Variational continual learning

    Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. Variational continual learning. In Int. Conf. on Learning Representations, 2018

    2018

  38. [46]

    The loss surface of deep and wide neural networks

    Quynh Nguyen and Matthias Hein. The loss surface of deep and wide neural networks. In Int. Conf. Mach. Learning , pages 2603–2612, 2017

    2017

  39. [47]

    On face segmentation, face swapping, and face perception

    Yuval Nirkin, Iacopo Masi, Anh Tran Tuan, Tal Hassner, and Gerard Medioni. On face segmentation, face swapping, and face perception. In Int. Conf. on Automatic Face and Gesture Recognition, pages 98–105. IEEE, 2018

    2018

  40. [48]

    A survey on transfer learn- ing

    Sinno Jialin Pan and Qiang Yang. A survey on transfer learn- ing. Trans. Knowledge and Data Eng. , 22(10):1345–1359, 2010

    2010

  41. [49]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. J. Mach. Learning Researc...

    2011

  42. [50]

    Hy- perface: A deep multi-task learning framework for face de- tection, landmark localization, pose estimation, and gender recognition

    Rajeev Ranjan, Vishal M Patel, and Rama Chellappa. Hy- perface: A deep multi-task learning framework for face de- tection, landmark localization, pose estimation, and gender recognition. Trans. Pattern Anal. Mach. Intell., 41(1):121– 135, 2019

    2019

  43. [51]

    An all-in-one convolutional neural net- work for face analysis

    Rajeev Ranjan, Swami Sankaranarayanan, Carlos D Castillo, and Rama Chellappa. An all-in-one convolutional neural net- work for face analysis. In Int. Conf. on Automatic Face and Gesture Recognition, pages 17–24. IEEE, 2017

    2017

  44. [52]

    CHILD: A first step towards continual learn- ing

    Mark B Ring. CHILD: A first step towards continual learn- ing. Mach. Learn., 28(1):77–104, 1997

    1997

  45. [53]

    Dex: Deep expectation of apparent age from a single image

    Rasmus Rothe, Radu Timofte, and Luc Van Gool. Dex: Deep expectation of apparent age from a single image. InProc. Int. Conf. Comput. Vision Workshops, pages 10–15, 2015

    2015

  46. [54]

    Be- yond sharing weights for deep domain adaptation

    Artem Rozantsev, Mathieu Salzmann, and Pascal Fua. Be- yond sharing weights for deep domain adaptation. Trans. Pattern Anal. Mach. Intell., 41(4):801–814, 2019

    2019

  47. [55]

    Moon: A mixed objective optimization network for the recognition of facial attributes

    Ethan M Rudd, Manuel G ¨unther, and Terrance E Boult. Moon: A mixed objective optimization network for the recognition of facial attributes. In European Conf. Comput. Vision, pages 19–35. Springer, 2016

    2016

  48. [56]

    Meta-learning with latent embedding optimization

    Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Had- sell. Meta-learning with latent embedding optimization. In Int. Conf. on Learning Representations, 2019

    2019

  49. [57]

    Learning with kernels: support vector machines, regularization, optimiza- tion, and beyond

    Bernhard Scholkopf and Alexander J Smola. Learning with kernels: support vector machines, regularization, optimiza- tion, and beyond. MIT press, 2001

    2001

  50. [58]

    Refining architectures of deep convolutional neural networks

    Sukrit Shankar, Duncan Robertson, Yani Ioannou, Antonio Criminisi, and Roberto Cipolla. Refining architectures of deep convolutional neural networks. In Proc. Conf. Comput. Vision Pattern Recognition, pages 2212–2220, 2016

    2016

  51. [59]

    Unbiased look at dataset bias

    A Torralba and AA Efros. Unbiased look at dataset bias. In Proc. Conf. Comput. Vision Pattern Recognition , pages 1521–1528. IEEE Computer Society, 2011

    2011

  52. [60]

    Extreme 3D face reconstruc- tion: Looking past occlusions

    Anh Tuan Tran, Tal Hassner, Iacopo Masi, Eran Paz, Yuval Nirkin, and G ´erard Medioni. Extreme 3D face reconstruc- tion: Looking past occlusions. In Proc. Conf. Comput. Vision Pattern Recognition, 2018

    2018

  53. [61]

    Conditional similarity networks

    Andreas Veit, Serge Belongie, and Theofanis Karaletsos. Conditional similarity networks. In Proc. Conf. Comput. Vi- sion Pattern Recognition, pages 830–838, 2017

    2017

  54. [62]

    Cos- face: Large margin cosine loss for deep face recognition

    Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cos- face: Large margin cosine loss for deep face recognition. 10 In Proc. Conf. Comput. Vision Pattern Recognition , pages 5265–5274, 2018

    2018

  55. [63]

    Walk and learn: Facial attribute representation learning from egocen- tric video and contextual data

    Jing Wang, Yu Cheng, and Rogerio Schmidt Feris. Walk and learn: Facial attribute representation learning from egocen- tric video and contextual data. InProc. Conf. Comput. Vision Pattern Recognition, pages 2295–2304, 2016

    2016

  56. [64]

    Towards unified depth and seman- tic prediction from a single image

    Peng Wang, Xiaohui Shen, Zhe Lin, Scott Cohen, Brian Price, and Alan L Yuille. Towards unified depth and seman- tic prediction from a single image. In Proc. Conf. Comput. Vision Pattern Recognition, pages 2800–2809, 2015

    2015

  57. [65]

    A survey of transfer learning

    Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. A survey of transfer learning. Journal of Big Data , 3(1):9, 2016

    2016

  58. [66]

    Welinder, S

    P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Be- longie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technol- ogy, 2010

    2010

  59. [67]

    Zero-shot learning-a comprehensive evalu- ation of the good, the bad and the ugly

    Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning-a comprehensive evalu- ation of the good, the bad and the ugly. Trans. Pattern Anal. Mach. Intell., 2018

    2018

  60. [68]

    Deep multi-task representation learning: A tensor factorisation approach

    Yongxin Yang and Timothy Hospedales. Deep multi-task representation learning: A tensor factorisation approach. In Int. Conf. on Learning Representations, 2017

    2017

  61. [69]

    Transfer learning via learning to transfer

    Wei Ying, Yu Zhang, Junzhou Huang, and Qiang Yang. Transfer learning via learning to transfer. InInt. Conf. Mach. Learning, pages 5072–5081, 2018

    2018

  62. [70]

    How transferable are features in deep neural networks? In Neural Inform

    Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Neural Inform. Process. Syst., pages 3320–3328, 2014

    2014

  63. [71]

    Taskonomy: Disentangling task transfer learning

    Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proc. Conf. Comput. Vision Pattern Recognition, pages 3712–3722, 2018

    2018

  64. [72]

    Understanding deep learning re- quires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning re- quires rethinking generalization. In Int. Conf. on Learning Representations, 2017

    2017

  65. [73]

    A modulation module for multi-task learning with applications in image retrieval

    Xiangyun Zhao, Haoxiang Li, Xiaohui Shen, Xiaodan Liang, and Ying Wu. A modulation module for multi-task learning with applications in image retrieval. InEuropean Conf. Com- put. Vision, pages 401–416, 2018. A. Proof of theorem 1 From the definition of ~Trf(T Z→ T Y ), we have:...

    2018

  66. [74]

    (6) in the paper)

    Loop through the training labels of both tasks T Z and T Y and compute the empirical joint distribution ˆP (y, z) by counting (Eq. (6) in the paper)

  67. [75]

    (17) above

    Loop through the training labels again and compute the CE using Eq. (17) above. That is, H(Y|Z) =− 1 n n∑ i=1 log ˆP (yi|zi). Thus, computing the CE only requires running two loops through the training labels. This process is computationally efficient. In the most extreme case,...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.