Pith. sign in

REVIEW 4 major objections 4 minor 45 references

Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that multi-task learning over heterogeneous unsupervised tasks can be prevented from deviating to one dominant task by combining a metric-based regularizer with a temporal task ensemble (TTE), and that the resulting…

desk verdict Useful empirical task-combination study, but the headline TTE mechanism is not implemented by its own equations: the stress-test concern holds up, and the KLD regularizer's target v_p is never defined. read the letter →

arxiv 1908.11024 v1 pith:TSYK5UUJ submitted 2019-08-29 cs.CV

Dae Ha Kim , Seung Hyun Lee , Byung Cheol Song This is my paper · ORCID
classification cs.CV
keywords heterogeneousunsupervisedtasksmulti-tasklearningtemporaltaskensemblemetric-basedregularizationself-supervisedknowledgetransferfeaturerepresentationobjectdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to fix a bias problem in unsupervised multi-task learning: when several self-supervised tasks share one encoder, the task with the strongest gradient can dominate and the shared feature representation becomes specialized instead of general. It claims that adding a Kullback-Leibler divergence regularizer on the latent feature distribution, plus a temporal task ensemble that averages encoder weights with adaptive task-specific coefficients, keeps all tasks roughly equal in influence. If this works, one can pre-train an encoder on unlabeled data with several tasks and then transfer it to a target task, without the usual bias that harms downstream performance. The paper reports that this scheme beats the MSVL baseline on image classification, object detection, and embedding clustering, with the gains most consistent when four or five tasks are combined.

What carries the argument

The central mechanism is the temporal task ensemble applied to the encoder weights immediately before the pooling layer. At each epoch $t$, it computes the temporal gradient $\Delta\varphi^t_k = \|\varphi^t_k - \varphi^{t-1}\|_1$ for each task $k$, the task gradient $\Delta\varphi^t_o = \|\varphi^t_r - \varphi^t_j\|_{\text{CAD}}$ using Canberra distance, and updates the encoder as $\varphi^t = \varphi^{t-1} + \sum_k \alpha^t_k \Delta\varphi^t_k + \beta^t \Delta\varphi^t_o$, with coefficients $\alpha^t_k, \beta^t$ shrunk when the corresponding loss increases. A moving average over $T=5$ epochs produces the final encoder. The companion regularizer is the KLD term $\Omega = D_{KL}(v \| v_p)$ that forces the latent feature distribution $v$ toward a reference; the ablation shows several other distribution metrics behave similarly.

What would settle it

A concrete test would be to record per-task loss gradients on the encoder's shared parameters during TTE training and compare which task has the largest gradient norm with which task has the largest temporal gradient; if the task with the largest $\Delta\varphi^t_k$ is not the one dominating the representation, TTE's balancing premise fails and the reported gains over MSVL should not reproduce.

Watch

Extended reading notes

Core claim

On the paper's own terms, the core discovery is that task imbalance in multi-task unsupervised learning can be detected and counteracted using weight-space statistics: the L1 distance between a task's encoder weights across consecutive epochs measures that task's influence at that epoch, and the Canberra distance between two tasks' encoder weights measures the overall dynamic range of task influence. By adaptively blending these signals into a weighted update of the shared encoder, with coefficients driven by loss decreases, the network is kept from drifting toward one task. The resulting encoder, averaged over five epochs, transfers to target tasks better than the MSVL baseline in all three target settings tested, with the largest reported gains on object detection, where the five-task TTE encoder reaches 69.35 mAP versus 68.75 for MSVL on VOC0712.

Load-bearing premise

The balancing scheme assumes that the L1 temporal gradient and Canberra task gradient actually measure each task's influence on the shared encoder, and that adjusting them evens out task dominance; if weight-space distances do not track true task influence, the entire ensemble has no grounding.

Editorial extensions

If this is right

  • TTE-based pre-training yields higher classification accuracy than MSVL on CIFAR-10s and STL10 under both soft-target and FSP DNN transfer, for example 71.38 and 68.06 percent versus 70.78 and 66.42 percent for the four-task setting.
  • For object detection with SSD300 on VOC0712, the five-task TTE encoder reaches 69.35 mAP versus 68.75 for MSVL, showing the balancing pays off on detection.
  • The distribution metric is not the load-bearing choice: Hellinger and Wasserstein distances perform comparably to KLD in the ablation, so any constraint on the latent feature map seems to help.
  • In knowledge distillation on CIFAR-10, TTE-based unsupervised pre-training (90.43 percent with FSP DNN transfer) exceeds supervised classification pre-training (88.70 percent), so balancing unsupervised tasks can beat a supervised source.
  • Removing the task gradient hurts less than removing the distribution metric, so the regularizer is the stronger of the two components in their design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the balancing truly reflects task influence, the same temporal-gradient trick could be applied beyond static multi-task learning, for instance to continual learning or to reweighting tasks in a stream, since it uses only weight differences between consecutive epochs.
  • A testable extension is to probe the frozen encoder learned by TTE with a linear classifier for each pretext task; the balancing claim would be confirmed if TTE's encoder shows more uniform per-task separability than MSVL's.
  • Because the reference distribution $v_p$ in the KLD regularizer is never defined, the regularizer's effect may actually come from any stabilizer on the latent distribution, such as batch-normalization-like feature normalization, and a simple variance penalty could be tried in its place.
  • The reported gains are largest when the task set has four or five tasks; this suggests an optimal diversity level for the ensemble, which the authors do not explore, so a follow-up could vary task count systematically and measure transfer performance.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes two techniques for multi-task self-supervised pretraining: a distribution-metric (KLD) regularization term intended to stabilize encoder feature maps, and a temporal task ensemble (TTE) that adaptively fuses per-task encoder updates to prevent the network from being biased to a specific task. The pretrained encoder is then transferred to target tasks (classification, object detection, deep embedding clustering) and compared with MSVL [3]. Experiments report accuracy and mAP improvements of TTE over MSVL for most settings, along with ablations over different distribution metrics and encoder architectures.

Significance. If correct, the idea of balancing task influence in unsupervised multi-task learning would be a useful contribution, and the ablation over distribution metrics (KLD, JSD, Hellinger, Wasserstein, Jeffrey, chi-squared) plus the use of three transfer protocols is a well-motivated experimental design. However, the central TTE update in Eqs. (7)-(9) as written cannot perform the claimed fusion, and the reference distribution v_p in Eq. (3) is never defined; these are not mere presentation issues. Since the experimental numbers cannot be tied to a well-specified algorithm, the significance of the reported gains cannot be assessed. The paper also provides no variance or significance information for the three-trial averages and does not validate the self-implemented MSVL baseline, further limiting the support for the headline claim.

major comments (4)
  1. [§3.3, Eqs. (7)-(9)] Eq. (7) defines Δφ^t_k as ||φ^t_k − φ^{t−1}||_1, which is a nonnegative scalar, and Eq. (8) defines Δφ^t_o as a Canberra distance scalar; substituting these scalars into Eq. (9) yields φ^t = φ^{t−1} + (Σ_k α^t_k Δφ^t_k + β^t Δφ^t_o)·1, a uniform shift of every encoder weight with no task-specific direction. Even under the alternative reading of 'element-wise L1 distance' as an element-wise absolute difference, the update adds absolute differences without sign information and is not an ensemble of task-specific weight updates. The central balancing mechanism described in Figure 3 and Section 3.3 is therefore not actually specified by the equations; the authors must correct the definition (e.g., using the difference tensor φ^t_k − φ^{t−1} in Eq. (9)) and re-evaluate the experiments.
  2. [§3.1, Eq. (3)] The regularization term is D_KL(v || v_p), but v_p is never defined anywhere in the manuscript. The text states that a transformation filter F converts feature maps z into vectors v and that learning reduces the metric distance between v and v_p, but no equation or construction specifies v_p (e.g., uniform distribution, batch statistics, or a learned prior). Without v_p, Eq. (3) is not a computable loss, so the claimed effect of the regularization term on the ablations in Table 5 cannot be verified.
  3. [Supplementary, Eqs. (11)-(13)] The adaptive coefficient update in the supplementary material is inconsistent with its stated purpose. Since [·]₋ = min(0,·), both m and n in Eq. (13) are non-positive; hence α^t_k = α^{t−1}_k/(1+m) is greater than or equal to α^{t−1}_k, with equality when the loss increases and strict increase when the loss decreases. Thus the coefficients can only grow or stay constant over epochs, which does not 'limit the influence of the temporal ensemble process when the loss gap becomes large' and can lead to exponential growth. This needs to be fixed or clarified with a different formula (e.g., using the absolute or positive part of the loss difference).
  4. [§4, Tables 2-4] All reported numbers are averages of three trials with no standard deviation, variance, or significance tests, and many TTE versus MSVL differences are around 0.3-1 percentage points; several cells in Table 2 show MSVL ahead (e.g., ILSVRC2012 FSP with (Jig++)+Col+Seg+Rcn on C10s: TTE 70.45 vs MSVL 70.65; Places365 FSP with Jig+Col on STL10: TTE 63.10 vs MSVL 65.21). Moreover, the MSVL baseline is 'implemented ourselves' without any validation against the numbers in [3]. The claim that TTE outperforms the SOTA baseline is therefore not statistically supported; the authors should report error bars or paired per-trial comparisons and demonstrate that their MSVL reproduction matches the original method.
minor comments (4)
  1. [Section 2.1 title] The title of Section 2.1, 'Unsupervised Taks', has a typo and should be 'Unsupervised Tasks'; also, the word 'un-/self-supervised' is hyphenated inconsistently throughout the paper.
  2. [Equations numbering] The text refers to 'Eq. (12)', 'Eq. (13)', and 'Eq. (14)' when the corresponding displayed equations are numbered (2), (3), and (4), respectively; please renumber the equations consistently.
  3. [Table 5] In Table 5, the row labels and the '!' symbol are unclear; please clarify what the '!' indicates (presumably presence of the task gradient) and define it in the caption.
  4. [Section 4.4] The qualitative claim that TTE yields higher recall, NMI, and ARI than MSVL is made in the text, but no numeric values are reported; please include the quantitative metrics to substantiate the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the claimed gains are measured against an external MSVL baseline with hand-set hyperparameters; no load-bearing self-citation or fitted-input-as-prediction chain is present.

full rationale

The paper's central comparison is TTE against MSVL [3], an externally published method that the authors state they implemented themselves ('In case of MSVL, we implemented ourselves' in Tables 2-3). No parameter is fitted to the target results: alpha_0, beta_0, lambda, and T are fixed values set before the experiments (alpha_0=0.4..., beta_0=5e-3, lambda=1e-3, T=5). The claimed contributions, the KLD regularizer and the TTE update, are not defined in terms of the target-task accuracies they predict, so there is no self-definitional or fitted-input-called-prediction loop. The paper contains no uniqueness theorem or prior-work ansatz by the same authors; the cited unsupervised tasks and transfer methods are standard external works. The noted weaknesses are non-circular: the reference distribution v_p in Eq. (3) is never defined, and Eq. (7) as written returns a scalar L1 norm while Eq. (9) adds it to a weight tensor, which would make the TTE update unable to carry task-specific direction. These are correctness/reproducibility defects, not circular reductions: they do not make the claimed output equal to the input by construction. Task selection 'experimentally selected ... that demonstrate synergy' (Sec. 1) is a selection-bias concern, not a circularity in the derivation chain. Accordingly the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central scheme rests on several hand-set coefficients, lambda, T, alpha_0, and beta_0, and on heuristic assumptions that weight-space distances and loss gaps reflect task influence. No invented physical or mathematical entities are introduced; TTE and the regularization term are training procedures.

free parameters (5)
  • lambda = 1e-3
    Balance factor for the reconstruction regularization term L_reg in Eq. (6), set without reported sensitivity analysis.
  • T = 5
    Moving-average window for the final encoder weights in Eq. (10), hand-set.
  • alpha_0_r = 0.4
    Initial TTE coefficient for the reconstruction task in Eq. (11), reported in the supplementary material.
  • alpha_0_s, alpha_0_c, alpha_0_j = 0.2 each
    Initial TTE coefficients for segmentation, colorization, and jigsaw tasks.
  • beta_0 = 5e-3
    Initial task-gradient coefficient in Eq. (12).
assumptions (5)
  • domain assumption The L1 temporal gradient in Eq. (7) measures the impact of task k on the encoder network.
    Section 3.3 defines Delta_phi^k_t this way without evidence or derivation; the entire TTE balancing depends on it.
  • domain assumption The Canberra-distance task gradient in Eq. (8) between first and last task weights represents the dynamic range of all tasks.
    Section 3.3 states this interpretation and uses it in Eq. (9).
  • ad hoc to paper A reference distribution v_p exists for the KLD regularizer in Eq. (3).
    v_p is used in the objective but never defined, so its existence and choice are assumed.
  • domain assumption The four chosen unsupervised tasks are heterogeneous and their losses can be combined by simple summation in Eq. (4).
    The paper does not study loss scales or gradient magnitudes across tasks beyond the heuristic coefficients.
  • domain assumption The adaptive coefficient rule in Eqs. (11) to (13) balances task influence.
    No theoretical or empirical demonstration is provided, and the rule as written appears inconsistent with its stated purpose.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks." pith.science (2026). https://pith.science/paper/TSYK5UUJ

@misc{pith2026190811024,
  author       = {Pith},
  title        = {Pith review of: Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TSYK5UUJ}},
  note         = {Machine review of arXiv:1908.11024}
}
read the original abstract

One of the ways to improve the performance of a target task is to learn the transfer of abundant knowledge of a pre-trained network. However, learning of the pre-trained network requires high computation capability and large-scale labeled dataset. To mitigate the burden of large-scale labeling, learning in un/self-supervised manner can be a solution. In addition, using unsupervised multi-task learning, a generalized feature representation can be learned. However, unsupervised multi-task learning can be biased to a specific task. To overcome this problem, we propose the metric-based regularization term and temporal task ensemble (TTE) for multi-task learning. Since these two techniques prevent the entire network from learning in a state deviated to a specific task, it is possible to learn a generalized feature representation that appropriately reflects the characteristics of each task without biasing. Experimental results for three target tasks such as classification, object detection and embedding clustering prove that the TTE-based multi-task framework is more effective than the state-of-the-art (SOTA) method in improving the performance of a target task.

Figures

Figures reproduced from arXiv: 1908.11024 by the authors.

Figure 1
Figure 1. The main concept of the proposed method. Unbalanced [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of the proposed method. The target network solves the target tasks by transferring the knowledge of the source [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The operation of TTE at epoch t. The order of the tasks used in this TTE process can be changed at every epoch. fk indicates the network corresponding to task k, and con￾sists of a common encoder network (fenc) and a header net￾work (fhk ) as in Eq. (12). fk is learned based on x and weights set W = {φ, θ1, ..., θk}. φ in W stands for train￾able weights of the encoder network, e.g., AlexNet, VGG, and ResNet [16, 34,… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Example retrieval results on STL10 dataset. (Upper) us [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 36 canonical work pages

  1. [3]

    Doersch and A

    C. Doersch and A. Zisserman. Multi-task self-supervised visual learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 2051–2060, 2017. 1, 2, 3, 6, 7

    work page 2017

  2. [1]

    S. Chen, C. Zhang, and M. Dong. Coupled end-to-end trans- fer learning with generalized fisher information. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4329–4338, 2018. 5, 7

    work page 2018

  3. [2]

    Coates, A

    A. Coates, A. Ng, and H. Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intel- ligence and statistics, pages 215–223, 2011. 5

    work page 2011

  4. [4]

    Erhan, Y

    D. Erhan, Y . Bengio, A. Courville, P.-A. Manzagol, P. Vin- cent, and S. Bengio. Why does unsupervised pre-training help deep learning? Journal of Machine Learning Research, 11(Feb):625–660, 2010. 1

    work page 2010

  5. [5]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) chal- lenge. International journal of computer vision, 88(2):303– 338, 2010. 5

    work page 2010

  6. [6]

    A. L. Gibbs and F. E. Su. On choosing and bounding prob- ability metrics. International statistical review, 70(3):419– 435, 2002. 4, 7, 8

    work page 2002

  7. [7]

    Gkioxari, R

    G. Gkioxari, R. Girshick, and J. Malik. Contextual action recognition with r* cnn. In Proceedings of the IEEE inter- national conference on computer vision , pages 1080–1088,

    work page

  8. [8]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014. 2

    work page 2014

Show all 45 references
  1. [9]

    Grandvalet and Y

    Y . Grandvalet and Y . Bengio. Semi-supervised learning by entropy minimization. In Advances in neural information processing systems, pages 529–536, 2005. 4

    2005

  2. [10]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- ing for image recognition. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 770–778, 2016. 1, 4, 7

    2016

  3. [11]

    Hinton, O

    G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 5, 7, 11

    2015 arXiv

  4. [12]

    Kanezaki

    A. Kanezaki. Unsupervised image segmentation by back- propagation. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1543–1547. IEEE, 2018. 2

    2018

  5. [13]

    Kendall, Y

    A. Kendall, Y . Gal, and R. Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and seman- tics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7482–7491, 2018. 1, 2, 3

    2018

  6. [14]

    D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  7. [15]

    Krizhevsky and G

    A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009. 5

    2009

  8. [16]

    Krizhevsky, I

    A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pages 1097–1105, 2012. 1, 4, 6, 7

    2012

  9. [17]

    Larsson, M

    G. Larsson, M. Maire, and G. Shakhnarovich. Learning rep- resentations for automatic colorization. In European Con- ference on Computer Vision, pages 577–593. Springer, 2016. 2

    2016

  10. [18]

    Litjens, T

    G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, and C. I. S ´anchez. A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88, 2017. 2

    2017

  11. [19]

    W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.- Y . Fu, and A. C. Berg. Ssd: Single shot multibox detector. In European conference on computer vision , pages 21–37. Springer, 2016. 6

    2016

  12. [20]

    Long and J

    M. Long and J. Wang. Learning multiple tasks with deep relationship networks. arXiv preprint arXiv:1506.02117, 2,

    arXiv

  13. [21]

    D. J. MacKay and D. J. Mac Kay. Information theory, infer- ence and learning algorithms . Cambridge university press,

  14. [22]

    Maninis, J

    K.-K. Maninis, J. Pont-Tuset, P. Arbel ´aez, and L. Van Gool. Deep retinal image understanding. In International confer- ence on medical image computing and computer-assisted in- tervention, pages 140–148. Springer, 2016. 2

    2016

  15. [23]

    Manning, P

    C. Manning, P. Raghavan, and H. Sch ¨utze. Introduction to information retrieval. Natural Language Engineering , 16(1):100–103, 2010. 8

    2010

  16. [24]

    Misra, A

    I. Misra, A. Shrivastava, A. Gupta, and M. Hebert. Cross- stitch networks for multi-task learning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 3994–4003, 2016. 1, 3

    2016

  17. [25]

    T. N. Mundhenk, D. Ho, and B. Y . Chen. Improvements to context based self-supervised learning. In CVPR, pages 9339–9348, 2018. 1, 2

    2018

  18. [26]

    Ng et al

    A. Ng et al. Sparse autoencoder. CS294A Lecture notes , 72(2011):1–19, 2011. 2, 4

    2011

  19. [27]

    Noroozi and P

    M. Noroozi and P. Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. InEuropean Con- ference on Computer Vision, pages 69–84. Springer, 2016. 1, 2, 4

    2016

  20. [28]

    Noroozi, A

    M. Noroozi, A. Vinjimoor, P. Favaro, and H. Pirsiavash. Boosting self-supervised learning via knowledge transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9359–9367, 2018. 1, 2

    2018

  21. [29]

    Pinto and A

    L. Pinto and A. Gupta. Learning to push by grasping: Us- ing multiple tasks for effective learning. In 2017 IEEE In- ternational Conference on Robotics and Automation (ICRA), pages 2161–2168. IEEE, 2017. 2, 3

    2017

  22. [30]

    Qu, T.-T

    Y . Qu, T.-T. Wong, and P.-A. Heng. Manga colorization. In ACM Transactions on Graphics (TOG), volume 25, pages 1214–1220. ACM, 2006. 2

    2006

  23. [31]

    Robbins and S

    H. Robbins and S. Monro. A stochastic approximation method. The annals of mathematical statistics , pages 400– 407, 1951. 6 9

    1951

  24. [32]

    Ronneberger, P

    O. Ronneberger, P. Fischer, and T. Brox. U-net: Convo- lutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention , pages 234–241. Springer,

  25. [33]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision , 115(3):211–252,

  26. [34]

    Simonyan and A

    K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 1, 4, 7

    2014 arXiv

  27. [35]

    Vincent, H

    P. Vincent, H. Larochelle, Y . Bengio, and P.-A. Manzagol. Extracting and composing robust features with denoising au- toencoders. In Proceedings of the 25th international confer- ence on Machine learning , pages 1096–1103. ACM, 2008. 2

    2008

  28. [36]

    Xia and B

    X. Xia and B. Kulis. W-net: A deep model for fully unsupervised image segmentation. arXiv preprint arXiv:1711.08506, 2017. 2, 4

    2017 arXiv

  29. [37]

    J. Xie, R. Girshick, and A. Farhadi. Unsupervised deep em- bedding for clustering analysis. In International conference on machine learning, pages 478–487, 2016. 7, 8

    2016

  30. [38]

    J. Yim, D. Joo, J. Bae, and J. Kim. A gift from knowl- edge distillation: Fast optimization, network minimization and transfer learning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4133– 4141, 2017. 5, 7, 11

    2017

  31. [39]

    A. R. Zamir, T. Wekel, P. Agrawal, C. Wei, J. Malik, and S. Savarese. Generic 3d representation via pose estimation and matching. In European Conference on Computer Vision, pages 535–553. Springer, 2016. 2, 3

    2016

  32. [40]

    L. Zhang. Transfer adaptation learning: A decade survey. arXiv preprint arXiv:1903.04687, 2019. 5

    1903 arXiv

  33. [41]

    Zhang, P

    R. Zhang, P. Isola, and A. A. Efros. Colorful image col- orization. In European conference on computer vision, pages 649–666. Springer, 2016. 1, 2, 4

    2016

  34. [42]

    Zhang and Q

    Y . Zhang and Q. Yang. A survey on multi-task learning. arXiv preprint arXiv:1707.08114, 2017. 1

    2017 arXiv

  35. [43]

    B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In Advances in neural information processing sys- tems, pages 487–495, 2014. 5 10 Supplementary Materials

    2014

  36. [44]

    TTE coefficients The two coefficients used during temporal task ensemble (TTE), i.e., αt k andβt are adaptively determined based on the loss values of the current epoch as well as the previous epoch (see. Eq. (11)-(13)). αt k =αt−1 k /(1 +m) (11) βt =βt−1/(1 +n) (12) where m = [...

  37. [45]

    Basically we use the typical knowledge transfer methods [11, 38]

    Knowledge Transfer Methods This section discusses in detail how to transfer domain knowledge. Basically we use the typical knowledge transfer methods [11, 38]. However, since we use the unsupervised learning scheme in source domain, we add some variations to the existing knowl...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.