Pith. sign in

REVIEW 4 major objections 5 minor 32 references

VLCD: Vision-Language Contrastive Distillation for Accurate and Efficient Automatic Placenta Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a text-anchored contrastive-distillation procedure lets small image encoders match or beat a ResNet-50 teacher on placenta-pathology tasks while running close to four times faster.

desk verdict A sensible distillation recipe for medical VLC, but the headline result rests on a lambda chosen on the test set. read the letter →

arxiv 2506.02229 v1 pith:EWDADWTM submitted 2025-06-02 cs.CV cs.AIcs.CLcs.LG

Manas Mehta , Yimu Pan , Kelly Gallagher , Alison D. Gernand , Jeffery A. Goldstein , Delia Mwinyelle , Leena Mithal , James Z. Wang This is my paper
classification cs.CVcs.AIcs.CLcs.LG
keywords knowledgedistillationvision-languagecontrastivelearningplacentapathologymodelcompressionmedicalimageanalysisunsupervisedpretrainingrobustnessAUC-ROC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes VLCD, a knowledge-distillation strategy for vision-language contrastive pretraining that needs no class labels, and pairs it with unsupervised predistillation on ImageNet. It claims this lets small student encoders match or surpass a ResNet-50 teacher on five placental-pathology tasks while running about four times faster with roughly a quarter of the parameters. The authors argue this makes automated placenta analysis practical on low-cost devices in resource-constrained settings, and report that predistillation stabilizes and improves performance, especially on low-quality iPad photographs.

What carries the argument

The central object is the generalized norm distillation loss $\mathcal{L}_{\mathrm{gnd}}$, which treats each text feature $F(v_j)$ from a frozen BERT encoder as a point in a continuous label space and anchors the student feature toward it, scaled by the larger of the student and teacher feature norms. This adapts the norm distillation loss of [28] to vision-language contrastive pretraining, where no class labels exist. The second component is unsupervised predistillation, which runs the same distance-based distillation on ImageNet for one epoch to move the student's initialization closer to the teacher's solution before fine-tuning on placenta data.

What would settle it

Shuffle the pathology-report texts against their images, or replace the text features with random vectors of the same dimension, and retrain VLCD; if the distilled students still match the teacher, the text anchor is not the active ingredient and the gains come from the feature-distance term alone.

Watch

Extended reading notes

Core claim

VLCD replaces the discrete class vectors used in norm distillation with a continuous label function $F(v_j)$ built from text features of pathology reports, so teacher knowledge can transfer to a student during vision-language pretraining without ground-truth labels. The paper reports that a distilled MobileNetV3 reaches AUC-ROC of 83.1, 82.2, 74.7, 70.6, and 91.7 on meconium, fetal inflammatory response, maternal inflammatory response, histological chorioamnionitis, and sepsis, versus teacher ResNet-50 scores of 81.3, 81.3, 75.0, 72.3, and 92.0, while using 7.1M versus 27.7M parameters and 3.9x higher inference throughput. On the low-quality iPad dataset, predistillation raises VLCD's mean AUC and sharply reduces run-to-run variance, which the authors take as evidence of robustness in real-world deployment.

Load-bearing premise

The student can inherit teacher knowledge only if the text features produced by a pretrained language model genuinely align with visual features of placental pathology in the shared embedding space.

Editorial extensions

If this is right

  • Distilled MobileNetV3 outperforms the undistilled MobileNetV3 from [19] on all five tasks, and matches or beats the ResNet-50 teacher on four of five.
  • Students of different architectures (MobileNetV3, EfficientNet-B0, EfficientFormer-L1) all stay within roughly one to two AUC points of the teacher, so VLCD is model-agnostic.
  • Inference speed improves by 1.7 to 4 times and parameter count drops to 25 to 50 percent of ResNet-50, making deployment feasible on smartphones and tablets.
  • Predistillation raises mean AUC on low-quality iPad images and cuts variance, improving robustness in realistic lower-quality settings.
  • The best distillation weight is $\lambda = 0.1$; too small distills too little teacher knowledge and too large destabilizes training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the text anchors come from a frozen language model, VLCD inherits any noise or bias in report-vision alignment; ablating or shuffling the text features would show how much of the gain is genuinely linguistic.
  • All supervision traces back to the same placenta dataset and the same teacher, so the student cannot exceed the teacher's ceiling; evaluating on independently labeled placental data would strengthen the deployment claim.
  • The predistillation stage was tested with a single one-epoch pass over ImageNet and one distance function; scaling it or using other unlabeled corpora might extend robustness gains to other medical imaging domains.
  • The paper reports only AUC-ROC; calibration and decision thresholds remain untested, so clinical deployment would require additional evaluation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes VLCD, a knowledge-distillation loss for vision-language contrastive pretraining that replaces the discrete class direction e_k of norm distillation with a text feature F(v_j), and an unsupervised predistillation stage on ImageNet. A ResNet-50 teacher pretrained on placenta image-text pairs is distilled into MobileNetV3, EfficientNet-B0, and EfficientFormer-L1. The authors report AUC-ROC on five placenta-pathology tasks and a small iPad-captured robustness set, claiming that the student models match or surpass the teacher while being substantially smaller and faster.

Significance. If the empirical claims are reliable, the paper offers a practical route to deployable placenta-analysis models in resource-constrained settings. The text-anchored distillation idea for medical vision-language contrastive learning is novel, and the inclusion of multiple student architectures plus an ablation over the distillation weight λ is a strength. The paper is also transparent about hyperparameters. The main reservations concern the evaluation protocol, especially hyperparameter selection on the test data, rather than the internal consistency of the method.

major comments (4)
  1. [§4.4, Table 6; §4.3, Table 3] The hyperparameter λ is selected by comparing AUC-ROC on the primary fine-tuning dataset in Table 6, and the same data are then used to report the headline results in Table 3. No held-out validation split is described. Because the reported gaps between VLCD students and the teacher are small (e.g., H.Chorio 70.6 vs. 72.3; Sepsis 91.7 vs. 92.0), selection on the test set can readily account for the apparent match-or-surpass pattern. The manuscript should either select λ on a validation split and report test results from the untouched split, or report all λ settings as primary outcomes rather than singling out the best one.
  2. [§4.3, Table 3] The claim that the distilled MobileNetV3 'significantly outperforms' the undistilled MobileNetV3 on all tasks is contradicted by the reported H.Chorio scores (70.6 vs. 70.9) and is not supported by any significance test for the other tasks. The comparison against the teacher is also within one standard deviation on several tasks (e.g., MIR 74.7 vs. 75.0; Sepsis 91.7 vs. 92.0). Please qualify the wording and add paired tests or confidence intervals over the five runs.
  3. [§3.1, Eq. (6)] The function F(v_j) is never explicitly defined. The text says e_k is generalized to 'the entire unit sphere function F' and that F(v_j) is used as the label, but it is unclear whether F is the identity on normalized text features, a learned projection, or something else. The denominator also mixes student and teacher norms without specifying how the two feature spaces are aligned. This matters because Eq. (6) is the core contribution and the implementation cannot be reproduced without this definition.
  4. [§4.1, Table 5; §4.4, Table 6] The iPad robustness evaluation uses 'over 50' images and reports very large standard deviations (e.g., VLCD w/o predistillation MIR 48.1±47.1 in Table 5; λ=1* C.Chorio at 55.3±228.1 in Table 6). The claim that predistillation improves robustness rests primarily on this small and noisy sample, and the exact number of images is not given. Please report the exact sample size, per-run results, and a more appropriate uncertainty estimate, such as bootstrap confidence intervals.
minor comments (5)
  1. [§3.1, Eq. (5)] There is an index inconsistency in Eq. (5): the inner sum is over j∈I_i while the outer sum runs over k; it should presumably be j∈I_k.
  2. [§2.2, Eq. (4)] Equation (4) uses ℓ̃ without defining the tilde; please state the relationship between ℓ in Eq. (3) and the symmetrized version used here.
  3. [§4.2, Table 1] The predistillation stage is described as using ImageNet, but the exact subset, number of images, and preprocessing steps are not given; please add these details for reproducibility.
  4. [§4.3, Table 4] The text says improvements in Table 4 are 'highlighted in green,' but color coding is not accessible in monochrome print; please use symbols or explicit annotations instead.
  5. [§5, Conclusion] The limitation paragraph correctly notes that student performance is bounded by the teacher, but this sits uneasily with the abstract's 'match or surpass the teacher' wording; consider a more precise phrasing in the abstract.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the VLCD student is trained by an explicit loss and then evaluated by measured downstream AUC-ROC; the match-or-surpass claim is an empirical result, not an identity. Self-citations provide the dataset, teacher, and baseline, which is standard for a knowledge-distillation study and does not force the outcome.

full rationale

The paper's central claim is that VLCD-distilled student networks match or surpass the teacher model on five placenta-pathology tasks while being smaller and faster. The training objective is L_VLCD = L_t + lambda * L_gnd (Eq. 7), where L_gnd (Eq. 6) is a text-anchored generalization of the norm distillation loss from [28], using frozen BERT text features F(v_j) as continuous labels and the teacher's feature norm in the denominator. This is an optimization objective, not a derivation that smuggles in the reported result: the downstream AUC-ROC scores are obtained by linear evaluation on random splits of the fine-tuning dataset, and the student is not guaranteed to match the teacher (e.g., VLCD MobileNet gets 70.6 on histological chorioamnionitis versus the teacher's 72.3). The comparison teacher, placenta dataset, and report preprocessing are inherited from the authors' prior work [19,20], but those are standard experimental assets for a compression study, not load-bearing circular assumptions. The paper also states the limitation that student performance is bounded by the teacher, which further confirms that the claim is empirical rather than definitional. One methodological concern is that lambda is selected in Section 4.4 using the primary fine-tuning dataset, and the same dataset is used for the headline results in Table 3, which is a form of evaluation-set selection that can inflate performance; however, this is a correctness and experimental-design issue, not a circular derivation, and the hard rules require exhibiting a specific reduction by construction before flagging circularity. No such reduction is present in the equations or the evaluation protocol.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central method is an empirical training objective, so the ledger is dominated by hyperparameters and domain assumptions rather than invented entities. The most load-bearing assumptions are that the teacher is a reliable supervisor and that frozen BERT text features align with placental pathology visual features. No new particle, force, or external entity is introduced.

free parameters (4)
  • lambda (distillation weight) = 0.1
    Chosen as best in ablation Table 6 on the primary fine-tuning dataset; not derived from first principles.
  • alpha and tau in CLIP loss = 0.5 / 0.1
    Set in Table 1; standard loss weighting and temperature, chosen by the authors.
  • C (linear evaluation regularization) = 3.16
    Listed in Table 1 for the logistic regression evaluation; a tuned hyperparameter.
  • Predistillation epochs = 1
    One epoch on ImageNet is an ad hoc choice reported in Table 1 and Section 4.4.
assumptions (4)
  • domain assumption The teacher ResNet-50 trained on the placenta dataset provides reliable feature targets for distillation.
    Assumed throughout Section 4; teacher features are the only source of visual knowledge for the student beyond the frozen text encoder.
  • ad hoc to paper Text descriptions can be treated as continuous class labels aligned in the shared embedding space.
    Core to the definition of L_gnd in Eq. 6; the paper asserts this generalization in Section 3.1 without independent validation of text-vision alignment.
  • domain assumption ImageNet predistillation with a placenta-trained teacher improves the student's initialization and robustness.
    The mechanism in Fig. 2 is a geometric intuition, not a proven property; it is tested only indirectly through downstream AUC results.
  • domain assumption The placenta dataset labels used for fine-tuning evaluation are correct.
    Label quality is inherited from [19, 20]; no inter-observer or pathologist agreement analysis is reported here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VLCD: Vision-Language Contrastive Distillation for Accurate and Efficient Automatic Placenta Analysis." pith.science (2026). https://pith.science/paper/EWDADWTM

@misc{pith2026250602229,
  author       = {Pith},
  title        = {Pith review of: VLCD: Vision-Language Contrastive Distillation for Accurate and Efficient Automatic Placenta Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EWDADWTM}},
  note         = {Machine review of arXiv:2506.02229}
}
read the original abstract

Pathological examination of the placenta is an effective method for detecting and mitigating health risks associated with childbirth. Recent advancements in AI have enabled the use of photographs of the placenta and pathology reports for detecting and classifying signs of childbirth-related pathologies. However, existing automated methods are computationally extensive, which limits their deployability. We propose two modifications to vision-language contrastive learning (VLC) frameworks to enhance their accuracy and efficiency: (1) text-anchored vision-language contrastive knowledge distillation (VLCD)-a new knowledge distillation strategy for medical VLC pretraining, and (2) unsupervised predistillation using a large natural images dataset for improved initialization. Our approach distills efficient neural networks that match or surpass the teacher model in performance while achieving model compression and acceleration. Our results showcase the value of unsupervised predistillation in improving the performance and robustness of our approach, specifically for lower-quality images. VLCD serves as an effective way to improve the efficiency and deployability of medical VLC approaches, making AI-based healthcare solutions more accessible, especially in resource-constrained environments.

Figures

Figures reproduced from arXiv: 2506.02229 by the authors.

Figure 1
Figure 1. A diagram illustrating our proposed approach. (a) Unsupervised predistilla [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. A diagram illustrating the value of unsupervised predistillation. In (a), the [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Representative samples of placenta images from our dataset. These images [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 28 canonical work pages

  1. [1]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Bannur, S., Hyland, S., Liu, Q., P´erez-Garc´ıa, F., Ilse, M., Castro, D.C., Boecking, B., Sharma, H., Bouzid, K., Thieme, A., Schwaighofer, A., Wetscherek, M., Lungren, M.P., Nori, A., Alvarez-Valle, J., Oktay, O.: Learning to exploit temporal structure for biomedical vision- language processing. In: Proceedings of the IEEE/CVF Conference on Computer Vis...

    work page 2023

  2. [2]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Chen, P., Liu, S., Zhao, H., Jia, J.: Distilling knowledge via knowledge review. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5008–5017 VLCD: Vision-Language Contrastive Distillation 13 (2021)

    work page 2021

  3. [3]

    In: Proceedings of the International Conference on Machine Learning, pp

    Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: Proceedings of the International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)

    work page 2020

  4. [4]

    In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Chen, X., He, Y., Xue, C., Ge, R., Li, S., Yang, G.: Knowledge boosting: Rethinking medical contrastive vision-language pre-training. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 405–415. Springer Nature Switzerland, Cham (2023)

    work page 2023

  5. [5]

    In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Inter- vention, pp

    Chen, Y., Wu, C., Zhang, Z., Goldstein, J.A., Gernand, A.D., Wang, J.Z.: PlacentaNet: Auto- matic morphological characterization of placenta photos with deep learning. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Inter- vention, pp. 487–495. Springer (2019)

    work page 2019

  6. [6]

    Computerized Medical Imaging and Graphics84, 101744 (2020)

    Chen, Y., Zhang, Z., Wu, C., Davaasuren, D., Goldstein, J.A., Gernand, A.D., Wang, J.Z.: Ai-plax: Ai-based placental assessment and examination using photos. Computerized Medical Imaging and Graphics84, 101744 (2020)

    work page 2020

  7. [7]

    In: The World Factbook

    CIA: Country comparisons - infant mortality rate. In: The World Factbook. Central Intelligence Agency (2023)

    work page 2023

  8. [8]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    work page 2009

Show all 32 references
  1. [9]

    arXiv preprint arXiv:1810.04805 (2018)

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  2. [10]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Dong, X., Bao, J., Zheng, Y., Zhang, T., Chen, D., Yang, H., Zeng, M., Zhang, W., Yuan, L., Chen, D., Wen, F., Yu, N.: Maskclip: Masked self-distillation advances contrastive language- image pretraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2023

  3. [11]

    Vital Statistics Rapid Release Report (2023)

    Ely, D.M., Driscoll, A.K.: Infant mortality in the united states: Provisional data from the 2022 period linked birth/infant death file. Vital Statistics Rapid Release Report (2023)

    2023

  4. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Fang, Z., Wang, J., Hu, X., Wang, L., Yang, Y., Liu, Z.: Compressing visual-linguistic model via knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1428–1438 (2021)

    2021

  5. [13]

    Frontiers in Immunology11, 531543:1–14 (2020)

    Goldstein, J.A., Gallagher, K., Beck, C., Kumar, R., Gernand, A.D.: Maternal-fetal inflam- mation in the placenta and the developmental origins of health and disease. Frontiers in Immunology11, 531543:1–14 (2020)

    2020

  6. [14]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    He, K., Chen, X., Xie, S., Li, Y., Doll ´ar, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)

    2022

  7. [15]

    arXiv preprint arXiv:1503.02531 (2015)

    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

    2015 arXiv

  8. [16]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Kim, B., Jo, Y., Kim, J., Kim, S.: Misalign, contrast then distill: Rethinking misalignments in language-image pre-training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2563–2572 (2023)

    2023

  9. [17]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Li, X., Fang, Y., Liu, M., Ling, Z., Tu, Z., Su, H.: Distilling large vision-language model with out-of-distribution generalizability. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2492–2503 (2023)

    2023

  10. [18]

    IEEE Transactions on Medical Imaging42(12), 3579–3589 (2023)

    Liu, B., Lu, D., Wei, D., Wu, X., Wang, Y., Zhang, Y., Zheng, Y.: Improving medical vision- language contrastive pretraining with semantics-aware triage. IEEE Transactions on Medical Imaging42(12), 3579–3589 (2023)

    2023

  11. [19]

    In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Pan, Y., Cai, T., Mehta, M., Gernand, A.D., Goldstein, J.A., Mithal, L., Mwinyelle, D., Gal- lagher, K., Wang, J.Z.: Enhancing automatic placenta analysis through distributional feature recomposition in vision-language contrastive learning. In: Proceedings of the International...

    2023

  12. [20]

    In: Proceedings of the International Conference on Medical Image Computing and Computer- Assisted Intervention, pp

    Pan, Y., Gernand, A.D., Goldstein, J.A., Mithal, L., Mwinyelle, D., Wang, J.Z.: Vision-language contrastive learning approach to robust automatic placenta analysis using photographic images. In: Proceedings of the International Conference on Medical Image Computing and Compute...

    2022

  13. [21]

    IEEE Transactions on Medical Imaging40(12), 3820–3831 (2021)

    Qin, D., Bu, J.J., Liu, Z., Shen, X., Zhou, S., Gu, J.J., Wang, Z.H., Wu, L., Dai, H.F.: Efficient medical image segmentation based on knowledge distillation. IEEE Transactions on Medical Imaging40(12), 3820–3831 (2021)

    2021

  14. [22]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Radenovic, F., Dubey, A., Kadian, A., Mihaylov, T., Vandenhende, S., Patel, Y., Wen, Y., Ramanathan, V., Mahajan, D.: Filtering, distillation, and hard negatives for vision-language pre-training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2023

  15. [23]

    In: Proceedings of the International Conference on Machine Learning, pp

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: Proceedings of the International Conference on Machine Learning, pp. 8748–

  16. [24]

    Archives of Pathology & Laboratory Medicine132(4), 641–651 (2008)

    Roberts, D.J.: Placental pathology, a survival guide. Archives of Pathology & Laboratory Medicine132(4), 641–651 (2008)

    2008

  17. [25]

    Sagay, A.S.: Histological chorioamnionitis. J. West Afr. Coll. Surg.6(3), x–xiii (2016)

    2016

  18. [26]

    Computers in Biology and Medicine155, 106476 (2023)

    Sepahvand, M., Abdali-Mohammadi, F.: Joint learning method with teacher–student knowl- edge distillation for on-device breast cancer image classification. Computers in Biology and Medicine155, 106476 (2023)

    2023

  19. [27]

    In: Proceedings of the International Conference on Machine Learning, pp

    Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momen- tum in deep learning. In: Proceedings of the International Conference on Machine Learning, pp. 1139–1147. PMLR (2013)

    2013

  20. [28]

    arXiv preprint arXiv:2305.17007 (2023)

    Wang, Y., Cheng, L., Duan, M., Wang, Y., Feng, Z., Kong, S.: Improving knowledge distillation via regularizing feature norm and direction. arXiv preprint arXiv:2305.17007 (2023)

    2023 arXiv

  21. [29]

    In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Xing, X., Hou, Y., Li, H., Yuan, Y., Li, H., Meng, M.Q.H.: Categorical relation-preserving contrastive knowledge distillation for medical image classification. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 16...

    2021

  22. [30]

    In: Proceedings of the International Conference on Learning Representations (2018)

    Ye, J., Lu, X., Lin, Z., Wang, J.Z.: Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers. In: Proceedings of the International Conference on Learning Representations (2018)

    2018

  23. [31]

    arXiv preprint arXiv:1612.03928 (2016)

    Zagoruyko, S., Komodakis, N.: Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928 (2016)

    2016 arXiv

  24. [32]

    Proceedings of the AAAI Conference on Artificial Intelligence36(3), 3508–3516 (2022)

    Zheng, K., Wang, Y., Yuan, Y.: Boosting contrastive learning with relation knowledge dis- tillation. Proceedings of the AAAI Conference on Artificial Intelligence36(3), 3508–3516 (2022)

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.