Pith. sign in

REVIEW 5 major objections 8 minor 45 references

Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation

T0 review · 5 major / 8 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Stochastic self-distillation with student-guided attention lets a single model trained twice match or beat ensemble and weight-averaging baselines without extra deployed size.

desk verdict A genuinely new single-teacher stochastic distillation recipe whose efficiency claim is under-measured; worth refereeing after the complexity accounting is fixed. read the letter →

arxiv 2504.14307 v2 pith:2TBMLEAH submitted 2025-04-19 cs.LG

Muhammad Haseeb Aslam , Clara Martinez , Marco Pedersoli , Alessandro Koerich , Ali Etemad , Eric Granger This is my paper · ORCID
classification cs.LG
keywords knowledgedistillationself-distillationdropoutstudent-guidedattentionensemblelearningtimeseriesrepresentationresource-constraineddeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the accuracy gains usually bought with ensemble learning can be obtained from a single network by making the teacher stochastic at distillation time. Instead of training many teachers, it freezes one trained model and runs it several times with dropout active, generating multiple feature representations of the same input. To keep the noise in those representations from hurting the student, the student's own current representation is used as a query: each stochastic teacher representation is scored, the low-scoring ones are discarded, and the rest are attention-weighted into one attended feature vector that the student is trained to match. The authors report accuracy gains over the teacher and over existing baselines on affective computing, wearable and biosignal time series, activity recognition, and image classification, with the deployed model unchanged in size and only modest extra training cost. If this holds, ensemble-like accuracy becomes available on wearable and other resource-constrained devices without storing or running multiple models.

What carries the argument

The load-bearing object is the attended teacher representation $\hat{f}_T(x)$ built by student-guided attention. Each stochastic teacher feature $f_T^i(x)$ is produced by one dropout-masked forward pass of the frozen teacher, and the student's own feature $f_S(x)$ acts as a query; the dot products become softmax attention weights, temperature $h$ spreads them, and thresholding at the $\epsilon$-th percentile removes outliers before the weighted sum is formed. That single vector carries the argument because it converts a noisy set of stochastic representations into an implicit ensemble of task-relevant teacher views, which the student then matches with a feature-level mean-squared-error distillation loss.

What would settle it

On a fixed dataset and teacher, compare SSD against a student that distills the unweighted average of all stochastic teacher representations and against an oracle that is told, per sample, which stochastic representation has the lowest task loss; if SSD does not beat the unweighted average by a clear margin, or if random initialization performs as well as teacher-weight initialization, the student-guided filter is not actually carrying the claimed gain.

Watch

Extended reading notes

Core claim

The paper's central claim is that a student trained with stochastic self-distillation (SSD) can outperform its own teacher and match or beat ensemble and weight-averaging baselines, provided the teacher is run with dropout during distillation and the student filters the resulting stochastic features. The filtering step is student-guided: for each input, the student computes dot products between its own feature vector and each of $n$ stochastic teacher features, softmaxes them with a temperature $h$, masks every weight below the $\epsilon$-th percentile, and forms the attended teacher vector $\hat{f}_T(x)=\sum_i \hat{\alpha}_i f_T^i(x)$. The total loss is $L_{\text{total}}=L_{\text{task}}+\lambda L_{\text{dist}}$, where $L_{\text{dist}}$ is a mean-squared-error alignment between the student feature and $\hat{f}_T(x)$. The authors argue this is not merely teaching the student to use dropout, because distilling the unfiltered average of stochastic teacher representations includes noise, while the student-guided attention removes it. They also report that initializing the student with the teacher's weights is required: with random initialization the student's guide representation is misaligned and performance falls below the baseline.

Load-bearing premise

The student, because it starts from the teacher's weights, stays close enough to the teacher's feature space during training that its own representation can be trusted to pick the teacher representations worth learning from.

Editorial extensions

If this is right

  • The deployed model stays exactly the same size as the baseline, so ensemble-like accuracy does not require storing or running multiple models at inference time.
  • Training cost stays near two runs of one network plus $n$ extra forward passes per batch, far below the repeated fine-tuning needed for weight averaging or 25-model ensembles.
  • Because the filter is dynamic, using a percentile threshold rather than a fixed top-$k$, it adapts to how many stochastic representations are task-relevant at each distillation step.
  • Architectures with several dropout layers generate more diverse teacher representations and show larger gains, while architectures with a single dropout layer still improve.
  • The same recipe improves unsupervised time-series representation-learning baselines and image classifiers, indicating it is not tied to one training objective or data type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test, but the same filtering should transfer to other stochastic perturbations of the teacher, such as input noise or random feature masking, since the attention mechanism only needs a set of teacher views and a student anchor.
  • If the method is understood as selecting the task-relevant modes of a dropout-induced posterior over features, the best temperature $h$ and threshold $\epsilon$ should track the spread of that posterior; this gives a testable prediction about when SSD will fail, for instance under strong distribution shift.
  • The percentile filter acts like a per-sample ensemble size, so adapting $\epsilon$ per sample or class might improve accuracy beyond the fixed global value used in the paper; this is an extension, not a claim of the paper.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper proposes Stochastic Self-Distillation (SSD), a training strategy that generates multiple stochastic feature representations from a single pre-trained teacher using distillation-time dropout, then filters and weights these representations via a student-guided attention mechanism (SGKD) before applying feature-level knowledge distillation. The student is initialized with the teacher's weights and uses its own current representation as a reference ('authority') to rank and select task-relevant teacher representations. The method is evaluated on affective computing datasets (Biovid, StressID), wearable/biosignal datasets from the UCR Archive, the HAR dataset, and CIFAR-10/100, and is compared against ensembles and weight-averaging methods. The authors claim that SSD outperforms state-of-the-art methods without increasing model size and with negligible computational complexity relative to ensemble and weight-averaging baselines.

Significance. If the accuracy and efficiency claims hold, SSD offers a practical recipe for obtaining ensemble-like gains from a single model with two training runs, which is relevant for deployment on wearable and resource-constrained devices. The paper includes several useful ablations (dropout rate, number of stochastic representations, attention regularization, and student initialization) and provides downloadable code. However, the central efficiency claim rests on an incomplete computational-accounting, and the SOTA claims are weakened by missing error bars, hand-set hyperparameters, and a selectively reported UCR evaluation. These issues are addressable, and the core idea is worth further consideration.

major comments (5)
  1. [Section 4.2, Appendix B, Algorithm 1] The reported train-time FLOP comparison (0.87 to 21.8 G-FLOPs) omits the n=30 stochastic teacher forward passes per sample that are required at every student training iteration (Algorithm 1, lines 3-10). Over E student epochs, this adds roughly n×E full teacher forward passes, which can be comparable to or larger than the fine-tuning cost of a model-soup or ensemble member, especially when the teacher is used only briefly for the student stage. The claim that SSD 'incurs negligible computational complexity' is therefore unsupported by the measurements presented; the authors should provide a complete training-cost accounting or substantially soften the claim.
  2. [Table 1, Table 4] The Biovid accuracy (Table 1) and HAR accuracy (Table 4) are reported as single values without error bars or multiple-seed statistics. Since the reported improvements over the baselines are modest (e.g., 2.5% on Biovid, 1.8% on HAR), the absence of variance estimates makes it impossible to assess whether these gains are statistically meaningful, which is load-bearing for the SOTA claim. The authors should report mean and standard deviation over at least three independent runs, or justify why a single run is adequate for these datasets.
  3. [Section 4.1, Appendix C.2] The hyperparameters epsilon (set to 90), temperature H (5 for TS2Vec, 15 for SoftCLT), and distillation-weight lambda (0.2) are hand-set without a stated validation protocol. Although Appendix C.2 shows some sensitivity of epsilon on one dataset, the choice of epsilon=90 is not derived from a principled selection procedure, and H and lambda are not ablated at all. Without evidence that these values are stable across datasets or at least chosen fairly, the reported gains may be overfitted to the specific test sets.
  4. [Table 3] The UCR evaluation is restricted to 12 selected wearable/biosignal datasets, and the selection criteria are not provided. On MedicalImages, TS2Vec+SSD (0.7710) degrades clearly relative to the TS2Vec baseline (0.8092), yet the average still improves because of gains on other datasets. The authors should either present results on the full UCR archive or justify the subset choice, and they should explicitly discuss degradations such as MedicalImages rather than only highlighting the average.
  5. [Section 3.2, Section 4.3, Fig. 6] The proposed method's core assumption—that the student representation, initialized with the teacher's weights and trained with L_task, remains a reliable anchor for identifying task-relevant teacher features—is load-bearing. The ablation in Fig. 6 and Table 8 shows that random initialization degrades performance below the baseline, confirming this assumption. The authors should discuss the risk of student drift during training (i.e., the student moving outside the teacher's feature space) and explain why the dot-product ranking remains valid when the student parameters evolve substantially during the distillation stage.
minor comments (8)
  1. [Eq. (1)] The summation symbol is typeset as 'PN' rather than a proper summation sign, and the index N is used inconsistently with the earlier notation n for the number of stochastic representations; please standardize the notation.
  2. [Algorithm 1] Line 6 uses 'n' in the loop but Eq. (1) uses 'N'; unify the symbols to avoid confusion.
  3. [Section 4.1] The sentence 'All experiments were performed on the NVIDIA A100-SXM4-40GB GPUs with the epsilon value of 90' mixes hardware description with a hyperparameter choice; consider moving the epsilon value to the hyperparameter settings paragraph.
  4. [Appendix D] The notation L_SSD is introduced in Appendix D but not used in the main text, and the equations mix E_D[fT(x)] with the earlier f_i^T(x) notation; aligning the notation would improve readability.
  5. [References] Some references, such as [7], appear not to be cited in the body of the paper; please check that all entries in the bibliography are used in the text.
  6. [Fig. 2 and Fig. 7] Figures 2 and 7 convey essentially the same comparison and could be merged to avoid redundancy.
  7. [Abstract] The abstract contains the phrase 'Experimental results4' with a footnote marker, but the footnote only appears in the full text; either remove the marker or explain it in the abstract.
  8. [Abstract and Section 4.2] The claim of operating 'without increasing the model size at both training and testing time' is ambiguous because during student training both teacher and student models are in memory; consider rewording to clarify that the deployed (inference-time) model size is unchanged.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SSD is an empirical training recipe with explicit ablations; the efficiency concern is an accounting issue, not a circular derivation.

full rationale

The paper's central claims are empirical and validated against held-out benchmarks, not derived from its own inputs by construction. The method is defined by Eqs. (1)-(5) and Algorithm 1, where the student representation is explicitly used as an attention authority; this self-referential design is acknowledged in the text and tested in ablations such as Table 5, Fig. 6, and Appendix C. No fitted parameter is renamed as a prediction, no load-bearing self-citation supplies the core mechanism, and no uniqueness theorem is imported to force a choice. The only notable issue is that Appendix B's efficiency comparison omits the n=30 stochastic teacher forward passes per sample during student training, which is a computational accounting or correctness concern, not a circularity of the kind defined here. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method's performance depends on several hand-chosen hyperparameters and on two domain assumptions (dropout diversity and student-as-anchor) that are not derived in the paper. No new physical or architectural entities are introduced.

free parameters (5)
  • Number of stochastic representations n = 30 for Biovid/StressID/HAR/CIFAR; 15 for TS2Vec and SoftCLT
    Controls the size of the stochastic ensemble. Chosen by hand and varied in ablations; performance drops for very large n.
  • Teacher dropout rate p = 0.2 for most experiments; student dropout 0.1 for TS2Vec
    Controls diversity of teacher representations. Ablation shows performance peaks at 0.2.
  • Temperature h (H) = 5 for TS2Vec, 15 for SoftCLT; unspecified for other datasets
    Smooths attention weights in Eq. (1); set per method without a stated selection rule.
  • Masking percentile epsilon = 90
    Filters teacher representations by percentile of attention weights; Table 10 scans this value and shows sensitivity.
  • Distillation loss weight lambda = 0.2 for TS2Vec/SoftCLT; unspecified for other datasets
    Balances task loss and distillation loss; set by hand.
assumptions (3)
  • domain assumption Self-distillation from an overparameterized network with an early-stopped teacher allows the student to surpass the teacher.
    Invoked in the Introduction as motivation, based on prior work (Furlanello et al., Dong et al.). The paper does not re-derive this.
  • domain assumption Dropout-induced stochasticity in a frozen teacher approximates diversity from an ensemble of independently trained models.
    The core of SSD; the paper relies on Monte Carlo dropout behavior but provides no new theoretical justification beyond citing Gal and Ghahramani.
  • ad hoc to paper The student representation, initialized with teacher weights and trained with L_task, remains a reliable anchor for task-relevant teacher features.
    Student-guided attention in Eq. (1) assumes dot-product similarity to fS identifies informative teacher representations. Section 4.3 shows random initialization fails, confirming this assumption is load-bearing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation." pith.science (2026). https://pith.science/paper/2TBMLEAH

@misc{pith2026250414307,
  author       = {Pith},
  title        = {Pith review of: Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2TBMLEAH}},
  note         = {Machine review of arXiv:2504.14307}
}
read the original abstract

Advances in self-distillation have shown that when knowledge is distilled from a teacher to a student using the same deep learning (DL) architecture, the student performance can surpass the teacher particularly when the network is overparameterized and the teacher is trained with early stopping. Alternatively, ensemble learning also improves performance, although training, storing, and deploying multiple models becomes impractical as the number of models grows. Even distilling an ensemble to a single student model or weight averaging methods first requires training of multiple teacher models and does not fully leverage the inherent stochasticity for generating and distilling diversity in DL models. These constraints are particularly prohibitive in resource-constrained or latency-sensitive applications such as wearable devices. This paper proposes to train only one model and generate multiple diverse teacher representations using distillation-time dropout. However, generating these representations stochastically leads to noisy representations that are misaligned with the learned task. To overcome this problem, a novel stochastic self-distillation (SSD) training strategy is introduced for filtering and weighting teacher representation to distill from task-relevant representations only, using student-guided knowledge distillation (SGKD). The student representation at each distillation step is used as authority to guide the distillation process. Experimental results on real-world affective computing, wearable/biosignal datasets from the UCR Archive, the HAR dataset, and image classification datasets show that the proposed SSD method can outperform state-of-the-art methods without increasing the model size at both training and testing time, and incurs negligible computational complexity compared to state-of-the-art ensemble learning and weight averaging methods.

Figures

Figures reproduced from arXiv: 2504.14307 by the authors.

Figure 1
Figure 1. Illustration of the proposed SSD training strategy. The teacher [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Comparison of the SSD method with baseline (BL), traditional ensembles [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. t-SNE plots of three teacher representations [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Effect of various dropout rates on the performance and variance quantifi [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Effect of attention weights αi regularization on student performance can be observed from Fig. 5a that, without regularization, the attention weight is extremely high for α2. On the other hand, the weights are more spread out with regularization, showing that the stude…
Figure 6
Figure 6. Figure 6: Performance of SSD with and without student parameters initialization [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Accuracy on HAR data of the SSD method with baseline (BL), traditional [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 35 canonical work pages

  1. [1]

    ICLR (2020)

    Allen-Zhu, Z., Li, Y.: Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. ICLR (2020)

    work page 2020

  2. [2]

    In: ESANN (2013)

    Anguita, D., Ghio, A., Oneto, L., Parra, X., Reyes-Ortiz, J.L.: A public domain dataset for human activity recognition using smartphones. In: ESANN (2013)

    work page 2013

  3. [3]

    In: IEEE/CVF CVPRw (2023)

    Aslam, M.H., Osama Zeeshan, M., Pedersoli, M., Koerich, A.L., Bacon, S., Granger, E.: Privileged knowledge distillation for dimensional emotion recognition in the wild. In: IEEE/CVF CVPRw (2023)

    work page 2023

  4. [4]

    Even your Teacher Needs Guidance: Ground-Truth Targets Dampen Regularization Imposed by Self-Distillation

    Borup, K., Andersen, L.N.: Even your teacher needs guidance: Ground-truth targets dampen regularization imposed by self-distillation. arXiv 2102.13088 (2021)

    work page Pith review arXiv 2021

  5. [5]

    In: Proceedings of the 12th ACM SIGKDD

    Bucilua, C., Caruana, R., Niculescu-Mizil, A.: Model compression. In: Proceedings of the 12th ACM SIGKDD. Association for Computing Machinery (2006)

    work page 2006

  6. [6]

    In: Neural IPS Datasets and Benchmarks Track (2023)

    Chaptoukaev, H., Strizhkova, V., et al., M.P.: Stress ID : a multimodal dataset for stress identification. In: Neural IPS Datasets and Benchmarks Track (2023)

    work page 2023

  7. [7]

    Chen, Y., Wang, N., Zhang, Z.: Darkrank: Accelerating deep metric learning via cross sample similarities transfer (2017)

    work page 2017

  8. [8]

    IEEE/CAA Journal of Automatica Sinica (2019)

    Dau et al.: The ucr time series archive. IEEE/CAA Journal of Automatica Sinica (2019)

    work page 2019

Show all 45 references
  1. [9]

    arXiv 1912.02757 (2020)

    Fort, S., Hu, H., Lakshminarayanan, B.: Deep ensembles: A loss landscape perspective. arXiv 1912.02757 (2020)

    2020 arXiv

  2. [10]

    In: ICML (2018)

    Furlanello, T., Lipton, Z.C., Tschannen, M., Itti, L., Anandkumar, A.: Born again neural networks. In: ICML (2018)

    2018

  3. [11]

    Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: Representing model uncertainty in deep learning (ICML 2016)

    2016

  4. [12]

    arXiv 1503.02531 (2015)

    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv 1503.02531 (2015)

    2015 arXiv

  5. [13]

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models (2021)

    2021

  6. [14]

    SAC '23, ACM (2023)

    Ji, X., Zhao, T., Li, W., Zomaya, A.: Automatic pain assessment with ultra-short electrodermal activity signal. SAC '23, ACM (2023)

    2023

  7. [15]

    ESWA 235 (2024)

    Jiang, M., Rosio, R., et al., S.S.: Personalized and adaptive neural networks for pain detection from multi-modal physiological features. ESWA 235 (2024)

    2024

  8. [16]

    IEEE JSTSP (2016)

    Kächele, M., Thiam, P., Amirian, M.e.a.: Methods for person-centered continuous pain intensity assessment from bio-physiological channels. IEEE JSTSP (2016)

    2016

  9. [17]

    In: NeurIPS 2017

    Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: NeurIPS 2017

    2017

  10. [18]

    In: ICLR 2024

    Lee, S., Park, T., Lee, K.: Soft contrastive learning for time series. In: ICLR 2024

    2024

  11. [19]

    IEEE TNNLS (2023)

    Li, S., Lin, M., Wang, Y., Wu, Y., Tian, Y., Shao, L., Ji, R.: Distilling a powerful student model via online knowledge distillation. IEEE TNNLS (2023)

    2023

  12. [20]

    et al.: Ensembles of low-rank expert adapters (2025),

    Li, Y. et al.: Ensembles of low-rank expert adapters (2025),

    2025

  13. [21]

    learning (2021)

    Lin, T., et al: Ensemble distillation for robust model fusion in fed. learning (2021)

    2021

  14. [22]

    In: IEEE EMBC 2018

    Lopez-Martinez, D., Picard, R.: Continuous pain intensity estimation from autonomic signals with recurrent neural networks. In: IEEE EMBC 2018

    2018

  15. [23]

    In: IEEE ACIIw 2017

    Lopez-Martinez, D., Picard, R.: Multi-task neural networks for personalized pain recognition from physiological signals. In: IEEE ACIIw 2017

    2017

  16. [24]

    Frontiers in Physiology 14 (2023)

    Lu, Z., Ozek, B., Kamarthi, S.: Transformer encoder with multiscale deep learning for pain classification using physiological signals. Frontiers in Physiology 14 (2023)

    2023

  17. [25]

    arXiv 2002.05715 (2020)

    Mobahi, H., Farajtabar, M., Bartlett, P.L.: Self-distillation amplifies regularization in hilbert space. arXiv 2002.05715 (2020)

    2020 arXiv

  18. [26]

    In: NeurIPS 2019

    Ovadia, Y., Fertig, E., et al., J.J.R.: Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In: NeurIPS 2019

    2019

  19. [27]

    In: CVPR '19

    Park, W., Kim, D., Lu, Y., Cho: Relational knowledge distillation. In: CVPR '19

  20. [28]

    PlusOne Journal 2021

    Pouromran, F., Radhakrishnan, S., Kamarthi, S.: Exploration of physiological sensors, features, and ml models for pain intensity estimation. PlusOne Journal 2021

    2021

  21. [29]

    In: ICLR 2015

    Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: Hints for thin deep nets. In: ICLR 2015

    2015

  22. [30]

    AAAI 38(13), 14875--14885 (Mar 2024)

    Sarkar, P., Etemad, A.: Xkd: Cross-modal knowledge distillation with domain alignment for video representation learning. AAAI 38(13), 14875--14885 (Mar 2024)

    2024

  23. [31]

    Springer International Publishing, ICOST (2022)

    Shi, H., Chikhaoui, B., Wang, S.: Tree-based models for pain detection from biomedical signals. Springer International Publishing, ICOST (2022)

    2022

  24. [32]

    Stanton, S., Izmailov, P., Kirichenko, P., Alemi, A.A., Wilson, A.G.: Does knowledge distillation really work? arXiv 2106.05945 (2021)

    2021 arXiv

  25. [33]

    Sensors 19, 4503 (2019)

    Thiam, P., Bellmann, P., Kestler, H.A., Schwenker, F.: Exploring deep physiological models for nociceptive pain recognition. Sensors 19, 4503 (2019)

    2019

  26. [34]

    Frontiers in Physiology 2022

    Thiam, P., Hihn, H., Braun, D.A., Kestler, H.A., Schwenker, F.: Multi-modal pain intensity assessment based on physiological signals. Frontiers in Physiology 2022

    2022

  27. [35]

    In: IEEE ICC 2013

    Walter, S., Werner, e.a.: The biovid heat pain database: Data for the advancement and systematic validation of an automated pain recognition. In: IEEE ICC 2013

    2013

  28. [36]

    In: 42nd IEEE EMBC (2020)

    Wang, R., Xu, K., Feng, H., Chen, W.: Hybrid rnn-ann based deep physiological network for pain recognition. In: 42nd IEEE EMBC (2020)

    2020

  29. [37]

    IEEE TAC 2016

    Werner, P., Al-Hamadi, A., Limbrecht-Ecklundt, K., Walter, S., Gruss, S., Traue, H.C.: Automatic pain assessment with facial activity descriptors. IEEE TAC 2016

    2016

  30. [38]

    In: ICPR 2014

    Werner, P., Al-Hamadi, A., Niese, R., Walter, S., Gruss, S., Traue, H.C.: Automatic pain recognition from video and biomedical signals. In: ICPR 2014

    2014

  31. [39]

    arXiv 2203.05482 (2022)

    Wortsman, M., Ilharco, G., Gadre, S.Y., et al., R.R.: Model soups. arXiv 2203.05482 (2022)

    2022 arXiv

  32. [40]

    In: AAAI (2021)

    Yue, Z., Wang, Y., Duan, J., Yang, T., Huang, C., Tong, Y., Xu, B.: Ts2vec: Towards universal representation of time series. In: AAAI (2021)

    2021

  33. [41]

    arXiv 1612.03928 (2017)

    Zagoruyko, S., Komodakis, N.: Paying more attention to attention: Improving the performance of cnns via attention transfer. arXiv 1612.03928 (2017)

    2017 arXiv

  34. [42]

    IEEE T-PAMI 44(8), 4388--4403 (2021)

    Zhang, L., Bao, C., Ma, K.: Self-distillation: Towards efficient and compact neural networks. IEEE T-PAMI 44(8), 4388--4403 (2021)

    2021

  35. [43]

    arXiv 2006.05065 (2020)

    Zhang, Z., Sabuncu, M.R.: Self-distillation as instance-specific label smoothing. arXiv 2006.05065 (2020)

    2020 arXiv

  36. [44]

    , " * write output.state after.block = add.period write

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.