Pith. sign in

REVIEW 3 major objections 5 minor 70 references

Role of Mixup in Topological Persistence Based Knowledge Distillation for Wearable Sensor Data

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mixup and knowledge distillation share a smoothing mechanism, and per-teacher mixup strengths let a small wearable-sensor model absorb topological knowledge.

desk verdict A useful empirical sweep of mixup for multi-teacher KD on wearable sensor data, but the headline claim is undermined by test-set hyperparameter selection and small effect sizes. read the letter →

arxiv 2502.00779 v1 pith:FZAN6DSS submitted 2025-02-02 cs.LG cs.AIeess.SP

Eun Som Jeon , Hongjun Choi , Matthew P. Buman , Pavan Turaga This is my paper · ORCID
classification cs.LGcs.AIeess.SP
keywords knowledgedistillationmixuptopologicaldataanalysispersistenceimagewearablesensorhumanactivityrecognitionmulti-teachertime-seriesclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that mixup augmentation and knowledge distillation work through the same mechanism—label smoothing—and that exploiting this overlap lets a small time-series-only student model absorb topological knowledge that normally requires expensive persistence-image computation. It distills the student from two teachers, one trained on raw accelerometer time series and one on persistence images, using an annealing strategy, and applies mixup while training the student. The paper reports that mixup consistently helps in this multi-teacher setting, and that giving each teacher its own mixup strength works best: 71.22% on GENEActiv and 88.13% on PAMAP2. If true, this would let wearable devices run accurate activity recognition without computing topological features at test time.

What carries the argument

The engine is Equation (7), a two-teacher distillation objective in which each teacher contributes its own knowledge-distillation loss and its own mixup-augmented loss, with mixup strengths sampled from $\mathrm{Beta}(\alpha_1,\alpha_1)$ and $\mathrm{Beta}(\alpha_2,\alpha_2)$ respectively. Around it sit three control knobs: the annealing strategy, which initializes the student from a model trained from scratch to reduce the knowledge gap; the temperature $T$, which smooths each teacher's logits; and partial mixup, which restricts the number of mixed pairs per batch to avoid excessive smoothing. Persistence images supply the topological teacher's input as stable 2D grid representations of persistent homology.

What would settle it

Compare the recommended recipe against a strict protocol where $\alpha_1$, $\alpha_2$, temperature, and mixup-pair ratio are selected on a held-out validation set (or on held-out subjects) and the untouched test set is scored once. If accuracy no longer exceeds the equal-$\alpha$ mixup baseline, the per-teacher mixup recommendation reflects test-set selection rather than a general property.

Watch

Extended reading notes

Core claim

The central claim is that smoothness is the connecting link between mixup and knowledge distillation: KD softens the teacher's output distribution through temperature, while mixup softens labels by blending inputs and targets. Because both inject smoothness, applying mixup to the student in KD can create a synergetic effect, but too much smoothness degrades performance. The paper shows this on wearable sensor data by comparing single-teacher distillation from persistence images, multi-teacher distillation from time series plus persistence images, and the annealed multi-teacher variant (Ann.), both with and without mixup. It finds that Ann. with mixup is consistently best, and that the two teachers transfer different statistical knowledge, so using different mixup strengths for each teacher yields the highest accuracy.

Load-bearing premise

The paper's headline results depend on choosing temperature, per-teacher mixup strengths, and mixup-pair ratios on the same GENEActiv and PAMAP2 test sets that later appear as reported accuracies, so the load-bearing premise is that those test-set-selected choices are a reliable guide to unseen subjects and datasets.

Editorial extensions

If this is right

  • A time-series-only student can carry topological knowledge at inference time, avoiding the cost of computing persistence images on a wearable device.
  • Mixup is best applied to the student rather than to the teachers in this multi-teacher setting.
  • Different teachers need different mixup strengths; a single shared $\alpha$ leaves accuracy behind.
  • Temperature and mixup both add smoothness, so too much smoothness can hurt, and partial mixup provides a control knob.
  • The annealed multi-teacher student's solution space stays close to the from-scratch solution, which the parametric plots connect to flatter, less overfit behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a held-out validation protocol on new subjects would directly test whether per-teacher mixup strengths generalize, since the reported hyperparameters were selected on the same test sets that are scored.
  • Beyond the paper: because the mechanism is smoothness, the per-teacher mixup recipe should transfer to other paired representations, such as spectrograms or wavelet features as a second teacher, provided the second teacher softens different information than the first.
  • Beyond the paper: the same idea could apply to single-modality multi-teacher distillation, where teachers with different capacities or training schedules naturally produce differently smoothed outputs and could each receive its own mixup strength.
  • Beyond the paper: the paper does not isolate whether the gain comes from topological content or from having a second, differently smoothed teacher; training a second teacher on a non-topological auxiliary representation would separate these explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates the role of mixup augmentation in knowledge distillation when topological persistence images are used as a second teacher modality for wearable-sensor activity recognition. It compares single- and multi-teacher distillation strategies (standard KD, Base, and the annealed multi-teacher variant labeled Ann.), with and without mixup applied to teachers and/or student, while varying temperature, partial mixup ratios, and per-teacher mixup strengths, on GENEActiv and PAMAP2. The paper reports that the annealed multi-teacher strategy with per-teacher mixup hyperparameters gives the best accuracy (e.g., 71.22% on GENEActiv and 88.13% on PAMAP2) and concludes that the smoothness injected by mixup improves knowledge distillation. It also presents parametric plots of interpolation between trained solutions and a sensitivity analysis of the mixup strength alpha, and it argues that topological features complement time-series features in the distillation process.

Significance. If the empirical claims hold, the paper would provide a useful practical recipe for injecting topological persistence knowledge into a lightweight time-series-only student and would extend the image-domain understanding of mixup and knowledge distillation to multimodal time-series data. The study is wide in coverage: it includes single-teacher and multi-teacher distillation, several baselines (AT, SP, DIST, SimKD, AVER, EBKD, CA-MKD), two public datasets, multiple teacher-student architectures, standard deviations over three runs, and a rough efficiency comparison. These are genuine strengths. However, the central quantitative claims rest on very small accuracy differences, many within one standard deviation of three runs, and the headline configurations are selected on the same test sets later used for reporting. The contribution is therefore best understood as an exploratory empirical study whose conclusions require a more rigorous evaluation protocol before they can be accepted as general recommendations.

major comments (3)
  1. [Sections 4.5-4.6, Tables 8-16, Figures 9-10] Key hyperparameters appear to be selected on the same test sets used for the final reported results. The temperature T, the partial-mixup proportions (PMU 0.1, PMU 0.5, FMU) in Tables 8-9, and the per-teacher alpha pairs in Tables 10-15 are all chosen by comparing test-set accuracies on the GENEActiv held-out subjects and PAMAP2 leave-one-subject-out folds that are later reported as the headline outcomes. No validation split, nested cross-validation, or multiple-comparison control is described. Because the grids include many configurations (e.g., seven alpha pairs times two teacher families in Tables 10-11), the best observed pair (0.2, 0.15) on GENEActiv and (0.1, 0.15) on PAMAP2 may be a selection artifact, and the claim that per-teacher mixup strengths yield the best student is not supported by the current protocol. I ask the authors to either introduce a validation split for hyperparameter selection, use nested cross-validation, or report an independent evaluation of the selected configuration on truly held-out data.
  2. [Abstract and Section 4.3, Figure 5, Table 8] The abstract's unqualified statement that "applying mixup to training a student in KD improves performance" is contradicted by several of the paper's own results. For example, Table 8 shows that on GENEActiv with WRN16-3 teachers, TS+KD with FMU reaches 68.94% versus 69.50% without mixup, and Figure 5 shows mixup degrading PI-alone KD and Base KD in several configurations. The claim should be qualified to the specific strategies where improvement is observed (e.g., the annealed multi-teacher setup) and should acknowledge the documented degradation cases. As written, the abstract overstates the findings relative to the evidence in the manuscript.
  3. [Section 4.1.2 and Tables 8-15] The reported improvements are generally smaller than the run-to-run variability. On PAMAP2, the standard deviations are around 2.2 percentage points across all reported cells, while the headline differences between configurations are between 0.1 and 0.5 percentage points (e.g., 88.13 vs. 87.98 in Table 15 and 87.98 vs. 87.12 in Table 9). On GENEActiv, the best gains are about 0.5 points (71.22 vs. 70.72 in Table 10) with standard deviations of 0.1-0.2. With three runs per configuration, these differences are not statistically distinguishable, and no significance test, confidence interval, or paired analysis across subjects or folds is provided. Without such analysis, the central conclusion that mixup, and especially per-teacher mixup strengths, improves the annealed student is not established.
minor comments (5)
  1. [Section 4.1.1] The phrase "writ-worn tri-axial accelerometer" contains a typo; it should be "wrist-worn."
  2. [Table 1] The GFLOPs and processing-time columns are misaligned in the rendered table, which makes the efficiency comparison difficult to read.
  3. [Section 4.5.1 and Figures 9-10] The sentence beginning "For both KD with time-series and Ann..." contains a duplicated teacher description ("WRN16-3 teacher and T is 12 for WRN16-3 teacher"); please clarify which teacher/student configuration each temperature statement refers to.
  4. [Introduction, Section 1] The sentence "In section 6, we discuss our findings and conclusions" is inconsistent with the actual structure, where Section 5 is Discussion and Section 6 is Conclusion; please update the section references.
  5. [Section 3.2] The training objective in Eq. (7) is introduced without a step-by-step training recipe; adding pseudocode or a short algorithm box would improve reproducibility, since the per-teacher mixup pairs and sampling order are load-bearing for the method.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the paper's central claims are empirical measurements on public benchmark datasets, and the cited self-works are prior experiments, not definitions of the result.

full rationale

This is an empirical study rather than a formal derivation, so there is no equation-level circularity in which an output is identified with an input by construction. The multi-teacher KD loss (Eqs. 5-7) and the mixup loss (Eqs. 1-2) are standard definitions taken from the literature; the paper's claims about which strategy works best are supported by Tables 2-16 and Figures 4-11, which are measurements on the public GENEActiv and PAMAP2 datasets with comparisons against external baselines. The 'smoothness connects mixup and KD' premise is imported from the authors' prior WACV 2023 study [15], but it is used as a testable hypothesis and is not invoked as a uniqueness theorem or as a formal proof step; the present paper's contribution is the new empirical evaluation on time-series and topological persistence, which stands or falls on the data. Self-citations to [3] and [15] are extensions of the authors' own earlier methods, but the measured accuracies are not defined by those citations. The main methodological weakness is that hyperparameters such as temperature, partial-mixup proportions, and per-teacher alpha pairs appear to be selected using the same test sets that are later reported as outcomes (Sec. 4.5-4.6, Tables 8-16), and gains are small with overlapping standard deviations; this is a generalization and selection-bias concern, not a circularity concern, because the reported numbers are empirical results rather than predictions forced by construction. No fitted parameter is renamed as a prediction, and no load-bearing premise is justified solely by a self-citation chain. Accordingly the circularity score is 1.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on a standard deep-learning pipeline plus several domain assumptions about persistence images, logit comparability, annealing, and the smoothness link. The most fragile element is the 'smoothness' transfer assumption and the test-set tuning, which the axiom ledger captures as ad hoc to this paper's design.

free parameters (7)
  • alpha (mixup strength) = 0.1 default; varied 0.05 to 0.4
    Controls strength of input/label interpolation in Eq. (1); set by hand and tuned in Table 16.
  • temperature T = 4 default; 12 often best
    Controls softness of teacher logits in Eq. (4); tuned per dataset in Figs. 6, 9, 10.
  • tau (KD loss weight) = 0.7 (GENEActiv), 0.99 (PAMAP2)
    Balances hard-label and KD terms in Eq. (3); taken from prior work rather than fitted here.
  • eta (teacher weighting) = 0.7 (GENEActiv), 0.3 (PAMAP2)
    Weights the two teachers in Eq. (5); taken from prior work.
  • alpha1, alpha2 = (0.15, 0.2) for GENEActiv, (0.1, 0.15) for PAMAP2
    Per-teacher mixup strengths; selected by test-set search in Tables 10-11.
  • PMU proportion = 0.1, 0.5, or 1.0 depending on configuration
    Fraction of mixup pairs per batch; tuned on test set in Tables 8-9, 14-15.
  • PI parameters = Gaussian std 0.25 (GENEActiv) / 0.015 (PAMAP2); range [-10,10] / [-1,1]; image 64x64
    Control the persistence image representation in Section 4.1.2; inherited from prior studies, not fitted to this paper's results.
assumptions (4)
  • domain assumption Persistence images computed from time series capture complementary shape information that improves activity classification.
    Invoked in Section 2.2 and used to justify the PI teacher; relies on prior work [2,3,4] without re-derivation.
  • domain assumption Logits from teachers with different architectures and input modalities can be combined into a single student via a weighted KD loss.
    Equations (5)-(7) assume comparability of teacher logits; the paper does not analyze the conditions under which this holds.
  • ad hoc to paper The smoothing mechanism linking mixup and KD in image models transfers to time-series and topological persistence data.
    Framing in Sections 1 and 3.2; the paper assumes this connection without directly testing it, and relies on it to motivate the experimental design.
  • domain assumption Annealing, i.e., initializing the student from a model learned from scratch, reduces the knowledge gap between heterogeneous teachers.
    Adopted from Jeon et al. [3] and used in the Ann. strategy; not independently justified here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Role of Mixup in Topological Persistence Based Knowledge Distillation for Wearable Sensor Data." pith.science (2026). https://pith.science/paper/FZAN6DSS

@misc{pith2026250200779,
  author       = {Pith},
  title        = {Pith review of: Role of Mixup in Topological Persistence Based Knowledge Distillation for Wearable Sensor Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FZAN6DSS}},
  note         = {Machine review of arXiv:2502.00779}
}
read the original abstract

The analysis of wearable sensor data has enabled many successes in several applications. To represent the high-sampling rate time-series with sufficient detail, the use of topological data analysis (TDA) has been considered, and it is found that TDA can complement other time-series features. Nonetheless, due to the large time consumption and high computational resource requirements of extracting topological features through TDA, it is difficult to deploy topological knowledge in various applications. To tackle this problem, knowledge distillation (KD) can be adopted, which is a technique facilitating model compression and transfer learning to generate a smaller model by transferring knowledge from a larger network. By leveraging multiple teachers in KD, both time-series and topological features can be transferred, and finally, a superior student using only time-series data is distilled. On the other hand, mixup has been popularly used as a robust data augmentation technique to enhance model performance during training. Mixup and KD employ similar learning strategies. In KD, the student model learns from the smoothed distribution generated by the teacher model, while mixup creates smoothed labels by blending two labels. Hence, this common smoothness serves as the connecting link that establishes a connection between these two methods. In this paper, we analyze the role of mixup in KD with time-series as well as topological persistence, employing multiple teachers. We present a comprehensive analysis of various methods in KD and mixup on wearable sensor data.

Figures

Figures reproduced from arXiv: 2502.00779 by the authors.

Figure 1
Figure 1. time-series data and its corresponding PD and PI. Higher persistence in PD is represented with brighter color in PI. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Strategies to leverage topological persistence in KD. (a) utilizes a single teacher trained with PIs. (b) uses different teachers trained with [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Approaches for incorporating mixup in KD. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Results of various models trained from scratch with or without mixup. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Results of various models trained with KD and mixup. TS and PI are results of students trained with KD. M. denotes using mixup. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Results of various models with different temperature in KD. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: t-SNE plots of output for various models on GENEActiv. A teacher is WRN16-3 and a student is WRN16-1, which are trained with [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Results of various approaches in KD, trained with mixup. Brackets denote (Teacher, Student). [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Results of various models with different temperature and mixup in KD on GENEActiv. Mixup is applied when a student is trained. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Results of various models with different temperature and mixup in KD on PAMAP2. Mixup is applied when a student is trained. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Parametric plots with accuracy (%) for various pairs of models on GENEActiv. Brackets denote solutions (z ∗ a , z ∗ b ). κ = 0 implies to z ∗ a and κ = 1 to z ∗ b . “Student” is a model learned from scratch. 4.7.2. Mixup Hyper-parameter α To explore the performance on…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 55 canonical work pages

  1. [1]

    Nawar, F

    A. Nawar, F. Rahman, N. Krishnamurthi, A. Som, P. Turaga, Topological descriptors for parkinson’s disease classification and regression analysis, in: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine & Biology Society, 2020, pp. 793– 797

    work page 2020

  2. [2]

    A. Som, H. Choi, K. N. Ramamurthy, M. P. Buman, P. Turaga, Pi-net: A deep learning approach to extract topological persistence images, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 834–835

    work page 2020

  3. [3]

    E. S. Jeon, H. Choi, A. Shukla, Y . Wang, M. P. Buman, P. Turaga, Topological knowledge distillation for wearable sensor data, in: Proceed- ings of the Asilomar Conference on Signals, Systems, and Computers, 2022, pp. 837–842. doi:10.1109/IEEECONF56349.2022. 10052019

    work page arXiv 2022

  4. [4]

    Adams, T

    H. Adams, T. Emerson, M. Kirby, R. Neville, C. Peterson, P. Shipman, S. Chepushtanova, E. Hanson, F. Motta, L. Ziegelmeier, Persistence images: A stable vector representation of persistent homology, Journal of Machine Learning Research 18 (2017)

    work page 2017

  5. [5]

    L. M. Seversky, S. Davis, M. Berger, On time-series topological data analysis: New data and opportunities, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016, pp. 59–67

    work page 2016

  6. [6]

    Munch, A user’s guide to topological data analysis, Journal of Learning Analytics 4 (2) (2017)

    E. Munch, A user’s guide to topological data analysis, Journal of Learning Analytics 4 (2) (2017)

    work page 2017

  7. [7]

    Barnes, L

    D. Barnes, L. Polanco, J. A. Perea, A comparative study of machine learning methods for persistence diagrams, Frontiers of Artificial Intelligence 4 (2021) 681174

    work page 2021

  8. [8]

    Edelsbrunner, J

    H. Edelsbrunner, J. L. Harer, Computational topology: an introduction, American Mathematical Society, 2022

    work page 2022

Show all 70 references
  1. [9]

    E. S. Jeon, H. Choi, A. Shukla, Y . Wang, M. P. Buman, P. Turaga, Constrained adaptive distillation based on topological persistence for wearable sensor data, IEEE Transactions on Instrumentation and Measurement 72 (2023) 1–14. doi:10.1109/TIM.2023.3329818

    2023

  2. [10]

    Rieck, T

    B. Rieck, T. Yates, C. Bock, K. Borgwardt, G. Wolf, N. Turk-Browne, S. Krishnaswamy, Uncovering the topology of time-varying fmri data using cubical persistence, Advances in Neural Information Processing Systems 33 (2020) 6900–6912. 19

    2020

  3. [11]

    Jiang, B

    F. Jiang, B. Xu, Z. Zhu, B. Zhang, Topological data analysis approach to extract the persistent homology features of ballistocardiogram signal in unobstructive atrial fibrillation detection, IEEE Sensors Journal 22 (7) (2022) 6920–6930

    2022

  4. [12]

    Yan, Y .-S

    Y . Yan, Y .-S. Liu, C.-D. Li, J.-H. Wang, L. Ma, J. Xiong, X.-X. Zhao, L. Wang, Topological descriptors of gait nonlinear dynamics toward freezing-of-gait episodes recognition in parkinson’s disease, IEEE Sensors Journal 22 (5) (2022) 4294–4304

    2022

  5. [13]

    Chazal, B

    F. Chazal, B. Michel, An introduction to topological data analysis: Fundamental and practical aspects for data scientists, Frontiers in Artificial Intelligence 4 (2021)

    2021

  6. [14]

    H. Wang, S. Lohit, M. N. Jones, Y . Fu, What makes a ”good” data augmentation in knowledge distillation - a statistical perspective, Advances in Neural Information Processing Systems 35 (2022) 13456–13469

    2022

  7. [15]

    H. Choi, E. S. Jeon, A. Shukla, P. Turaga, Understanding the role of mixup in knowledge distillation: An empirical study, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 2319–2328

    2023

  8. [16]

    X. Li, H. Xiong, C. Xu, D. Dou, Smile: Self-distilled mixup for efficient transfer learning, arXiv preprint arXiv:2103.13941 (2021)

    2021 arXiv

  9. [17]

    C. Yang, Z. An, H. Zhou, L. Cai, X. Zhi, J. Wu, Y . Xu, Q. Zhang, Mixskd: Self-knowledge distillation from mixup for image recognition, in: European Conference on Computer Vision, Springer, 2022, pp. 534–551

    2022

  10. [18]

    G. Xu, Z. Liu, C. C. Loy, Computation-efficient knowledge distillation via uncertainty-aware mixup, Pattern Recognition 138 (2023) 109338

    2023

  11. [19]

    C. M. Bishop, Training with noise is equivalent to tikhonov regularization, Neural computation 7 (1) (1995) 108–116

    1995

  12. [20]

    S. Chen, E. Dobriban, J. H. Lee, A group-theoretic framework for data augmentation, Journal of Machine Learning Research 21 (245) (2020) 1–71

    2020

  13. [21]

    R. Shen, S. Bubeck, S. Gunasekar, Data augmentation as feature manipulation, in: International conference on machine learning, PMLR, 2022, pp. 19773–19808

    2022

  14. [22]

    Allen-Zhu, Y

    Z. Allen-Zhu, Y . Li, Towards understanding ensemble, knowledge distillation and self-distillation in deep learning, in: The Eleventh Interna- tional Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=Uuf2q9TfXGA

    2023

  15. [23]

    S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, Y . Yoo, Cutmix: Regularization strategy to train strong classifiers with localizable features, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 6023–6032

    2019

  16. [24]

    Allen-Zhu, Y

    Z. Allen-Zhu, Y . Li, Feature purification: How adversarial training performs robust deep learning, in: 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), IEEE, 2022, pp. 977–988

    2021

  17. [25]

    T. Zhao, Y . Liu, L. Neves, O. Woodford, M. Jiang, N. Shah, Data augmentation for graph neural networks, in: Proceedings of the aaai conference on artificial intelligence, V ol. 35, 2021, pp. 11015–11023

    2021

  18. [26]

    D. Zou, Y . Cao, Y . Li, Q. Gu, The benefits of mixup for feature learning, in: International Conference on Machine Learning, PMLR, 2023, pp. 43423–43479

    2023

  19. [27]

    Beyer, X

    L. Beyer, X. Zhai, A. Royer, L. Markeeva, R. Anil, A. Kolesnikov, Knowledge distillation: A good teacher is patient and consistent, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10925–10934

    2022

  20. [28]

    Zhang, M

    H. Zhang, M. Cisse, Y . N. Dauphin, D. Lopez-Paz, mixup: Beyond empirical risk minimization, in: Proceedings of the International Confer- ence on Learning Representations, 2018. URL https://openreview.net/forum?id=r1Ddp1-Rb

    2018

  21. [29]

    Verma, A

    V . Verma, A. Lamb, C. Beckham, A. Najafi, I. Mitliagkas, D. Lopez-Paz, Y . Bengio, Manifold mixup: Better representations by interpolating hidden states, in: Proceedings of the International Conference on Machine Learning, 2019, pp. 6438–6447

    2019

  22. [30]

    J.-H. Kim, W. Choo, H. Jeong, H. O. Song, Co-mixup: Saliency guided joint mixup with supermodular diversity, arXiv preprint arXiv:2102.03065 (2021)

    2021 arXiv

  23. [31]

    L. N. Darlow, A. Joosen, M. Asenov, Q. Deng, J. Wang, A. Barker, Tsmix: time series data augmentation by mixing sources, in: Proceedings of the 3rd Workshop on Machine Learning and Systems, 2023, pp. 109–114

    2023

  24. [32]

    Aggarwal, J

    K. Aggarwal, J. Srivastava, Embarrassingly simple mixup for time-series, arXiv preprint arXiv:2304.04271 (2023)

    2023 arXiv

  25. [33]

    Y . Zhou, L. You, W. Zhu, P. Xu, Improving time series forecasting with mixup data augmentation, in: ECML PKDD 2023 International Workshop on Machine Learning for Irregular Time Series, 2023. URL https://www.amazon.science/publications/improving-time-series-forecasting-with-mi...

    2023

  26. [34]

    Y . Wang, R. Behroozmand, L. P. Johnson, L. Bonilha, J. Fridriksson, Topological signal processing and inference of event-related potential response, Journal of Neuroscience Methods 363 (2021) 109324. doi:https://doi.org/10.1016/j.jneumeth.2021.109324

    2021

  27. [35]

    Gholizadeh, W

    S. Gholizadeh, W. Zadrozny, A short survey of topological data analysis in time series and systems analysis, arXiv preprint arXiv:1809.10745 (2018)

    2018 arXiv

  28. [36]

    S. Zeng, F. Graf, C. Hofer, R. Kwitt, Topological attention for time series forecasting, Advances in Neural Information Processing Systems 34 (2021) 24871–24882

    2021

  29. [37]

    B. J. Stolz, Outlier-robust subsampling techniques for persistent homology, Journal of Machine Learning Research 24 (2023)

    2023

  30. [38]

    Bucilu ˇa, R

    C. Bucilu ˇa, R. Caruana, A. Niculescu-Mizil, Model compression, in: Proceedings of the ACM International Conference on Knowledge Discovery and Data Mining (KDD), 2006, pp. 535–541

    2006

  31. [39]

    Hinton, O

    G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, in: Proceedings of the NeurIPS Deep Learning and Represen- tation Learning Workshop, V ol. 2, 2015

    2015

  32. [40]

    J. H. Cho, B. Hariharan, On the efficacy of knowledge distillation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 4794–4802

    2019

  33. [41]

    J. Gou, B. Yu, S. J. Maybank, D. Tao, Knowledge distillation: A survey, International Journal of Computer Vision 129 (6) (2021) 1789–1819

    2021

  34. [42]

    Zagoruyko, N

    S. Zagoruyko, N. Komodakis, Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer, in: Proceedings of the International Conference on Learning and Representations (ICLR), 2017, pp. 1–13

    2017

  35. [43]

    F. Tung, G. Mori, Similarity-preserving knowledge distillation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1365–1374

    2019

  36. [44]

    Y . Liu, W. Zhang, J. Wang, Adaptive multi-teacher multi-level knowledge distillation, Neurocomputing 415 (2020) 106–113

    2020

  37. [45]

    Zhang, D

    H. Zhang, D. Chen, C. Wang, Confidence-aware multi-teacher knowledge distillation, in: Proceedings of the IEEE International Conference 20 on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 4498–4502

    2022

  38. [46]

    S. You, C. Xu, C. Xu, D. Tao, Learning from multiple teacher networks, in: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 1285–1294

    2017

  39. [47]

    Q. Wang, S. Lohit, M. J. Toledo, M. P. Buman, P. Turaga, A statistical estimation framework for energy expenditure of physical activities from a wrist-worn accelerometer, in: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Soc...

    2016

  40. [48]

    E. S. Jeon, A. Som, A. Shukla, K. Hasanaj, M. P. Buman, P. Turaga, Role of data augmentation strategies in knowledge distillation for wearable sensor data, IEEE Internet of Things Journal 9 (14) (2022) 12848–12860

    2022

  41. [49]

    Reiss, D

    A. Reiss, D. Stricker, Introducing a new benchmarked dataset for activity monitoring, in: Proceedings of the International Symposium on Wearable Computers, 2012, pp. 108–109

    2012

  42. [50]

    Jordao, A

    A. Jordao, A. C. Nazare Jr, J. Sena, W. R. Schwartz, Human activity recognition based on wearable sensor data: A standardization of the state-of-the-art, arXiv preprint arXiv:1806.05226 (2018)

    2018 arXiv

  43. [51]

    N. Saul, C. Tralie, Scikit-tda: Topological data analysis for python (2019). doi:10.5281/zenodo.2533369. URL https://doi.org/10.5281/zenodo.2533369

    2019 doi

  44. [52]

    Zagoruyko, N

    S. Zagoruyko, N. Komodakis, Wide residual networks, in: Proceedings of the British Machine Vision Conference, 2016

    2016

  45. [53]

    Komodakis, S

    N. Komodakis, S. Zagoruyko, Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer, in: Proceedings of the International Conference on Learning and Representations (ICLR), 2017, pp. 1–13

    2017

  46. [54]

    Chen, J.-P

    D. Chen, J.-P. Mei, H. Zhang, C. Wang, Y . Feng, C. Chen, Knowledge distillation with the reused teacher classifier, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11933–11942

    2022

  47. [55]

    Huang, S

    T. Huang, S. You, F. Wang, C. Qian, C. Xu, Knowledge distillation from a stronger teacher, Advances in Neural Information Processing Systems 35 (2022) 33716–33727

    2022

  48. [56]

    K. Kwon, H. Na, H. Lee, N. S. Kim, Adaptive knowledge distillation based on entropy, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 7409–7413

    2020

  49. [57]

    Cortes, V

    C. Cortes, V . Vapnik, Support-vector networks, Machine learning 20 (3) (1995) 273–297

    1995

  50. [58]

    H. Choi, Q. Wang, M. Toledo, P. Turaga, M. Buman, A. Srivastava, Temporal alignment improves feature quality: an experiment on activity recognition with accelerometer data, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 349–357

    2018

  51. [59]

    Y . Chen, Y . Xue, A deep learning approach to human activity recognition based on single accelerometer, in: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, 2015, pp. 1488–1492

    2015

  52. [60]

    Ha, J.-M

    S. Ha, J.-M. Yun, S. Choi, Multi-modal convolutional neural networks for activity recognition, in: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, 2015, pp. 3017–3022

    2015

  53. [61]

    S. Ha, S. Choi, Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors, in: Proceedings of the International Joint Conference on Neural Networks, 2016, pp. 381–388

    2016

  54. [62]

    Catal, S

    C. Catal, S. Tufekci, E. Pirmit, G. Kocabag, On the use of ensemble of classifiers for accelerometer-based activity recognition, Applied Soft Computing 37 (2015) 1018–1022

    2015

  55. [63]

    H.-J. Kim, M. Kim, S.-J. Lee, Y . S. Choi, An analysis of eating activities for automatic food type recognition, in: Proceedings of the Asia Pacific Signal and Information Processing Association Annual Summit and Conference, 2012, pp. 1–5

    2012

  56. [64]

    Rosenberg, J

    A. Rosenberg, J. Hirschberg, V-measure: A conditional entropy-based external cluster evaluation measure, in: Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 2007, pp. 410–420

    2007

  57. [65]

    DeVries, G

    T. DeVries, G. W. Taylor, Improved regularization of convolutional neural networks with cutout, arXiv preprint arXiv:1708.04552 (2017)

    2017 arXiv

  58. [66]

    I. J. Goodfellow, O. Vinyals, A. M. Saxe, Qualitatively characterizing neural network optimization problems, arXiv preprint arXiv:1412.6544 (2014)

    2014 arXiv

  59. [67]

    F. Zhu, Z. Cheng, X.-Y . Zhang, C.-L. Liu, Rethinking confidence calibration for failure prediction, in: Proceedings of the European Confer- ence on Computer Vision (ECCV), 2022, pp. 518–536

    2022

  60. [68]

    N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, P. T. P. Tang, On large-batch training for deep learning: Generalization gap and sharp minima, in: Proceedings of the International Conference on Learning and Representations (ICLR), 2017

    2017

  61. [69]

    W. Han, X. Dong, Y . Zhang, D. Crandall, C.-Z. Xu, J. Shen, Asymmetric convolution: An efficient and generalized method to fuse feature maps in multiple vision tasks, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  62. [70]

    X. Dong, J. Shen, F. Porikli, J. Luo, L. Shao, Adaptive siamese tracking with a compact latent network, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (7) (2022) 8049–8062. 21

    2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.