Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Smooth-Distill: A Self-distillation Framework for Multitask Learning with Wearable Sensor Data

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A wearable-sensing CNN that learns from its own smoothed weights matches two-stage self-distillation at half the training time, beating single-task and multitask baselines on both activity and placement recognition.

desk verdict Solid engineering paper: the new sleep dataset and multitask application are useful, but Eq. 4 is uncredited Mean Teacher and the split protocol is ambiguous, so the headline claims are not yet supportable. read the letter →

arxiv 2507.00061 v1 pith:DJVJPHJY submitted 2025-06-27 cs.LG cs.AIeess.SP

Hoang-Dieu Vu , Duc-Nghia Tran , Quang-Tu Pham , Hieu H. Pham , Nicolas Vuillerme , Duc-Tan Tran This is my paper · ORCID
classification cs.LGcs.AIeess.SP
keywords self-distillationmultitasklearninghumanactivityrecognitionsensorplacementdetectionwearableaccelerometerdatasleeppostureexponentialmovingaverageteacherknowledgedistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the regularization benefits of knowledge distillation for multitask learning need no separate teacher: a CNN predicting both activity/sleep posture and sensor placement from accelerometer windows can be trained to match a smoothed copy of its own parameters, updated after every step as an exponential moving average of the student (Eq. 4 with $\beta = 0.999$). On three datasets, a newly collected 12-posture sleep set and the public MHealth and WISDM sets, Smooth-Distill reports the highest or tied-highest test accuracy and F1 for both tasks in nearly every cell of Table 5, with its largest gains on the imbalanced MHealth activity task (test F1 53.97 versus 48.87 for born-again distillation). Because no teacher is pre-trained, training time on the Sleep dataset drops from 8280 to 4368 seconds and energy from 0.920 to 0.485 kWh compared with the two-stage baseline, which matters for wearable settings where models are retrained frequently. If true, the result turns self-distillation from a doubling of training cost into a one-pass regularizer.

What carries the argument

The load-bearing object is the smoothed teacher, defined by the recurrence $\theta^T_t = \beta\,\theta^T_{t-1} + (1-\beta)\,\theta^S_t$ with $\beta = 0.999$: at every minibatch the student is updated by backpropagation on the combined task losses, and the teacher is then nudged a fraction $(1-\beta)$ toward the student, so the teacher lags behind the student and carries a lower-variance average of its history. Its job is to emit stable soft targets whose KL divergence to the student's predictions, added to each task's cross-entropy with weight $\lambda = 0.5$ in Eq. 5, steers the student toward confident and consistent predictions on both tasks while smoothing the noise of individual gradient steps, which the paper credits for the observed convergence stability and reduced overfitting relative to plain multitask training. The accompanying architecture MTL-net is a CNN that takes 100-sample accelerometer windows with a step of 60 and branches into two classification heads, one per task.

What would settle it

Re-run the three experiments with a strict per-participant split (train on some participants, test on held-out participants, with identical folds and hyperparameters): if Smooth-Distill's edge over plain multitask training shrinks to within noise or reverses on the Sleep and WISDM datasets, window leakage rather than the distillation mechanism explains the reported gains. A second, cheaper check is to train Smooth-Distill with $\lambda = 0$ (distillation term removed) and with the teacher frozen at its initialization; if either variant matches the full method, the EMA teacher itself is not the active ingredient.

Watch

Extended reading notes

Core claim

Smooth-Distill's central claim is that a historical average of the training model is a sufficient teacher for multitask distillation. The teacher is never trained: after each student update its parameters move by $\theta^T_t = \beta\,\theta^T_{t-1} + (1-\beta)\,\theta^S_t$ with $\beta = 0.999$, so the teacher is an exponential moving average of the student's own weights, and the student minimizes, for each of two tasks, its cross-entropy to the labels plus $\lambda$ times the KL divergence between its softened logits and the teacher's, with temperature $\tau = 3.0$ and task weight $\alpha = 0.5$ (Eq. 5). With $\lambda = 0.5$, this yields the best or tied-best test accuracy and F1 across the Sleep, MHealth, and WISDM datasets for both posture/activity recognition and device-placement detection (Table 5), including 68.59% accuracy on WISDM Task 1, about four points above the single-task baseline. On the Sleep dataset the training time is 4368 seconds versus 8280 for Born-Again self-distillation, with 0.485 kWh versus 0.920 kWh of energy (Table 7). The paper's paired t-tests place Smooth-Distill and Born-Again in a statistically indistinguishable top tier ($p > 0.7$), both significantly ahead of single-task, multitask, and dropout-based distillation ($p < 0.001$); so within the paper's own statistics the claim is parity in accuracy with Born-Again, superiority in cost, and clear superiority over the non-distillation baselines.

Load-bearing premise

The results stand or fall on the 80:20 train/test split being leakage-free: the paper says the windows were split 80:20 but never states whether all windows of a participant stayed on one side, so if windows from the same person appear in both partitions the reported accuracies are optimistically inflated and generalization to new users is not demonstrated.

Editorial extensions

If this is right

  • Knowledge distillation for multitask wearable sensing becomes a one-pass procedure: no teacher is pre-trained, so a model can be retrained as often as new sensor data or labels arrive at roughly half the training time and energy of born-again distillation.
  • On imbalanced activity sets the benefit concentrates where it matters: Smooth-Distill raises MHealth Task 1 test F1 to 53.97 versus 48.87 for born-again and about 50 for single-task training, indicating the smoothing regularizer helps rare classes rather than only easy ones.
  • The benefit is not tied to one architecture: Smooth-Distill training improves LSTM, bidirectional LSTM, GRU, AnpoNet, and the CNN-based MTL-Net across the three datasets, and models retain usable performance with as little as 10% of the training data, with gains leveling off near a 70% training ratio.
  • For accuracy-critical deployments the practical choice is cost-driven, because the t-tests place Smooth-Distill and born-again distillation in the same statistical tier; the roughly 47% reduction in training time and energy becomes the deciding factor.
  • The released Sleep dataset, with 12 postures across three wearing positions, provides a public benchmark for joint sleep-posture and sensor-placement recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I would expect the same EMA-teacher mechanism to transfer beyond accelerometry: nothing in the loss or update rule is modality-specific, so multitask gyroscope, PPG, ECG, or IMU pipelines, and task pairs beyond activity-plus-placement, should see similar one-pass regularization, although the paper only tests acceleration data.
  • An untested boundary condition follows from $\beta = 0.999$: early in training the teacher barely differs from the student, so under very short training budgets or very small batches the distillation signal is weak and could even destabilize; a sweep over training length and batch size would reveal where the advantage appears.
  • Because the paper reports an 80:20 split without stating whether it is per participant or per window, the honest generalization estimate is still open; a leave-participants-out rerun on all three datasets would settle whether the gains survive on unseen wearers.
  • Since the paper's own statistics tie Smooth-Distill to born-again accuracy, the practical reading is that the method's value is chiefly operational: near-parity accuracy at roughly half the training cost, which a deployment-style comparison on retraining frequency and energy budgets would demonstrate directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Smooth-Distill, a self-distillation framework for multitask learning on wearable accelerometer data. The method uses an exponential moving average of the student network's parameters as a teacher (Eq. 4), eliminating the separate teacher-training phase of conventional distillation. The authors introduce a new sleep-posture dataset with 12 classes and three sensor placements, and evaluate on that dataset plus MHealth and WISDM. They report that Smooth-Distill achieves higher accuracy and F1 than single-task, multitask, SD-Dropout, and Born-Again distillation baselines, with substantially lower training time than Born-Again. The paper also provides ablation studies on the distillation weight, model architecture, and training ratio, as well as statistical significance tests.

Significance. If the empirical claims are validated, the contribution is practically relevant: it offers a computationally cheaper alternative to two-stage knowledge distillation for multitask wearable-sensor classification, and the new 12-posture, 3-placement sleep dataset is a useful public resource. The paper ships a code link and describes careful ablations, which are strengths. However, the headline claim of consistent superiority is weakened by the paper's own significance tests, and the central generalization claim depends on an unspecified data-partitioning protocol. The novelty of the EMA-teacher mechanism is also somewhat overstated given prior work on mean-teacher style self-distillation.

major comments (3)
  1. [Section 4.2.1, Data Preprocessing] The train/test partition is described only as an '80:20 ratio' with no statement about whether the split is per participant or per window. Because windows are extracted with length 100 and step 60, overlapping windows from the same participant or recording can appear in both partitions if the split is per window. This would inflate test accuracy and make the generalization claims in Sections 5.1 and 6.5 unsupported. The authors must clarify the split granularity; if it is per window, they should add a subject-independent evaluation (e.g., leave-one-subject-out) to demonstrate generalization to new users, especially for the Sleep dataset where the reported 92.22% Task 1 test accuracy with a 0.53 standard deviation is suspiciously high for 12 fine-grained posture classes.
  2. [Section 5.1, Table 8, and Section 6.2] The claim that Smooth-Distill 'consistently outperforms alternative approaches' is contradicted by the paper's own significance tests: the p-value against Born-Again SD is 0.7173 for accuracy and 0.7431 for F1, which the text itself describes as 'statistically indistinguishable performance.' I recommend tempering the abstract and conclusion to 'matches Born-Again self-distillation at substantially lower training cost' unless the authors provide evidence of superiority on a metric that matters for deployment, such as new-user generalization. Additionally, the paper should clarify how many observations enter the paired t-test (five folds) and whether multiple-comparison corrections were applied, given that 10 pairwise comparisons are reported.
  3. [Section 6.1 and Table 7] The training-time comparison is internally inconsistent. The text says Smooth-Distill 'increases training time to approximately 1.72 times the baseline,' but Table 7 lists Singletask training time as 5012 s and Smooth-Distill as 4368 s, which is a ratio of 0.87. If the intended baseline is the full singletask approach (two separate models, 2 x 5012 = 10024 s), then Smooth-Distill is about 0.44x of that baseline, not 1.72x. The same inconsistency appears in Section 7. Please correct the ratio and the '72% more training time' statement. The supported claim is that Smooth-Distill takes roughly 47% less time than Born-Again (4368 vs 8280 s), which should be stated precisely.
minor comments (5)
  1. [Section 1, Section 4.1.1, Table 2] The contribution bullet states the Sleep dataset was collected from 23 participants, while Section 4.1.1 and Table 2 state 24 participants; please correct this inconsistency.
  2. [Section 4.2.1] The relation between 'five-fold cross-validation' and 'five independent random seeds' is ambiguous; please state how many runs are performed per fold and how seeds interact with fold divisions.
  3. [Section 3.3] The EMA-teacher mechanism is closely related to the Mean Teacher method (Tarvainen and Valpola, 2017), which is not cited; please discuss the difference relative to that work (e.g., distillation loss versus consistency loss, and the multitask setting) to position the novelty more precisely.
  4. [Table 8] The significance test is described as a 'paired t-test comparing mean fold accuracy,' but with only five folds the test has very low power; please report the number of paired observations and consider a signed-rank test or confidence intervals.
  5. [Figure 3] The confusion matrices are difficult to read in the grayscale preprint; please ensure the figure is legible in the final printed version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the EMA teacher in Eq. 4 is a smoothed function of the student's own parameters, not of the target labels or of the reported test outcomes.

full rationale

The paper's central derivation is empirical and self-contained rather than circular. The teacher model is defined by Eq. 4 as an exponential moving average of the student parameters, so its predictions are functions of the student's own history; the distillation loss in Eq. 2 measures KL divergence between teacher and student soft outputs, and the total loss in Eq. 5 still includes cross-entropy terms against ground-truth labels. Nothing in these equations encodes the reported accuracy or F1 results, and no parameter is fitted to the test set in a way that would force the advertised ranking of methods. The contributions are validated on two external public benchmarks (MHealth and WISDM) in addition to the newly collected Sleep dataset, so the headline performance claims are not derived from a self-referential premise. The only self-citation is reference [37], used to name the AnpoNet architecture in the ablation study; this is a comparison baseline, not a load-bearing justification for the main result. The acknowledged limitations about the original dataset and validation protocols (Section 6.5) concern generalization and dataset rigor, not circular reasoning. Overall, the Smooth-Distill derivation does not reduce to its inputs by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper is empirical; listed hyperparameters are experimental settings rather than parameters fitted to the outcome. No free parameter is used to force the central result; the reported gains are empirical outcomes of a fixed configuration.

free parameters (6)
  • beta (teacher smoothing) = 0.999
    EMA coefficient in Eq. 4, chosen without ablation; standard in mean-teacher methods.
  • lambda (distillation weight) = 0.5
    Ablated in Section 5.3.1; chosen as optimal for Sleep, marginal for other datasets.
  • alpha (task weight) = 0.5
    Fixed task-balancing weight for all multitask configurations.
  • tau (temperature) = 3.0
    Distillation temperature, fixed for Born-Again and Smooth-Distill.
  • window length = 100
    Sliding window size in samples, chosen for preprocessing.
  • step size = 60
    Sliding window stride in samples.
assumptions (4)
  • domain assumption The train/test split is leakage-free or subject-independent.
    Section 4.2.1 describes an 80:20 split but does not specify whether it is per participant or per window; the validity of all reported test accuracies depends on this assumption.
  • domain assumption The three datasets are correctly labeled and sensor placement annotations are accurate.
    The sleep dataset was self-collected; MHealth and WISDM are public benchmarks. No label-quality checks are reported.
  • domain assumption The two tasks (activity and placement) are related enough for multitask learning to help.
    Section 1 argues this based on cited literature rather than a formal analysis.
  • domain assumption The EMA teacher with beta=0.999 provides a stable, non-divergent training signal.
    Section 3.3 asserts stability; no theoretical proof is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Smooth-Distill: A Self-distillation Framework for Multitask Learning with Wearable Sensor Data." pith.science (2026). https://pith.science/paper/DJVJPHJY

@misc{pith2026250700061,
  author       = {Pith},
  title        = {Pith review of: Smooth-Distill: A Self-distillation Framework for Multitask Learning with Wearable Sensor Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJVJPHJY}},
  note         = {Machine review of arXiv:2507.00061}
}
read the original abstract

This paper introduces Smooth-Distill, a novel self-distillation framework designed to simultaneously perform human activity recognition (HAR) and sensor placement detection using wearable sensor data. The proposed approach utilizes a unified CNN-based architecture, MTL-net, which processes accelerometer data and branches into two outputs for each respective task. Unlike conventional distillation methods that require separate teacher and student models, the proposed framework utilizes a smoothed, historical version of the model itself as the teacher, significantly reducing training computational overhead while maintaining performance benefits. To support this research, we developed a comprehensive accelerometer-based dataset capturing 12 distinct sleep postures across three different wearing positions, complementing two existing public datasets (MHealth and WISDM). Experimental results show that Smooth-Distill consistently outperforms alternative approaches across different evaluation scenarios, achieving notable improvements in both human activity recognition and device placement detection tasks. This method demonstrates enhanced stability in convergence patterns during training and exhibits reduced overfitting compared to traditional multitask learning baselines. This framework contributes to the practical implementation of knowledge distillation in human activity recognition systems, offering an effective solution for multitask learning with accelerometer data that balances accuracy and training efficiency. More broadly, it reduces the computational cost of model training, which is critical for scenarios requiring frequent model updates or training on resource-constrained platforms. The code and model are available at https://github.com/Kuan2vn/smooth\_distill.

Figures

Figures reproduced from arXiv: 2507.00061 by the authors.

Figure 1
Figure 1. Standard knowledge distillation setup, where a student model learns from both ground-truth labels and the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed smooth-distill framework, consists of three main components: (Top) Standard [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Confusion Matrices for Task 1 (Posture/Activity Recognition) of Five Methods on the Sleep Dataset. [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Test Accuracy of Multitask Smooth-Distill for Task 1 (Posture/Activity Recognition) and Task 2 (Device [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Comparative Test Accuracy of Model Architectures on the Sleep Dataset Across Varying Training Ratios. [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Comparative Test Accuracy of Model Architectures on the MHealth Dataset Across Varying Training Ratios. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Comparative Test Accuracy of Model Architectures on the WISDM Dataset Across Varying Training Ratios. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Training Curves of Four Methods for Posture/Activity Recognition (Task 1) and Device Location Identification [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Inertia-1: An Open Exploration of Wearable Motion Foundation Models

    cs.LG 2026-07 conditional novelty 6.5 of 10

    Controlled large-scale pretraining on 18.2M hours of wearables shows self-supervised motion models beat scratch training, with triaxial fidelity, data diversity, and task-matched windows mattering more than model size alone.

Reference graph

Works this paper leans on

38 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables

    Taoran Sheng and Manfred Huber. “Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables”. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4.2 (June 2020), pp. 1–18. ISSN : 2474-9567. DOI: 10.1145/3397330 . URL: http://dx.doi. org/10.1145/3397330

    doi:10.1145/3397330 2020

  2. [2]

    Extending Stress Detection Reproducibility to Consumer Wearable Sensors

    Ohida Binte Amin et al. Extending Stress Detection Reproducibility to Consumer Wearable Sensors. 2025. DOI: 10.48550/ARXIV.2505.05694 . arXiv: 2505.05694 [cs.HC] . URL: https://arxiv.org/abs/2505. 05694

    arXiv doi:10.48550/arxiv.2505.05694 2025

  3. [3]

    Exploring the Applications of Explainability in Wearable Data Analytics: Systematic Literature Review

    Y . Abdelaal et al. “Exploring the Applications of Explainability in Wearable Data Analytics: Systematic Literature Review”. In: Journal of Medical Internet Research 26 (Dec. 2024). DOI: 10.2196/53863

    doi:10.2196/53863 2024

  4. [4]

    A Survey on Human Activity Recognition Using Deep Learning Techniques and Wearable Sensor Data

    Nidhi Dua et al. “A Survey on Human Activity Recognition Using Deep Learning Techniques and Wearable Sensor Data”. In: Machine Learning, Image Processing, Network Security and Data Sciences. Springer Nature Switzerland, Jan. 2022, pp. 52–71. ISBN : 9783031243523. DOI: 10.1007/978-3-031-24352-3_5

    work page doi:10.1007/978-3-031-24352-3_5 2022

  5. [5]

    Context-Aware Complex Human Activity Recognition Using Hybrid Deep Learning Models

    Adebola Omolaja, Abayomi Otebolaku, and Ali Alfoudi. “Context-Aware Complex Human Activity Recognition Using Hybrid Deep Learning Models”. In: Applied Sciences 12.18 (Sept. 2022), p. 9305. ISSN : 2076-3417. DOI: 10.3390/app12189305

    work page doi:10.3390/app12189305 2022

  6. [6]

    Optimal Sensor Placement and Multimodal Fusion for Human Activity Recognition in Agricultural Tasks

    Lefteris Benos et al. “Optimal Sensor Placement and Multimodal Fusion for Human Activity Recognition in Agricultural Tasks”. In: Applied Sciences 14.18 (Sept. 2024), p. 8520. ISSN : 2076-3417. DOI: 10.3390/ app14188520. URL: https://www.mdpi.com/2076-3417/14/18/8520

    work page 2024

  7. [7]

    Physical Human Activity Recognition Using Wearable Sensors

    Ferhat Attal et al. “Physical Human Activity Recognition Using Wearable Sensors”. In:Sensors 15.12 (Dec. 2015), pp. 31314–31338. ISSN : 1424-8220. DOI: 10.3390/s151229858 . URL: https://www.mdpi.com/1424- 8220/15/12/29858

    work page doi:10.3390/s151229858 2015

  8. [8]

    A Novel Wearable Foot and Ankle Monitoring System for the Assessment of Gait Biome- chanics

    Paul Faragó et al. “A Novel Wearable Foot and Ankle Monitoring System for the Assessment of Gait Biome- chanics”. In: Applied Sciences 11.1 (Dec. 2020), p. 268. ISSN : 2076-3417. DOI: 10.3390/app11010268. URL: https://www.mdpi.com/2076-3417/11/1/268

    work page doi:10.3390/app11010268 2020

Show all 38 references
  1. [9]

    A Comprehensive Comparison of Simple Step Counting Techniques Using Wrist- and Ankle-mounted Accelerometer and Gyroscope Signals

    Matthew B. Rhudy and Joseph M. Mahoney. “A Comprehensive Comparison of Simple Step Counting Techniques Using Wrist- and Ankle-mounted Accelerometer and Gyroscope Signals”. In: Journal of Medical Engineering &; Technology 42.3 (Apr. 2018). Epub 2018 May 30., pp. 236–243. ISSN :...

    2018

  2. [10]

    Wearable Sensor-Based Real-Time Gait Detection: A Systematic Review

    Hari Prasanth et al. “Wearable Sensor-Based Real-Time Gait Detection: A Systematic Review”. In:Sensors 21.8 (Apr. 2021), p. 2727. ISSN : 1424-8220. DOI: 10.3390/s21082727 . URL: https://www.mdpi.com/1424- 8220/21/8/2727

    2021 doi

  3. [11]

    The Effect of Sensor Placement and Number on Physical Activity Recognition and Energy Expenditure Estimation in Older Adults: Validation Study

    Anis Davoudi et al. “The Effect of Sensor Placement and Number on Physical Activity Recognition and Energy Expenditure Estimation in Older Adults: Validation Study”. In: JMIR mHealth and uHealth 9.5, e23681 (May 2021), e23681. ISSN : 2291-5222. DOI: 10.2196/23681

    2021 doi

  4. [12]

    A Machine Learning Classifier for Detection of Physical Activity Types and Postures During Free-Living

    Kerstin Bach et al. “A Machine Learning Classifier for Detection of Physical Activity Types and Postures During Free-Living”. In: Journal for the Measurement of Physical Behaviour 5.1 (Mar. 2022), pp. 24–31. ISSN : 2575-6613. DOI: 10.1123/jmpb.2021- 0015 . URL: https://journal...

    2022 doi

  5. [13]

    Domain Adversarial Convolutional Neural Network Improves the Accuracy and Generalizability of Wearable Sleep Assessment Technology

    Adonay S. Nunes et al. “Domain Adversarial Convolutional Neural Network Improves the Accuracy and Generalizability of Wearable Sleep Assessment Technology”. In: Sensors 24.24 (Dec. 2024), p. 7982. ISSN : 1424-8220. DOI: 10.3390/s24247982. URL: https://www.mdpi.com/1424-8220/24/24/7982

    2024 doi

  6. [14]

    Monitoring Sleep and Nightly Recovery with Wrist-Worn Wearables: Links to Training Load and Performance Adaptations

    Olli-Pekka Nuuttila et al. “Monitoring Sleep and Nightly Recovery with Wrist-Worn Wearables: Links to Training Load and Performance Adaptations”. In: Sensors 25.2 (Jan. 2025), p. 533. ISSN : 1424-8220. DOI: 10.3390/s25020533. URL: https://www.mdpi.com/1424-8220/25/2/533

    2025 doi

  7. [15]

    Sleep Classification From Wrist-worn Accelerometer Data Using Random Forests

    Kalaivani Sundararajan et al. “Sleep Classification From Wrist-worn Accelerometer Data Using Random Forests”. In: Scientific Reports 11.1 (Jan. 2021), p. 24. ISSN : 2045-2322. DOI: 10.1038/s41598-020-79217-x

    2021 doi

  8. [16]

    Device Position-Independent Human Activity Recognition with Wearable Sensors Using Deep Neural Networks

    Sakorn Mekruksavanich and Anuchit Jitpattanakul. “Device Position-Independent Human Activity Recognition with Wearable Sensors Using Deep Neural Networks”. In: Applied Sciences 14.5 (Mar. 2024), p. 2107. ISSN : 2076-3417. DOI: 10.3390/app14052107. URL: https://www.mdpi.com/207...

    2024 doi

  9. [17]

    FedOpenHAR: Federated Multitask Transfer Learning for Sensor- Based Human Activity Recognition

    Egemen ˙I¸ sgÜder and Özlem Durmaz˙Incel. “FedOpenHAR: Federated Multitask Transfer Learning for Sensor- Based Human Activity Recognition”. In: Journal of Computational Biology (Apr. 2025). Advance Online Publication. ISSN : 1557-8666. DOI: 10.1089/cmb.2024.0631

    2025

  10. [18]

    Accurate Human Activity Recognition with Multi-task Learning

    Yinggang Li et al. “Accurate Human Activity Recognition with Multi-task Learning”. In: CCF Transactions on Pervasive Computing and Interaction 2.4 (Oct. 2020), pp. 288–298. ISSN : 2524-5228. DOI: 10.1007/s42486- 020-00042-2. URL: https://doi.org/10.1007/s42486-020-00042-2

    2020 doi

  11. [19]

    Multitask Learning

    Rich Caruana. “Multitask Learning”. In: Machine Learning 28.1 (1997), pp. 41–75. ISSN : 0885-6125. DOI: 10.1023/a:1007379606734. URL: https://doi.org/10.1023/A:1007379606734

    1997 doi

  12. [20]

    An Overview of Multi-Task Learning in Deep Neural Networks

    Sebastian Ruder. An Overview of Multi-Task Learning in Deep Neural Networks. 2017. DOI: 10.48550/ARXIV. 1706.05098. arXiv: 1706.05098 [cs.LG]. URL: https://arxiv.org/abs/1706.05098

    2017 doi arXiv

  13. [21]

    Alex Kendall, Yarin Gal, and Roberto Cipolla.Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. 2017. DOI: 10.48550/ARXIV.1705.07115 . arXiv: 1705.07115 [cs.CV] . URL: https://arxiv.org/abs/1705.07115

    2017 doi arXiv

  14. [22]

    Multi-Task Learning as Multi-Objective Optimization

    Ozan Sener and Vladlen Koltun. Multi-Task Learning as Multi-Objective Optimization. 2018. DOI: 10.48550/ ARXIV.1810.04650. arXiv: 1810.04650 [cs.LG]. URL: https://arxiv.org/abs/1810.04650

    2018 doi arXiv

  15. [23]

    ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning

    Junguang Jiang et al. ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning. Jan. 2023. DOI: 10.48550/ARXIV.2301.12618

    2023 doi arXiv

  16. [24]

    Knowledge Distillation for Multi-task Learning

    Wei-Hong Li and Hakan Bilen. Knowledge Distillation for Multi-task Learning. 2020. DOI: 10.48550/ARXIV. 2007.06889. arXiv: 2007.06889 [cs.CV]. URL: https://arxiv.org/abs/2007.06889

    2020 doi arXiv

  17. [25]

    SD-MTCNN: Self-Distilled Multi-Task CNN

    Ankit Jha et al. “SD-MTCNN: Self-Distilled Multi-Task CNN”. In: British Machine Vision Conference. 2020. URL: https://api.semanticscholar.org/CorpusID:221668230

    2020

  18. [26]

    Distilling the Knowledge in a Neural Network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the Knowledge in a Neural Network . 2015. DOI: https : / / doi . org / 10 . 48550 / arXiv . 1503 . 02531. arXiv: 1503 . 02531 [stat.ML]. URL: https : //arxiv.org/abs/1503.02531

    2015 doi arXiv

  19. [27]

    Knowledge Distillation: A Survey

    Jianping Gou et al. “Knowledge Distillation: A Survey”. In: International Journal of Computer Vision 129.6 (Mar. 2021), pp. 1789–1819. ISSN : 1573-1405. DOI: 10.1007/s11263-021-01453-z . URL: http://dx.doi. org/10.1007/s11263-021-01453-z

    2021 doi

  20. [28]

    Revisiting Self-Distillation

    Minh Pham et al. Revisiting Self-Distillation. 2022. DOI: https://doi.org/10.48550/arXiv.2206.08491. arXiv: 2206.08491 [cs.LG]. URL: https://arxiv.org/abs/2206.08491

    2022 doi arXiv

  21. [29]

    Self-Supervised Time Series Representation Learning with Temporal-Instance Similarity Distillation

    Ainaz Hajimoradlou et al. “Self-Supervised Time Series Representation Learning with Temporal-Instance Similarity Distillation”. In: First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML

  22. [30]

    Self-Distilled Representation Learning for Time Series

    Felix Pieper et al. Self-Distilled Representation Learning for Time Series. 2023. DOI: https://doi.org/10. 48550/arXiv.2311.11335. arXiv: 2311.11335 [cs.LG]. URL: https://arxiv.org/abs/2311.11335

    2023 doi arXiv

  23. [31]

    Born Again Neural Networks

    Tommaso Furlanello et al. Born Again Neural Networks. 2018. DOI: 10.48550/ARXIV.1805.04770 . arXiv: 1805.04770 [stat.ML]. URL: https://arxiv.org/abs/1805.04770

    2018 doi arXiv

  24. [32]

    ADXL345 3-Axis, ±16 g, Digital Accelerometer

    Analog Devices. ADXL345 3-Axis, ±16 g, Digital Accelerometer. Product page and technical specifications. URL: https://www.analog.com/en/products/adxl345.html (visited on 06/24/2025)

    2025

  25. [33]

    Oresti Banos, Rafael Garcia, and Alejandro Saez. MHEALTH. UCI Machine Learning Repository. 2014. DOI: 10.24432/C5TW22. URL: https://doi.org/10.24432/C5TW22

    2014 doi

  26. [34]

    Shimmer Sensing Homepage

    Shimmer Sensing. Shimmer Sensing Homepage. Manufacturer of Shimmer wearable sensors. URL: https: //www.shimmersensing.com/ (visited on 06/25/2025)

    2025

  27. [35]

    WISDM Smartphone and Smartwatch Activity and Biometrics Dataset

    Gary Weiss. WISDM Smartphone and Smartwatch Activity and Biometrics Dataset . UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5HK59. 2019. 23 A PREPRINT - SEPTEMBER 6, 2025

    2019 doi

  28. [36]

    Self-Knowledge Distillation via Dropout

    Hyoje Lee et al. Self-Knowledge Distillation via Dropout. 2022. DOI: https://doi.org/10.48550/arXiv. 2208.05642. arXiv: 2208.05642 [cs.CV]. URL: https://arxiv.org/abs/2208.05642

    2022 doi arXiv

  29. [37]

    Human Sleep Position Classification Using A Lightweight Model And Acceleration Data

    Hoang-Dieu Vu et al. “Human Sleep Position Classification Using A Lightweight Model And Acceleration Data”. In: Sleep and Breathing 29.1 (Feb. 2025), p. 95. ISSN : 1522-1709. DOI: 10.1007/s11325-025-03247-w . URL: https://doi.org/10.1007/s11325-025-03247-w . 24

    2025 doi

  30. [2022]

    URL: https://openreview.net/forum?id=nhtkdCvVLIh

    2022. URL: https://openreview.net/forum?id=nhtkdCvVLIh

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.