Pith. sign in

REVIEW 4 major objections 7 minor 33 references

Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A reservoir-based student network distilled from a high-capacity MLP-Mixer teacher reaches over 80% accuracy on human activity recognition at about one-sixth the FLOPs of a convolutional-LSTM baseline.

desk verdict A plausible new architecture with an abstract that overclaims: the 80%+ accuracy claim only holds for the fine-tuned SHL 2023 model, not the four other datasets. read the letter →

arxiv 2505.22985 v1 pith:DYPQ37TA submitted 2025-05-29 cs.LG cs.AIcs.PF

Masaharu Kagiyama , Tsuyoshi Okita This is my paper
classification cs.LGcs.AIcs.PF
keywords humanactivityrecognitionechostatenetworkreservoircomputingknowledgedistillationMLP-Mixerenergyefficiencytime-seriesclassificationedge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a low-cost reservoir-based classifier can be competitive for human activity recognition on one-dimensional sensor signals. It introduces PatchEchoClassifier, which tokenizes the signal into patches and passes them through a fixed Echo State Network reservoir, then trains two output heads by knowledge distillation from a high-capacity MLP-Mixer teacher. The central empirical claim is that, after distillation and fine-tuning on the SHL 2023 dataset, this student reaches over 80% accuracy while needing about one-sixth the floating-point operations of the DeepConvLSTM baseline. The aim is an energy-efficient classifier that can run in real time on edge devices.

What carries the argument

The load-bearing object is PatchEchoClassifier: a tokenizer divides multichannel 1D signals into patches of size $p$, appends a class token and a distillation token, and feeds the sequence into an Echo State Network whose reservoir follows the fixed update $X_i^{\mathrm{ESN}} = \tanh(X_{i-1}^{\mathrm{ESN}} W_{\mathrm{reservoir}} + X_i W_{\mathrm{input}}^{\mathsf{T}})$. Two linear heads read out the class token and distillation token; training minimizes $L = (1-\alpha)L_{\mathrm{CE}} + \alpha L_{\mathrm{Dist}}$ with label-smoothed cross-entropy and either KL or JS divergence against the teacher's logits, and inference averages the two heads. Because the reservoir weights are initialized and frozen, only the readout heads are learned, which is what makes the model cheap to train and run. The architecture is justified by the paper's reading of evidence that transformer-style performance comes from the overall tokenize-mix-readout structure rather than the specific mixer, so an ESN can stand in for the mixer.

What would settle it

A direct measurement of electricity drawn during inference on the SHL 2023 sensor windows would settle the efficiency claim; if a quarter-size DeepConvLSTM uses less power per correct classification than PatchEchoClassifier, the paper's edge-AI conclusion fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that cross-architecture distillation works: a vision-style MLP-Mixer teacher can transfer its knowledge to a reservoir student whose internal weights are never learned. The resulting PatchEchoClassifier scores between 82.7% and 88.0% on the SHL 2023 held-out frames depending on reservoir size, patch size, and distillation loss, with the best result 88.0% for a 4000-parameter reservoir, patch size 128, and JS divergence. The same student, distilled with KL divergence on four other HAR datasets, lands at 59.8 to 76.6% accuracy, which the paper attributes to the ESN being suited to structured activities. On compute, the paper reports PatchEchoClassifier with 1000 parameters and patch size 128 at 1.16 billion FLOPs versus 9.11 billion for DeepConvLSTM, and it claims that under a power-saving weighting of FLOPs, heap size, and footprint, the reservoir student can have the best accuracy-to-energy ratio.

Load-bearing premise

The energy conclusion assumes that the number of floating-point operations, the peak memory used, and the stored model size measured on one synthetic input are good stand-ins for how much electricity the model would actually use on a device, and that the chosen weighting of those three numbers matches the deployment scenario.

Editorial extensions

If this is right

  • On the SHL 2023 task, a practitioner could replace a 9.11 GFLOP conv-LSTM classifier with a 1.16 GFLOP reservoir student and keep accuracy in the 80s, which is the paper's concrete trade-off.
  • Because only the two readout heads are trained while the reservoir stays fixed, the method inherits the fast training of echo state networks, so distillation itself is computationally cheap.
  • The best accuracy-to-energy ratio depends on which resource dominates: when FLOPs are weighted heavily, PatchEchoClassifier with 1000 parameters and patch size 32 comes out ahead, while DeepConvLSTM with one-quarter width wins under balanced, memory-saving, and storage-optimized weightings.
  • The architecture transfers a token-based distillation recipe from image models to 1D sensor signals, opening a route for distilling other high-capacity teachers into reservoir students.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct implication the authors do not spell out is that the FLOPs comparison may understate the energy case for reservoir hardware: echo state networks can in principle run on analog or optical substrates where the frozen reservoir costs near-zero incremental energy, so the reported compute ratio could be conservative on specialized hardware.
  • Since the four secondary datasets show a 20 to 33 point accuracy drop, a natural testable extension is whether a learned reservoir initialization or per-dataset patch/tokenizer tuning would close the gap with the same distillation pipeline.
  • The Energy Efficiency Score weights are chosen by hand; measuring real power, memory, and storage budgets on target devices would let those weights be fit empirically, and the model rankings would likely shift accordingly.
  • The paper's own ensemble experiment with three parallel 4000-parameter modules gave only 58.9% accuracy, suggesting that naive ensembling does not help; a more promising extension, which the paper lists as future work, is quantizing the frozen reservoir to shrink footprint further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes PatchEchoClassifier, a reservoir-computing (ESN) student model for one-dimensional sensor-signal human activity recognition, trained with a DeiT-style knowledge-distillation framework in which an MLP-Mixer teacher transfers knowledge through class and distillation tokens. The authors report FLOPs, heap-size, and footprint measurements on a toy input, accuracy on four public HAR datasets, and accuracy on the SHL 2023 dataset after distillation followed by fine-tuning. They define an Energy Efficiency Score and an Accuracy-to-Energy Ratio to compare models under several weighting scenarios. The main advertised claims are that the model exceeds 80% accuracy on multiple HAR datasets and that it requires about one-sixth of the FLOPs of DeepConvLSTM.

Significance. If the SHL 2023 results are reproducible, the architectural combination of an ESN as the token mixer with a DeiT-style distillation mechanism is a credible low-FLOPs option for HAR on edge devices, and the idea of distilling from a 1D MLP-Mixer into a fixed-reservoir student is worth exploring. The paper is also careful in some respects: it reports separate training/validation/test splits, cites the sliding-window leakage concern, and presents its toy-data FLOPs/heap/footprint measurements in tabular form. However, the paper provides no code, no seeds, and no multiple-run statistics, and the headline accuracy claim is contradicted by the paper's own cross-dataset table. The energy-efficiency contribution is conditional on hand-chosen weights in the EES metric and on toy-data proxies rather than on measured power consumption.

major comments (4)
  1. [Abstract; Section 6, Table 4 and Table 5] The abstract states that 'experimental evaluations on multiple HAR datasets demonstrate that our model achieves over 80 percent accuracy,' but the paper's own numbers do not support this. In Table 4, PatchEchoClassifier achieves 70.1% on ADL, 66.8% on PAMAP2, 76.6% on RealWorld, and 59.8% on WISDM; none of these exceed 80%. The only above-80% results appear in Table 5 for the SHL 2023 dataset after distillation followed by fine-tuning, with the highest at 88.0% (PatchEchoClassifier4000, p=128, JS). The accuracy claim should be restricted to the SHL 2023 setting, or the paper should identify which datasets and configurations actually exceed 80%. This is an internal inconsistency between the headline contribution and the reported evidence, not merely a wording preference.
  2. [Section 7, Eq. (7) and Table 6] The energy-efficiency conclusion is not robust to the choice of EES weights. Table 6 shows that under Balanced, Memory-saving, and Storage-optimized settings, DeepConvLSTM0.25 achieves the highest AER, and PatchEchoClassifier is best only under the Power-saving weighting. The conclusion in Section 9 that PatchEchoClassifier 'achieves higher efficiency and superior performance compared to DeepConvLSTM' is therefore only valid for one specific weighting scenario. Moreover, the EES is computed from FLOPs, heap size, and footprint measured on a single toy input, not from any actual power or energy measurement. The claims about edge-AI energy efficiency need either deployment-level measurements or a clearly conditional statement with a sensitivity analysis over the weights.
  3. [Abstract; Section 6, Table 1] The claimed 'about one-sixth of the FLOPS' compared with DeepConvLSTM is numerically inconsistent with Table 1. DeepConvLSTM is reported as 9,105,719,296 FLOPs and PatchEchoClassifier1000 (p=128) as 1,161,797,632 FLOPs, giving a ratio of about 7.8, i.e., roughly one-eighth rather than one-sixth. If the intended comparison is against DeepConvLSTM0.50, the ratio is about 2.0. The computational advantage is still substantial, but the quantified headline claim should be corrected.
  4. [Section 5; Section 6, Tables 4 and 5; Section 7] No seeds, repeated runs, or error bars are reported for any of the accuracy results. Because the reservoir weights are randomly initialized and then fixed, and because the text itself mentions variability arising from 'the randomness of the ESN's internal states' near Table 5, single-run numbers are not sufficient to assess whether differences such as 86.0% (KL) vs. 82.6% (JS) for PatchEchoClassifier1000 p=128, or the AER rankings in Table 6, are meaningful. The paper should report multiple seeds with standard deviations or confidence intervals, at least for Table 5 and for the AER comparisons built on it.
minor comments (7)
  1. [Equations (1)-(3)] The notation in the loss equations is ambiguous: the same symbol N is used for the number of elements of Zs_dist and for the number of classes, and the label-smoothing term uses N where the standard formulation uses K, the number of classes. The KL and JS expressions also mix probabilities and logits (e.g., Zs(y=yi) is multiplied by log softmax(...)), which makes the formulas hard to verify.
  2. [Section 1] The introduction says 'the teacher model is a large language model (LLM) and the student model is a liquid state machine,' but the actual teacher in Section 3 is a 1D MLP-Mixer and the student is an Echo State Network. This is a misleading description of the proposed method and should be corrected.
  3. [Section 7, Figures 4 and 5] The text refers to an 'Energy Consumption Index (ECI)' that is not defined, while the figures use the 'normalized value of the Energy Efficiency Score (EES)' on the horizontal axis. Please define the quantity plotted and make the terminology consistent.
  4. [Section 7, Eq. (7)] The min-max normalization used for FLOPs, heap size, and footprint before computing EES is not described precisely, including whether normalization is performed per metric over the set of compared models. Without this detail the numerical values of EES and AER are not reproducible.
  5. [Section 6, Tables 1-3] The FLOPs, heap, and footprint measurements are reported for a single toy input with no mention of the software/hardware environment beyond 'on CPU' for heap usage. Please state the framework version, CPU model, and measurement methodology so the numbers can be reproduced.
  6. [Table 4] The table reports PatchMixerClassifier and PatchEchoClassifier accuracy after distillation on four datasets, but no teacher accuracy is given. Since the goal is to assess how much knowledge is transferred from the 1DMLP-Mixer teacher, reporting the teacher's accuracy on the same test splits would make the comparison more informative.
  7. [Throughout] The paper mixes 'FLOPs' and 'FLOPS'; please use one standard spelling. There are also several typographical errors, including 'Efficeincy' in the caption of Figure 5 and 'PatchMixerClassifier' in some places instead of 'PatchEchoClassifier'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported accuracies and computational-cost measurements come from independent experiments against external baselines, not from fitted parameters or self-citation.

full rationale

The paper's central claims rest on four independent evidence streams: (1) accuracy results in Tables 4 and 5 obtained by training PatchEchoClassifier, PatchMixerClassifier, and DeepConvLSTM on standard HAR datasets and evaluating on held-out test frames; (2) FLOPs computed from a fixed toy input in Table 1; (3) heap-size and footprint measurements in Tables 2 and 3; and (4) comparison with external baselines such as DeepConvLSTM and MLP-Mixer. The ESN reservoir weights are randomly initialized and frozen, and the learned output heads and distillation losses are not fitted to the test set, so no fitted parameter is renamed as a prediction. The author-defined EES and AER metrics in Equations (7) and (8) use hand-chosen weights, and Table 6 indeed shows that the ranking depends on the chosen weighting scheme; however, this is a metric-selection and generalization concern, not circularity, because the inputs to those equations are independently measured FLOPs, heap size, footprint, and accuracy rather than quantities defined in terms of the conclusions. The abstract's unqualified 'over 80 percent accuracy' on multiple HAR datasets also conflicts with the 59.8-76.6 percent accuracies reported in Table 4 for four datasets, but that is an internal-consistency or support problem, not a circular derivation. No load-bearing self-citation, imported uniqueness theorem, or ansatz-smuggled-in-via-citation chain was found.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on unverified modeling assumptions such as ESN stability, the MetaFormer analogy, and teacher-signal informativeness, as well as on proxy measurements of energy. The main free parameters are architecture and loss hyperparameters, several of which are not reported. No new physical entities are introduced.

free parameters (6)
  • ESN reservoir size = 1000 or 4000
    Chosen by hand; larger sizes improve accuracy at higher energy cost, as shown in Tables 1 through 5.
  • Patch size = 32, 64, or 128
    Chosen by hand; affects both accuracy and FLOPs, heap, and footprint measurements.
  • Distillation temperature T = not reported
    Used in the KL distillation loss in Eq. (2), but its value is never stated.
  • Loss weight alpha = not reported
    Balances LCE and LDist in Eq. (4), yet no value is given in the experimental setup.
  • Label smoothing epsilon = not reported
    Appears in Eq. (1) but is not quantified anywhere in the paper.
  • EES weights (alpha, beta, gamma) = four scenarios, e.g. (1/3, 1/3, 1/3)
    Hand-chosen scenario weights drive the conclusions in Table 6; PatchEchoClassifier wins only under the Power-saving weighting.
assumptions (5)
  • domain assumption A fixed random ESN with tanh nonlinearity exhibits the echo state property for the input scales and sequence lengths used.
    Invoked in Eq. (5) as the core computation; no spectral radius or input scaling is reported, so stability is assumed rather than verified.
  • ad hoc to paper MetaFormer's conclusion that the token mixer can be replaced by pooling extends to an ESN as the token mixer.
    Section 3.2 cites MetaFormer to justify replacing the token mixer with an ESN; this is a modeling leap specific to this paper.
  • domain assumption The soft labels from the MLP-Mixer teacher provide a training signal that transfers to the reservoir student.
    The distillation method in Section 3.1 assumes the teacher distribution is informative for the student; standard in distillation but not verified per dataset.
  • domain assumption FLOPs, heap, and footprint measured on random toy data are representative proxies for energy consumption in real deployment.
    Section 6 measures these on toy data, and Section 7 builds the EES on them without actual power measurements.
  • domain assumption The SHL 2023 train/validation split avoids sliding-window overlap between train and test.
    Section 5 claims no overlap citing prior work, but the exact windowing and label assignment procedure is under-specified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition." pith.science (2026). https://pith.science/paper/DYPQ37TA

@misc{pith2026250522985,
  author       = {Pith},
  title        = {Pith review of: Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DYPQ37TA}},
  note         = {Machine review of arXiv:2505.22985}
}
read the original abstract

This paper aims to develop an energy-efficient classifier for time-series data by introducing PatchEchoClassifier, a novel model that leverages a reservoir-based mechanism known as the Echo State Network (ESN). The model is designed for human activity recognition (HAR) using one-dimensional sensor signals and incorporates a tokenizer to extract patch-level representations. To train the model efficiently, we propose a knowledge distillation framework that transfers knowledge from a high-capacity MLP-Mixer teacher to the lightweight reservoir-based student model. Experimental evaluations on multiple HAR datasets demonstrate that our model achieves over 80 percent accuracy while significantly reducing computational cost. Notably, PatchEchoClassifier requires only about one-sixth of the floating point operations (FLOPS) compared to DeepConvLSTM, a widely used convolutional baseline. These results suggest that PatchEchoClassifier is a promising solution for real-time and energy-efficient human activity recognition in edge computing environments.

Figures

Figures reproduced from arXiv: 2505.22985 by the authors.

Figure 1
Figure 1. An overview of the proposed distillation method, Mixer-Ech [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. The architecture of PatchEchoClassifier. As shown in Figu [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The architecture of PatchMixerClassifier. In the patch e [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: For each model, a line is plotted with accuracy on the vertica [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: For each model, a line is plotted with accuracy on the vertica [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 28 canonical work pages

  1. [1]

    Improving deep learning for har with shallow lstms

    Marius Bock, Alexander H¨ olzemann, Michael Moeller, an d Kristof Van Laerhoven. Improving deep learning for har with shallow lstms. In 2021 International Symposium on Wearable Computers , UbiComp ’21. ACM, September 2021

    work page 2021

  2. [2]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah , Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shy am, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herb ert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh , Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher H esse, Mark Chen, Eric Sigler, Mateusz Li...

    arXiv 2005

  3. [3]

    Dataset for ADL Recognition with Wrist-worn Ac- celerometer

    Barbara Bruno, Fulvio Mastrogiovanni, and Antonio Sgor - bissa. Dataset for ADL Recognition with Wrist-worn Ac- celerometer. UCI Machine Learning Repository, 2012. DOI: https://doi.org/10.24432/C5PC99

    work page doi:10.24432/c5pc99 2012

  4. [4]

    Emerging dynamic memristors for neuromorp hic reservoir computing

    Jie Cao, Xumeng Zhang, Hongfei Cheng, Jie Qiu, Xusheng Li u, Ming Wang, and Qi Liu. Emerging dynamic memristors for neuromorp hic reservoir computing. Nanoscale(14)2, pages 289–298, 2022

    work page 2022

  5. [5]

    Leveraging large language models for activity recognition in smart environments

    Ian Cleland, Luke Nugent, Federico Cruciani, and Chris N ugent. Leveraging large language models for activity recognition in smart environments. In 2024 International Conference on Activity and Be- havior Computing (ABC) , pages 1–8, 2024

    work page 2024

  6. [6]

    An image is worth 16x16 words: Transformer s for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov , Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehgha ni, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Usz koreit, and Neil Houlsby. An image is worth 16x16 words: Transformer s for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2021

    arXiv 2010

  7. [7]

    Gjoreski, M

    H. Gjoreski, M. Ciliberto, L. Wang, F.J.O. Morales, S. Me kki, S. Valentin, and D. Roggen. The university of sussex-huawei lo- comotion and transportation dataset for multimodal analyt ics with mobile devices, 2018

    work page 2018

  8. [8]

    Echo state network based on improved knowledge distillatio n for edge intelligence

    ZHOU Jian, JIANG Yuwen, XU Lijie, ZHAO Lu, and XIAO Fu. Echo state network based on improved knowledge distillatio n for edge intelligence. Chinese Journal of Electronics , 33(1):101–111, 2024

    work page 2024

Show all 33 references
  1. [9]

    Energy-efficient reservoir com puting based on solution-processed electrolyte/ferroelectric m emcapacitive synapses for biosignal classification

    Sai Jiang, Jinrui Sun, Mengjiao Pei, Lichao Peng, Qinyon g Dai, Chaoran Wu, Jiahao Gu, Yanqin Yang, Jian Su, Ding Gu, Han Zhang, Huafei Guo, and Yun Li. Energy-efficient reservoir com puting based on solution-processed electrolyte/ferroelectric m emcapacitive synapses for biosig...

    2024

  2. [10]

    Neuromorphic on-chip reservoir comp uting with spiking neural network architectures, 2024

    Samip Karki, Diego Chavez Arana, Andrew Sornborger, an d Francesco Caravelli. Neuromorphic on-chip reservoir comp uting with spiking neural network architectures, 2024. Knowledge Distillation for Reservoir-based Classifier: Hu man Activity Recognition IJABC: International Journ...

    2024

  3. [11]

    Kwapisz, Gary M

    Jennifer R. Kwapisz, Gary M. Weiss, and Samuel Moore. Ac tivity recognition using cell phone accelerometers. SIGKDD Explor., 12:74– 82, 2011

    2011

  4. [12]

    Quantized reservoi r computing for spectrum sensing with knowledge distillation

    Shiya Liu, Lingjia Liu, and Yang Yi. Quantized reservoi r computing for spectrum sensing with knowledge distillation. IEEE Transactions on Cognitive and Developmental Systems , 15(1):88–99, 2023

    2023

  5. [13]

    Photonic reservoir computing for energy e fficient and versatile machine learning application

    Kathy Ludge. Photonic reservoir computing for energy e fficient and versatile machine learning application. Proceedings of the Royal So- ciety of Victoria , pages 1–3, 2024

    2024

  6. [14]

    On the computational power of recurrent circuits of spiking neurons

    Wolfgang Maass and Henry Markram. On the computational power of recurrent circuits of spiking neurons. Journal of Computer and System Sciences, 69 (4): 593–616 , 2004

    2004

  7. [15]

    Neuromorphic comput ing gets ready for the (really) big time

    Wolfgang Maass and Henry Markram. Neuromorphic comput ing gets ready for the (really) big time. Communications of the ACM 57 (6) , pages 13—-15, 2014

    2014

  8. [16]

    Autosnn: Towards energy-efficient sp iking neural networks, 2022

    Byunggook Na, Jisoo Mok, Seongsik Park, Dongjin Lee, Hy eokjun Choe, and Sungroh Yoon. Autosnn: Towards energy-efficient sp iking neural networks, 2022

    2022

  9. [17]

    Relation al knowledge distillation, 2019

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relation al knowledge distillation, 2019

    2019

  10. [18]

    High spe ed human action recognition using a photonic reservoir computer

    Enrico Picco, Piotr Antonik, and Serge Massar. High spe ed human action recognition using a photonic reservoir computer. Neural Net- works, 165:662–675, August 2023

    2023

  11. [19]

    Hilbert distillation for cross-dimensionality ne tworks, 2022

    Dian Qin, Haishuai Wang, Zhe Liu, Hongjia Xu, Sheng Zhou , and Ji- ajun Bu. Hilbert distillation for cross-dimensionality ne tworks, 2022

    2022

  12. [20]

    Introducing a new ben chmarked dataset for activity monitoring

    Attila Reiss and Didier Stricker. Introducing a new ben chmarked dataset for activity monitoring. 2012 16th International Symposium on Wearable Computers , pages 108–109, 2012

    2012

  13. [21]

    Distilbert, a distilled version of bert: smaller, faster, c heaper and lighter

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thom as Wolf. Distilbert, a distilled version of bert: smaller, faster, c heaper and lighter. arXiv preprint arXiv:1910.01108 , 2020

    1910 arXiv

  14. [22]

    Leveraging the larg e language model for activity recognition: A comprehensive review

    Milyun Ni’ma Shoumi and Sozo Inoue. Leveraging the larg e language model for activity recognition: A comprehensive review. Interna- tional Journal of Activity and Behavior Computing , 2024(2):1–27, 2024

    2024

  15. [23]

    Logit standardization in knowledge distillation, 202 4

    Shangquan Sun, Wenqi Ren, Jingzhi Li, Rui Wang, and Xiao chun Cao. Logit standardization in knowledge distillation, 202 4

  16. [24]

    Jaeger, herbert and haas, h erald

    Harnessing Nonlinearity: Predicting Chaotic Systems and Saving En- ergy in Wireless Communication. Jaeger, herbert and haas, h erald. Science. 304 (5667) , pages 78—-80, 2004

    2004

  17. [25]

    On-body local ization of wearable devices: An investigation of position-aware acti vity recog- nition

    Timo Sztyler and Heiner Stuckenschmidt. On-body local ization of wearable devices: An investigation of position-aware acti vity recog- nition. In 2016 IEEE International Conference on Pervasive Com- puting and Communications (PerCom) , pages 1–9, 2016. Knowledge Distillation f...

    2016

  18. [26]

    Too good to be true: accuracy overestimation in (re)current practic es for hu- man activity recognition

    Andr´ es Tello, Victoria Degeler, and Alexander Lazovi k. Too good to be true: accuracy overestimation in (re)current practic es for hu- man activity recognition. In 2024 IEEE International Conference on Pervasive Computing and Communications Workshops and ot her Affiliated Even...

    2024

  19. [27]

    Mlp- mixer: An all-mlp architecture for vision

    Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Key sers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mlp- mixer: An all-mlp architecture for vision. arXiv preprint arXiv:2105.01601 , 2021

    2021 arXiv

  20. [28]

    Training data-effi cient im- age transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisc o Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-effi cient im- age transformers & distillation through attention. In International Conference on Machine Learning , volume 139, pages 10347–10357, July 2021

    2021

  21. [29]

    Crosskd: Cross-head knowledge distil lation for object detection

    Jiabao Wang, Yuming Chen, Zhaohui Zheng, Xiang Li, Ming -Ming Cheng, and Qibin Hou. Crosskd: Cross-head knowledge distil lation for object detection. arXiv preprint arXiv:2306.11369 , 2024

    2024 arXiv

  22. [30]

    Yaremkevich, Alexey V

    Dmytro D. Yaremkevich, Alexey V. Scherbakov, Luke De Cl erk, Ser- hii M. Kukhtaruk, Achim Nadzeyka, Richard Campion, Andrew W . Rushforth, Sergey Savel’ev, Alexander G. Balanov, and Manf red Bayer. On-chip phonon-magnon reservoir for neuromorphic c omput- ing. Nature Communica...

    2023

  23. [31]

    Energy efficient and robust reservoir computing system using ultrathin (3.5 nm) ferroelectric tunneling junction s for temporal data learning

    Jie Yu, Yi Li, Wenxuan Sun, Woyu Zhang, Zhaomeng Gao, and Danian Dong. Energy efficient and robust reservoir computing system using ultrathin (3.5 nm) ferroelectric tunneling junction s for temporal data learning. IEEE 2021 Symposium on VLSI Technology , pages 1– 2, 2021

    2021

  24. [32]

    Metaformer is actuall y what you need for vision, 2022

    Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xi nchao Wang, Jiashi Feng, and Shuicheng Yan. Metaformer is actuall y what you need for vision, 2022

    2022

  25. [33]

    Decoupled knowledge distillation, 2022

    Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun L iang. Decoupled knowledge distillation, 2022. Knowledge Distillation for Reservoir-based Classifier: Hu man Activity Recognition

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.