Pith. sign in

REVIEW 4 major objections 6 minor 36 references

Self-supervised Learning Method Using Transformer for Multi-dimensional Sensor Data Processing

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that a linear-embedding Transformer with binning and parallel output heads lifts sensor-based activity recognition accuracy by 10–15% over a vanilla Transformer.

desk verdict A workmanlike empirical study adapting DistilBERT and GPT-2 to sensor data, but a missing initialization disclosure and a weak vanilla baseline leave the headline 10–15% gain not fully attributable. read the letter →

arxiv 2505.21918 v1 pith:GNRSKLMD submitted 2025-05-28 cs.LG cs.AI

Haruki Kai , Tsuyoshi Okita This is my paper
classification cs.LGcs.AI
keywords humanactivityrecognitionTransformerself-supervisedlearningsensordatamaskedlanguagemodelinglinearembeddingbinningDistilBERT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that language-model Transformers can be adapted to multi-dimensional sensor data without a custom attention architecture. The recipe is to feed each n-dimensional sensor reading through a linear embedding layer, turn continuous values into k-bin labels for self-supervised pretraining, and give each sensor axis its own linear output head. Pretrained on unlabeled accelerometer data and fine-tuned for activity recognition, the model reports 10–15% higher accuracy than a vanilla Transformer across five datasets and beats ResNet on three of them. The authors' wider point is that the input/output interface, not the attention mechanism, is what makes NLP Transformers work for numerical sensor signals.

What carries the argument

The central object is the n-dimensional numerical processing Transformer, an NLP Transformer with three swapped parts. The embedding is a linear layer $h = Wx + b$ that maps each n-axis reading $x\in\mathbb{R}^n$ to a hidden vector, replacing the token embedding; the pretraining target is produced by binning each dimension into k labels with $y_i = \min(\lfloor k \cdot x^{\mathrm{scaled}}_i \rfloor, k-1)$; and the output is a set of n parallel linear heads $\hat{y}_i = f_i(H) = W_i H + b_i$, one per sensor axis. This machinery turns continuous multi-axis sensor streams into a token-like sequence for self-supervised pretraining, and the pretrained encoder is then reused for downstream activity classification with a new head.

What would settle it

Retrain the proposed model twice with identical data and hyperparameters, once from the official DistilBERT/GPT-2 checkpoints and once from random initialization, and compare downstream accuracy on all five datasets. If the random-init version reproduces the 10–15% edge over the vanilla Transformer, the architecture claim is confirmed; if it falls to the Table 2 'without Pre-Training' levels (roughly 0.82–0.90), the reported gains are transfer from language pretraining rather than from the proposed components.

Watch

Extended reading notes

Core claim

The central discovery, as the paper states it, is empirical: replacing a vanilla Transformer's token embedding with a linear layer, discretizing sensor values into bins to create self-supervised labels, and using parallel output heads for each sensor axis yields a 10–15% accuracy improvement over a vanilla Transformer on human activity recognition. On the ADL, Opportunity, and PAMAP2 datasets the proposed models also outperform ResNet18 and Random Forest. The DistilBERT variant pretrained with masked language modeling beats SENvT-u4 and SENvT-contrastive on all five datasets, and masking outperforms reconstruction and next-token pretraining even though reconstruction has the lowest pretraining loss. The paper also reports that bin size (100 vs 1000) had no clear effect and that ResNet remains stronger on REAL WORLD and WISDM.

Load-bearing premise

The paper never states whether the DistilBERT and GPT-2 models start with weights already trained on text or with random weights; if they carry language pretraining, the reported 10–15% gain may come from that prior training rather than from the proposed embedding, binning, and parallel output heads.

Editorial extensions

If this is right

  • The 10–15% accuracy gap over the vanilla Transformer indicates that integer tokenization of continuous sensor values loses information that a linear embedding preserves.
  • Masked-language-model pretraining transfers to sensor data better than reconstruction or next-token prediction, despite converging with the noisiest loss curve.
  • The proposed model surpasses ResNet on ADL, Opportunity, and PAMAP2, so pretrained transformers are competitive with convolutional baselines on some HAR benchmarks.
  • Beating SENvT-u4 and SENvT-contrastive on all five datasets suggests per-timestep linear projection can outperform one-dimensional patch embeddings for sensor inputs.
  • GPU inference meets the 30 fps real-time constraint (4.5–6.3 ms per sample), while CPU inference does not, so deployment choices depend on hardware.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference — because the linear embedding layer is the only place sensor dimensionality enters, the same recipe should extend to 6-axis and 9-axis IMU data, a step the paper motivates but does not run.
  • Inference — the paper's bin sizes (100 and 1000) and single masking ratio (25%) cover only a small slice of the design space; a sweep over bin count and mask ratio would map where the 10–15% gain is largest.
  • Inference — if the transformers were initialized from official language checkpoints, the 'without Pre-Training' rows in Table 2 (roughly 0.82–0.90 accuracy) suggest most of the gain is transfer from language pretraining rather than from the embedding, binning, and parallel-head design, so the paper should be read with that ambiguity in mind.
  • Inference — the same tokenization problem applies to multivariate time-series forecasting, where continuous values are often discretized; the parallel-head and binning design could be tested as an alternative to patch-based encoders in that setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes an adaptation of NLP Transformer models (DistilBERT and GPT-2) to multi-dimensional accelerometer sensor data for human activity recognition. The method has three components: a linear embedding layer that projects n-dimensional sensor readings into the model dimension, a binning preprocessing step that discretizes continuous values into classification labels for self-supervised pretraining, and parallel linear output heads, one per sensor dimension. Pretraining is performed on the capture24 dataset with three tasks (reconstruction, masked language modeling, and next-token prediction), followed by downstream activity classification on ADL, Opportunity, PAMAP2, REAL WORLD, and WISDM. The paper reports that the proposed models improve accuracy by 10%--15% over a vanilla Transformer, that the masked-language-model variant performs best, and that the proposed DistilBERT-based model outperforms SENvT variants on all five datasets while ResNet remains stronger on two of the five datasets.

Significance. If the attribution question is resolved, this is a useful empirical recipe: it shows that off-the-shelf NLP Transformer architectures can be adapted to multi-dimensional sensor data with a simple linear embedding, binning, and parallel output heads, and it evaluates the recipe on five public datasets against several baselines. The paper also provides parameter counts and training/inference time and memory measurements, which are helpful for practitioners assessing deployability. The main value is the comparative evaluation; however, the central 10%--15% claim is only as strong as the experimental controls, and the manuscript currently leaves a key control (model initialization) undisclosed and reports single runs without error bars or ablations.

major comments (4)
  1. [4.1, Algorithm 2, Table 4] The manuscript never states whether the DistilBERT and GPT-2 models are initialized from their official NLP-pretrained checkpoints or from randomly initialized weights before the capture24 pretraining loop. Algorithm 2 only describes the sensor-data training loop, and Section 4.2 says 'the pretrained model's weights, excluding the classification head, are loaded,' without saying what weights were loaded before pretraining. Table 4 lists parameter counts consistent with official DistilBERT/GPT-2 sizes, but parameter count does not establish initialization. If the proposed models inherit language-pretrained transformer weights while the vanilla Transformer baseline or the from-scratch rows do not, the 10%--15% gap could be caused by language transfer rather than by the proposed embedding, binning, and parallel output heads. Please state the exact initialization used for every row of Table 2, including the 'without Pre-Training' rows, and, if language-pretrained weights are used, include a from-scratch control for the proposed architecture so the comparison is clean.
  2. [Table 2, Section 5.2.2] The headline claim rests on single runs: the paper reports no error bars, no standard deviations over multiple seeds, and no significance tests. Some test sets are small (e.g., ADL has 127 test samples), so differences of 1--2% between proposed variants may be within noise. Moreover, the paper evaluates many configurations (2 bin sizes, 3 pretraining tasks, 3 model sizes) on the same test sets and selects the best configuration (MLM, bin size 1000) for the conclusion; this selection procedure can inflate apparent gains relative to the non-selected configurations. Please report means and standard deviations over at least three seeds, apply a paired significance test or confidence intervals where appropriate, and state how configuration choices were made to avoid post-hoc selection on the test sets.
  3. [3.1--3.3, Table 2] The three claimed contributions are never ablated. The proposed model differs from the vanilla Transformer by (i) linear embedding, (ii) binning, and (iii) parallel output heads, but the paper does not run variants that remove each component, nor a variant with linear embedding only (the Soft Sensing Transformer idea already mentioned in Section 2.1). As a result, the title and abstract attribute the 10%--15% improvement to the proposed combination, yet the evidence only shows the difference between the full proposed system and a token-ID baseline. Please add ablations such as 'linear embedding only', 'linear embedding + binning', and 'full model' so that the contribution of each component is isolated.
  4. [5.1, Table 2] The central comparison is against a vanilla Transformer that is forced to convert continuous min-max scaled values into integer token IDs (vocabulary sizes 10000 and 30000) after concatenating axes into a length-900 token sequence. This is a weak representation for numerical sensor data, and it is unclear from Section 5.1 whether this baseline is also pretrained on capture24 or whether it is trained from scratch at the downstream stage. Since Soft Sensing Transformer [31] already proposed linear transformation of sensor inputs without tokenization, comparing against that prior adaptation (or a vanilla Transformer with a linear input layer) would help isolate the contribution of binning and parallel output heads. Please add such a baseline or explicitly state in the contributions paragraph that the 10%--15% figure is relative to the token-ID vanilla Transformer only.
minor comments (6)
  1. [5.2.3, 5.2.4, Figure 3, Figure 4] There are several typos: 'Oppotunity' in the Section 5.2.3 heading, and Figure 3 and Figure 4 captions both incorrectly say 'Confusion matrix for Oppotunity' even though Figure 3 is for REAL WORLD and Figure 4 shows pretraining loss curves. Please correct the captions and headings.
  2. [5.2.3, Section 6] Section 5.2.3 refers to 'the performance of the pre-trained ResNet model', but Section 6 states that ResNet18 was trained only on the downstream task without any pretraining. This inconsistency should be resolved by saying 'baseline ResNet' or clarifying how ResNet was pretrained.
  3. [5.1] The heading 'Assumptions' for the experimental setup is not descriptive; consider renaming it to 'Experimental Setting' or 'Common Experimental Conditions'.
  4. [Table 5] The note on memory usage is ambiguous: it says 'Memory Usage is the amount of memory used when measured by inference in the CPU', but it is unclear whether this is peak memory, which tool measured it, and whether the value includes only inference or also pretraining. Please clarify.
  5. [Section 2.2 and References] Some references have formatting issues, such as trailing commas in [13] and [28], and the reference list omits some publication details (e.g., volume and pages for [1]). Please check the reference formatting against the journal style.
  6. [Reproducibility] No code, random seed, or detailed hyperparameter settings for the baselines are provided. For reproducibility, please include code or at least specify the random seeds, optimizer settings for all baselines, and the exact protocol used for the 'without Pre-Training' rows.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the claimed improvements are reported test-set accuracies against external benchmarks, not consequences of the method's definitions.

full rationale

The paper's central claim, a 10%–15% accuracy improvement over a vanilla Transformer, is an empirical result reported in Table 2 from held-out activity-recognition test sets. No equation or derivation in the paper reduces the downstream accuracy to the method's own definitions. The binning process generates self-supervised labels from the input sensor data, but the pre-training loss is not the reported evaluation metric; downstream performance is measured against public activity labels on ADL, Opportunity, PAMAP2, REAL WORLD, and WISDM. The same-author citation [19] is used as a baseline model for comparison, not as the justification for the proposed method's effectiveness. The main caveats, such as undisclosed initialization and selecting bin size/pre-training task based on the same test sets, are experimental confounds or benchmark selection, not circular reasoning. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central result rests on several hand-chosen hyperparameters and domain assumptions. The bin size, sequence length, mask ratio, clipping percentiles, and the next-token loss range are chosen without sensitivity analysis. The method assumes that self-attention over 300 time steps captures activity-relevant structure, that binning into 100 or 1000 bins preserves the information needed for pre-training, that capture24 pre-training transfers to the downstream datasets, and that the central-point label is representative of each 300-step window. No new physical or mathematical entities are introduced.

free parameters (5)
  • Bin size k = 100 or 1000
    Number of discrete bins for self-supervised labels; chosen by hand, two values tested, no clear winner.
  • Sequence length = 300
    Window length for all datasets and pre-training; chosen without ablation.
  • MLM mask ratio = 25%
    Fraction of input positions masked in MLM pre-training; chosen without sensitivity analysis.
  • Clipping percentiles = 5% and 95%
    Top and bottom 5% of values are clipped to the 5th and 95th percentile before min-max scaling.
  • Next-token loss start index = 70
    First 70 of 300 time steps are excluded from next-token prediction loss.
assumptions (5)
  • domain assumption Transformer self-attention over sequences of 300 accelerometer readings can capture the temporal structure relevant for activity recognition.
    The method feeds each 300-step window as a sequence to DistilBERT/GPT-2; if attention cannot model this scale of time series, the central claim fails. Invoked throughout Sections 3 and 4.
  • domain assumption Discretizing continuous sensor values into k bins preserves the information required for useful self-supervised representation learning.
    The binning step converts regression into classification; if binning discards discriminative fine-grained signal, pre-training quality drops. Stated in Section 3.2.
  • domain assumption Self-supervised pre-training on capture24 transfers to ADL, Opportunity, PAMAP2, REAL WORLD, and WISDM, which have different sensors, placements, sampling rates, and activity sets.
    The whole method relies on transfer from the pre-training dataset to downstream datasets, yet no analysis of domain shift is provided. Sections 4.1 and 4.2.
  • domain assumption The activity label at the center of each 300-point window is representative of the entire window.
    Downstream labels are assigned from the central point only; windows that cross activity boundaries may be mislabeled. Section 4.2.
  • domain assumption Optimization with AdamW and a single pre-training epoch yields a model whose representations are useful for downstream fine-tuning.
    The paper states the loss 'converged sufficiently within one epoch' without showing convergence criteria. Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-supervised Learning Method Using Transformer for Multi-dimensional Sensor Data Processing." pith.science (2026). https://pith.science/paper/GNRSKLMD

@misc{pith2026250521918,
  author       = {Pith},
  title        = {Pith review of: Self-supervised Learning Method Using Transformer for Multi-dimensional Sensor Data Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GNRSKLMD}},
  note         = {Machine review of arXiv:2505.21918}
}
read the original abstract

We developed a deep learning algorithm for human activity recognition using sensor signals as input. In this study, we built a pretrained language model based on the Transformer architecture, which is widely used in natural language processing. By leveraging this pretrained model, we aimed to improve performance on the downstream task of human activity recognition. While this task can be addressed using a vanilla Transformer, we propose an enhanced n-dimensional numerical processing Transformer that incorporates three key features: embedding n-dimensional numerical data through a linear layer, binning-based pre-processing, and a linear transformation in the output layer. We evaluated the effectiveness of our proposed model across five different datasets. Compared to the vanilla Transformer, our model demonstrated 10%-15% improvements in accuracy.

Figures

Figures reproduced from arXiv: 2505.21918 by the authors.

Figure 1
Figure 1. Conventional and proposed architectures 3.1 Embedding with Linear Layer In traditional natural language models, an embedding layer is used to convert input data into embedding vectors. In contrast, our method re￾places the embedding layer with a linear layer to transform the input data x ∈ R n into an embedding vector h ∈ R d . The proposed embedding using a linear layer is defined as follows: h = Wx + b • x ∈ R n i… view at source ↗
Figure 2
Figure 2. Confusion matrix for Oppotunity On the Opportunity dataset, downstream models pre-trained using the Masked Language Modeling (MLM) task achieved the highest accuracy, with improvements of up to 2.68% in accuracy and 2.63% in F1-score compared to other methods. These results surpassed those of both the reconstruction task and the next-token prediction task. Moreover, the performance of all proposed methods exceeded t… view at source ↗
Figure 3
Figure 3. Confusion matrix for Oppotunity On the REALWORLD dataset, none of the proposed pre-training tasks outperformed ResNet. However, among the proposed methods, pre-training with the Masked Language Modeling (MLM) task achieved the highest performance, surpassing the other tasks. Both the Opportunity and REALWORLD datasets demonstrated the effectiveness of the MLM task compared to other tasks. However, while all proposed… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Confusion matrix for Oppotunity [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 27 canonical work pages

  1. [31]

    Soft sensing transformer: hun- dreds of sensors are worth a single word

    Chao Zhang, Jaswanth Yella, Yu Huang, Xiaoye Qian, Sergei Petrov, Andrey Rzhetsky, and Sthitie Bom. Soft sensing transformer: hun- dreds of sensors are worth a single word. pages 1999–2008, 2021

    work page 1999

  2. [1]

    A random forest guided tour.Test, 25:197–227, 2016

    G´ erard Biau and Erwan Scornet. A random forest guided tour.Test, 25:197–227, 2016

    work page 2016

  3. [2]

    Dataset for ADL Recognition with Wrist-worn Ac- celerometer

    Mastrogiovanni Fulvio Bruno, Barbara and Antonio Sgor- bissa. Dataset for ADL Recognition with Wrist-worn Ac- celerometer. UCI Machine Learning Repository, 2012. DOI: https://doi.org/10.24432/C5PC99

    doi:10.24432/c5pc99 2012

  4. [3]

    Chan Chang, R

    S. Chan Chang, R. Walmsley, J. Gershuny, T. Harms, E. Thomas, K. Milton, P. Kelly, C. Foster, A. Wong, N. Gray, S. Haque, S. Hol- lowell, and A. Doherty. Capture-24: Activity tracker dataset for human activity recognition, 2021

    work page 2021

  5. [4]

    Deep learning for sensor-based human activity recog- nition: Overview, challenges, and opportunities.ACM Computing Surveys (CSUR), 54(4):1–40, 2021

    Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. Deep learning for sensor-based human activity recog- nition: Overview, challenges, and opportunities.ACM Computing Surveys (CSUR), 54(4):1–40, 2021

    work page 2021

  6. [5]

    Learning to rotate: Quaternion transformer for complicated periodical time series forecasting.KDD, 2022

    Weiqi Chen, Wenwei Wang, Bingqing Peng, Qingsong Wen, Tian Zhou, and Liang Sun. Learning to rotate: Quaternion transformer for complicated periodical time series forecasting.KDD, 2022

    work page 2022

  7. [6]

    Leveraging large language models for activity recognition in smart environments

    Ian Cleland, Luke Nugent, Federico Cruciani, and Chris Nugent. Leveraging large language models for activity recognition in smart environments. In2024 International Conference on Activity and Be- havior Computing (ABC), pages 1–8, 2024

    work page 2024

  8. [7]

    Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

Show all 36 references
  1. [8]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778, 2016

    2016

  2. [9]

    Hargpt: Are llms zero-shot human activity recognizers?, 2024

    Sijie Ji, Xinzhe Zheng, and Chenshu Wu. Hargpt: Are llms zero-shot human activity recognizers?, 2024

    2024

  3. [10]

    A review of privacy-preserving human and human activ- ity recognition.International Journal on Smart Sensing and Intelli- gent Systems, 13(1):1–13, 2020

    Im Y Jung. A review of privacy-preserving human and human activ- ity recognition.International Journal on Smart Sensing and Intelli- gent Systems, 13(1):1–13, 2020

    2020

  4. [11]

    Activity recognition using cell phone accelerometers.ACM SigKDD Explo- rations Newsletter, 12(2):74–82, 2011

    Jennifer R Kwapisz, Gary M Weiss, and Samuel A Moore. Activity recognition using cell phone accelerometers.ACM SigKDD Explo- rations Newsletter, 12(2):74–82, 2011

    2011

  5. [12]

    A survey on human activity recognition using wearable sensors.IEEE communications surveys & tutorials, 15(3):1192–1209, 2012

    Oscar D Lara and Miguel A Labrador. A survey on human activity recognition using wearable sensors.IEEE communications surveys & tutorials, 15(3):1192–1209, 2012

    2012

  6. [13]

    Enhancing the locality and break- ing the memory bottleneck of transformer on time series forecasting

    Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu- Xiang Wang, , and Xifeng Yan. Enhancing the locality and break- ing the memory bottleneck of transformer on time series forecasting. NeurIPS, 2019

    2019

  7. [14]

    Liu, and Schahram Dustdar

    Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal Self-supervised Learning Method Using Transformer for Multi-dimensional Sensor Data Processing IJABC: International Journal of Activity and Behavior...

    2022

  8. [15]

    Non- stationary transformers: Exploring the stationarity in time series forecasting.NeurIPS, 2022

    Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non- stationary transformers: Exploring the stationarity in time series forecasting.NeurIPS, 2022

    2022

  9. [16]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    I Loshchilov. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  10. [17]

    Chapter 14 - a study on smartphone sensor-based human activity recognition using deep learning approaches

    Riktim Mondal, Dibyendu Mukhopadhyay, Sayanwita Barua, Pawan Kumar Singh, Ram Sarkar, and Debotosh Bhattacharjee. Chapter 14 - a study on smartphone sensor-based human activity recognition using deep learning approaches. In Janmenjoy Nayak, Bighnaraj Naik, Danilo Pelusi, and A...

    2021

  11. [18]

    Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam

    Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Longterm forecasting with transformers.ICLR, 2023

    2023

  12. [19]

    Towards llms for sensor data: Multi-task self-supervised learning

    Tsuyoshi Okita, Kosuke Ukita, Koki Matsuishi, Masaharu Kagiyama, Kodai Hirata, and Asahi Miyazaki. Towards llms for sensor data: Multi-task self-supervised learning. UbiComp/ISWC ’23 Adjunct, page 499–504, 2023

    2023

  13. [20]

    Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    2019

  14. [21]

    Introducing a new benchmarked dataset for activity monitoring

    Attila Reiss and Didier Stricker. Introducing a new benchmarked dataset for activity monitoring. In2012 16th International Sympo- sium on Wearable Computers, pages 108–109, 2012

    2012

  15. [22]

    Mill` an

    Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Hol- leczek, Kilian F¨ orster, Gerhard Tr¨ oster, Paul Lukowicz, David Ban- nach, Gerald Pirkl, Alois Ferscha, Jakob Doppler, Clemens Holz- mann, Marc Kurz, Gerald Holl, Ricardo Chavarriaga, Hesam Sagha, Hamidreza Bayati, M...

    2010

  16. [23]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019

    V Sanh. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  17. [24]

    On-body localization of wearable devices: An investigation of position-aware activity recog- nition

    Timo Sztyler and Heiner Stuckenschmidt. On-body localization of wearable devices: An investigation of position-aware activity recog- nition. In2016 IEEE International Conference on Pervasive Com- puting and Communications (PerCom), pages 1–9, 2016

    2016

  18. [25]

    Akihisa Tsukamoto, Kenji Mase, and Yu Enokibori. Integrated uti- lization of imu-based human activity recognition datasets across var- ied configurations using mig har dataset.International Journal of Activity and Behavior Computing, 2024(2):1–21, 2024. Self-supervised Learnin...

    2024

  19. [26]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  20. [27]

    Advancing human activity recognition us- ing ultra-wideband channel impulse response snapshots

    Yu Wang and Ana Basiri. Advancing human activity recognition us- ing ultra-wideband channel impulse response snapshots. In2024 In- ternational Conference on Activity and Behavior Computing (ABC), pages 1–10, 2024

    2024

  21. [28]

    Auto- former: Decomposition transformers with auto-correlation for long- term series forecasting.NeurIPS, 2021

    Haixu Wu, Jiehui Xu, Jianmin Wang, , and Mingsheng Long. Auto- former: Decomposition transformers with auto-correlation for long- term series forecasting.NeurIPS, 2021

    2021

  22. [29]

    Deep trans- former models for time series forecasting: The influenza prevalence case.arXiv preprint arXiv:2001.08317, 2020

    Neo Wu, Bradley Green, Xue Ben, and Shawn O’Banion. Deep trans- former models for time series forecasting: The influenza prevalence case.arXiv preprint arXiv:2001.08317, 2020

    2001 arXiv

  23. [30]

    Adversarial sparse transformer for time series fore- casting.NeurIPS, 2020

    Sifan Wu, Xi Xiao, Qianggang Ding, Peilin Zhao, Ying Wei, and Junzhou Huang. Adversarial sparse transformer for time series fore- casting.NeurIPS, 2020

    2020

  24. [32]

    Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting

    Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. ICLR, 2023

    2023

  25. [33]

    Informer: Beyond efficient trans- former for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient trans- former for long sequence time-series forecasting. InProceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106– 11115, 2021

    2021

  26. [34]

    Informer: Beyond efficient trans- former for long sequence timeseries forecasting.AAAI, 2021

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient trans- former for long sequence timeseries forecasting.AAAI, 2021

    2021

  27. [35]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting.ICML, 2022

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting.ICML, 2022

    2022

  28. [36]

    Sensor data augmentation from skeleton pose sequences for improving human activity recognition

    Parham Zolfaghari, Vitor Fortes Rey, Lala Ray, Hyun Kim, Sungho Suh, and Paul Lukowicz. Sensor data augmentation from skeleton pose sequences for improving human activity recognition. In2024 In- ternational Conference on Activity and Behavior Computing (ABC), pages 1–8, 2024. ...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.