Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals

Asif Salekin; Tauhidur Rahman; Yuliang Chen; Yunfei Luo

REVIEW 6 major objections 5 minor 1 cited by

Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals

T0 review · 6 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A wearable-signal foundation model claims to beat task-specific baselines on 18 health tasks.

desk verdict Substantial wearable foundation model, but the evaluation protocol undermines the headline claim of consistent outperformance. read the letter →

arxiv 2412.09758 v2 pith:QDYSLJFO submitted 2024-12-12 cs.LG eess.SP

Yunfei Luo , Yuliang Chen , Asif Salekin , Tauhidur Rahman This is my paper · ORCID

classification cs.LGeess.SP

keywords wearablesensingfoundationmodelmultivariatephysiologicalsignalscontinuouswavelettransformchannel-awareattentionrepresentationalignmentzero-shottransfermaskedautoencoderhealthmonitoring

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

NormWear is a foundation model for multivariate wearable physiological signals. The paper tries to establish that one pretrained model, using a continuous-wavelet-transform tokenizer and a channel-aware transformer, can produce reusable representations for arbitrary combinations of sensors such as PPG, ECG, EEG, GSR, and IMU. The authors evaluate it on 11 unseen datasets covering 18 applications in mental health, physical state inference, vital sign estimation, and disease risk evaluation. They report that it consistently outperforms statistical, spectral, self-supervised time-series, and modality-specific baselines under zero-shot, partial-shot, and full-shot linear probing. If the claim holds, a single model and unified preprocessing pipeline could replace bespoke per-sensor systems in wearable health monitoring.

What carries the argument

The load-bearing machinery is a channel-aware masked autoencoder over continuous wavelet transform scalograms, followed by sensor-semantic alignment. A CWT with Mexican-hat wavelets at scales 1 to 64 turns each raw signal and its first two derivatives into three stacked scalograms, giving a modality-agnostic tokenization that preserves time and frequency information. A shared-weight transformer processes each channel independently, and a [CLS]-attention fusion layer exchanges information between channels with complexity $O(d \cdot C^2)$ instead of full cross-attention; because self-attention is permutation-invariant, the encoder accepts any number and order of channels. Structured masking along the scale and time axes, with an expected mask ratio of 0.8, forces reconstruction of the original signal through a lightweight decoder. For zero-shot inference, the MSiTF aggregation module weights patches by relevance to a text query, recency, and a Gumbel-sampled importance gate, then aligns the fused embedding to sentence embeddings from a frozen clinical language model through contrastive and distance losses.

What would settle it

Run a nested subject-stratified cross-validation on all 18 tasks, fixing all architecture choices on a separate validation set before touching the test folds, and compare macro-average AUC-ROC against TF-C, CLAP, and Chronos; if the margin collapses or reverses, the transfer claim is wrong. The paper's own tables already provide a concrete spot to check: WESAD full-shot linear probing is reported as 76.1 in Table 2 but 89.6 in the 5-fold subject-stratified cross-validation of Table 13, a gap far larger than the claimed margins over baselines.

Watch

Extended reading notes

Core claim

The central discovery claimed is generalizable sensor-agnostic representation learning for wearable physiological signals. NormWear is pretrained on around 2.5 million augmented segments, roughly 14,943 hours of PPG, ECG, EEG, GSR, PCG, and IMU data, with a masked-autoencoder objective. Each channel is tokenized into an RGB-like stack of continuous wavelet transform scalograms of the raw series and its first and second derivatives, processed independently by a shared-weight encoder, and fused through self-attention over per-channel [CLS] tokens. After pretraining, a memory-stream-inspired temporal fusion module aligns the signal embeddings with text embeddings so that unseen tasks can be answered by retrieving the closest text label. The paper reports peak performance across all task groups and a 3.9 to 6.1 percent macro-average improvement over the strongest baselines in full-shot linear probing, plus zero-shot and 10%-label transfer results.

Load-bearing premise

The claim rests on the benchmark protocol being a fair test: the model's masking, fusion, and tokenization choices were selected using the same downstream tasks, the held-out split is a single 80/20 subject-stratified split, and several datasets have fewer than 50 subjects, so a small evaluation instability could erase the reported margins.

Editorial extensions

If this is right

If the transfer claim is right, a new wearable health task can be tackled by feeding the sensor stream through the frozen encoder and, in the zero-shot setting, matching its embedding to text labels, with no per-task training or hand-tuned preprocessing.
Linear probing with only 10% of labels should already approach full-label performance on most of the 18 tasks, which matters for clinical datasets with scarce annotations.
A model can be deployed on different devices with different sensor counts or channel orders without retraining, because performance is reported to be stable under channel shuffling.
The fusion cost scaling as $O(d \cdot C^2)$ rather than $O(d \cdot (L \cdot C)^2)$ makes the approach feasible for edge devices as channel count grows.
Larger pretraining corpora should continue to improve downstream accuracy, based on the reported scaling trend from 37k to 2.5M segments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The uniform 65 Hz resampling and fixed wavelet scales limit the model to roughly the physiological frequency band; extending the same pipeline to audio or high-frequency biosignals would require adaptive scale selection, which the paper does not test.
Zero-shot accuracy likely depends on how labels are phrased; the paper varies templates during pretraining, but a reader could probe robustness by re-running retrieval with synonym prompts on each downstream dataset.
The channel-aware [CLS] fusion is not specific to physiology; the same architecture could be applied to multi-sensor industrial or environmental time series, but that extension is untested.
The paper's demographic analysis suggests signal embeddings and demographic features are complementary, so combining them is a natural step for clinical risk tasks; the occasional performance drops after concatenation mean confounding remains to be studied.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper. First, NormWear is a real system: CWT-based tokenization, channel-independent MAE pretraining, [CLS]-attention fusion, and text-aligned zero-shot retrieval combined for the first time for multivariate wearable signals, trained on 2.5M segments and evaluated on 18 health tasks. The engineering is substantial and the code is released. Second, the central claim—"consistently outperforms" all baselines under zero-shot, partial-shot, and full-shot—does not survive contact with the paper's own tables.

The combination is genuinely new relative to the cited time-series and biosignal foundation models. The complexity analyses in Appendix G are thoughtful, and the demographic analysis in Table 14 is a nice check that the representations are not just encoding age or sex.

The big problem is evaluation. Section 3.4 says the masking strategy was chosen by observing which one "yields the best performance for the downstream tasks," and Appendix C ablates fusion, masking, and input representation on the same 18 tasks used for final reporting. No separate validation set or nested selection is described. That means the reported margins are not unbiased estimates of a fixed model's generalization; they look like the result of selecting the best variant on the test tasks. Second, Table 13's 5-fold results diverge sharply from Table 2—WESAD goes from 76.1 to 89.6. The paper does not explain this. Third, the zero-shot evaluation is not fully unseen: the label-to-sentence templates used at inference overlap semantically with the pretraining text augmentation, so the alignment module has effectively seen these label concepts. Fourth, the "consistent outperformance" is contradicted at task level: NormWear loses to TF-C or modality-specific baselines on DriverFatigue, GAMEEMO, PPG-BP CVD, and some vital-sign tasks.

None of this means the system is useless. The architecture is plausible and the pretraining is large. But as written, the central claim is load-bearing and it fails. The honest conclusion is that this is a promising framework with an evaluation protocol that needs to be redone: hold out a set of tasks for architecture selection, report a single pre-registered split, and reconcile the k-fold numbers.

I would send it to peer review because there is enough substance here to warrant serious refereeing, but I would expect reviewers to demand a corrected evaluation before acceptance. For your own work, I would not cite the performance claims, though the architecture might be worth citing as related work.

Referee Report

6 major / 5 minor

Summary. The paper introduces NormWear, a pretrained foundation model for multivariate wearable physiological signals. The model uses CWT-based scalogram tokenization, a shared-weight per-channel Transformer encoder, a channel-aware [CLS]-token fusion mechanism, and a masked-autoencoder reconstruction objective. A separate text-alignment module (MSiTF) is trained with contrastive and distance losses to enable zero-shot inference. The authors evaluate on 11 public datasets spanning 18 tasks under zero-shot, partial-shot, and full-shot linear probing, and report average improvements over statistical, time-series SSL, spectral, and modality-specific baselines. The paper also includes ablations, complexity analyses, scaling experiments, and feature visualizations, with code and cleaned data released publicly.

Significance. If the performance claims were supported, NormWear would be a valuable contribution to wearable sensing: arbitrary multivariate channel configurations, modality-agnostic tokenization, and text-aligned zero-shot inference are practically important, and the breadth of the evaluation across 18 tasks is commendable. The release of code and data is a concrete reproducibility asset, and the complexity derivations in Appendices G and H are useful. However, the central claim of consistent and peak performance is contradicted by the paper's own tables, and the evaluation protocol has a model-selection flaw that undermines the generalization claim. The strengths of the system design do not compensate for the unsupported headline result.

major comments (6)

[Table 2 and Section 4.3] The claim that NormWear 'consistently achieves peak performance across all task groups' is contradicted by the reported full-shot results. NormWear is below at least one baseline on DriverFatigue (74.292 vs Chronos 76.722 and Modality-Specific 80.430), GAMEEMO (54.937 vs TF-C 56.275), PPG-BP (CV A) (70.625 vs Modality-Specific 73.125 and Statistical 71.250), and PPG-BP (CVD) (51.773 vs TF-C 58.674 and Chronos 58.31). The abstract's 'consistently outperforms' is therefore not supported by the manuscript's own table.
[Table 2 vs Table 13] The full-shot linear-probing results are not stable across evaluation protocols. For NormWear, WESAD is 76.060 in Table 2 but 89.585 in the 5-fold subject-stratified protocol of Table 13; GAMEEMO is 54.937 vs 67.863; DriverFatigue is 74.292 vs 68.971; and UCI-HAR is 98.954 vs 98.179. Both tables are described as linear probing on the same tasks, so the discrepancies indicate that different checkpoints, split definitions, or preprocessing are being compared, or that the metrics are highly split-sensitive. The headline margins in Figure 4 and Appendix D are therefore not a stable basis for the superiority claim.
[Section 3.4 and Appendix C] The model's design choices—masking strategy, fusion scheme, and input representation—are selected by downstream performance on the same 18 tasks used in the main evaluation, with no held-out tasks or separate validation set. Section 3.4 states that temporal-and-scale masking 'yields the best performance for the downstream tasks,' and Tables 8–10 report ablations on WESAD, UCI-HAR, DriverFatigue, and the other evaluation tasks. Consequently, the reported margins over baselines are post-selection numbers and cannot be read as evidence of a generalizable advantage of the fixed NormWear architecture.
[Section 3.5 and Appendices B.5/B.6] The zero-shot evaluation is not on unseen label semantics. The MSiTF alignment component is trained on natural-language templates derived from the same kinds of labels as the downstream zero-shot tasks (stress, emotion, activity), and the templates are augmented with GPT-3.5 variations. As a result, the zero-shot retrieval in Table 1 tests whether the pretrained alignment transfers known concepts to new sensor datasets, not whether the model generalizes to novel tasks. The Conclusion's limitation paragraph acknowledges the limited semantic corpus, but the abstract and Section 4.3 still draw the stronger conclusion of 'exceptional generalizability.'
[Appendix B.1 and Table 4] The evaluation uses a single 80/20 subject-stratified split on datasets with as few as 47, 26, 15, and 12 subjects, and the main tables report no variance or repeated-split statistics for this protocol. The large differences between Table 2 and Table 13 show that split choice materially changes the results. The paper needs to report repeated stratified splits or use the 5-fold protocol for the headline tables before any superiority claim can be evaluated.
[Appendix D] The permutation tests and Conover post-hoc comparisons are applied to the selected model after the architecture has been chosen on the same tasks, so the reported p<.01 values do not test the generalization claim. In addition, the claim that NormWear is statistically superior to all baselines is inconsistent with the point estimates in Table 2, which show losses on several tasks. The statistical analysis needs to be rerun under a validation protocol that separates model selection from evaluation.

minor comments (5)

[Throughout] There are numerous typos and inconsistent notations, including 'publicibly avaliable' (Section 1), 'herustic' (Section 3.1), 'afrer' (Section 3.4), and inconsistent capitalization of 'NormWear'/'NORMWEAR'/'NORM WEAR'.
[Section 3.3] The section heading 'Share-weighted Encoder' should read 'Shared-weight Encoder' to match the terminology used in the text.
[Appendix D] The text says 'Table 8 presents the statistical test results,' but the cited material is Figure 8; the cross-reference should be corrected.
[Table 5] The '#Samp (hours)' column mixes segment counts and hours in a confusing way; the paper should clarify whether numbers such as 230,962(385) mean 230,962 segments from 385 hours, and reconcile the 4,294-hour and 14,943-hour totals with the abstract's description.
[Appendix B.8 and Figure 4] Figure 4 applies a softmax transformation to raw scores, which visually amplifies differences; although this is acknowledged in Appendix B.8, the main-text discussion of '3.9% improvement' in Section 4.3 should state explicitly that the numbers refer to macro-averaged raw scores, not the transformed values in the radar-style plots.

Circularity Check

1 steps flagged · score 6.0 of 10

Final NormWear configuration is selected by maximizing performance on the same 18 downstream tasks later used to claim consistent outperformance; the reported margins are a selection outcome, not an independent prediction.

fitted input called prediction [Section 3.4; Appendix C (Tables 8–10); Figure 5]
"We observe that the temporal and scalar masking yields the best performance for the downstream tasks. ... Our empirical results show that [CLS]-attention fusion achieves the best downstreaming performance for our proposed NORM WEAR model. Details of all the ablation studies are reported in Appendix C."

The architecture decisions (masking strategy, fusion scheme, CWT versus raw input) are selected by comparing performance on the same downstream benchmarks used as evidence for the central claim. Appendix C evaluates the ablations on WESAD, UCI-HAR, DriverFatigue, Epilepsy, GAMEEMO, ECG-Abnormal, PPG-BP, PhysioNet EMG, Noninvasive-BP, PPG-Hgb, and Fetal-fPCG, and Figure 5 makes the greedy selection explicit ('+CWT (Best, Final NormWear)', 'Switch back to [CLS] Liaison Attn. (Best repeat)'). The final NormWear is therefore, by construction, the variant with the highest scores on those tasks among those tried; no separate validation set or held-out task group is described.

full rationale

No formal derivation in the paper reduces an equation to its own input, and there are no load-bearing self-citations or imported uniqueness theorems; the CWT tokenization, MAE-style reconstruction, channel-aware fusion, and CLIP-style alignment are standard components with independent support. The principal circularity is evaluative: the model configuration was chosen by maximizing performance on the same 18 downstream tasks that are later presented as proof that NormWear 'consistently achieves peak performance across all task groups.' This is a form of fitted-input-called-prediction: the input (architecture) was fitted to the output (benchmark scores), making the reported superiority partly a selection outcome. The zero-shot label-space overlap noted by a careful reader is a related weakness but does not itself make the result equivalent to its inputs by construction. If the architecture had been frozen before the benchmark comparisons, or if selection had been performed on a separate validation set or held-out tasks, the evidence would be substantially stronger.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several empirically chosen hyperparameters, none of which are derived from theory. The main domain assumptions concern the transferability of lab-collected pretraining data and the reliability of small test sets.

free parameters (5)

CWT scale range = 1 to 64
Chosen following prior literature to cover physiological frequency bands; affects tokenization and is not derived from first principles.
Masking ratios (temporal, scale) = 0.6, 0.5 (overall 0.8)
Selected via ablation on downstream tasks in Appendix C; these choices affect the learned representations and were tuned on the evaluation benchmarks.
MSiTF temperature tau = 0.5
Hyperparameter in the Gumbel-softmax importance score (Eq. 1), set by the authors; controls gate sharpness.
Fusion strategy = [CLS]-attention fusion
Selected as best among four fusion schemes by downstream task performance in Appendix C.
Downsampling rate and window length = 65 Hz, 6 seconds
Chosen as a uniform preprocessing setting to standardize heterogeneous sensor data; not derived from the data.

assumptions (5)

domain assumption Mexican hat CWT with scales 1-64 preserves enough information across PPG, ECG, EEG, GSR, IMU, and PCG for reconstruction and downstream transfer.
Invoked in Section 3.2; no quantitative criterion is given for sufficiency across modalities.
domain assumption Pretraining on 9 public datasets in lab/controlled settings transfers to unseen datasets and tasks in the wild.
Core transfer premise of the paper; evaluated empirically but with the benchmark-selection issues noted above.
domain assumption Text templates and GPT-3.5 paraphrases accurately capture the semantics of health labels for zero-shot alignment.
Used in Appendix B.5 and B.6; the quality of the text corpus is not independently validated.
domain assumption Linear probing with identical hyperparameters across models is a fair comparison.
Stated in Section 4.3; assumes that all models' representations are equally suited to linear probes and that pretraining differences do not bias the comparison.
domain assumption Subject-stratified 80/20 splits on datasets with 15-68 subjects yield reliable test estimates.
Appendix B.1 describes the split; for WESAD and the vital-sign datasets, test sets are extremely small, making estimates unstable.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals." pith.science (2026). https://pith.science/paper/QDYSLJFO

@misc{pith2026241209758,
  author       = {Pith},
  title        = {Pith review of: Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QDYSLJFO}},
  note         = {Machine review of arXiv:2412.09758}
}

read the original abstract

Time-series foundation models excel at tasks like forecasting across diverse data types by leveraging informative waveform representations. Wearable sensing data, however, pose unique challenges due to their variability in patterns and frequency bands, especially for healthcare-related outcomes. The main obstacle lies in crafting generalizable representations that adapt efficiently across heterogeneous sensing configurations and applications. To address this, we propose NormWear, the first multi-modal and ubiquitous foundation model designed to extract generalized and informative representations from wearable sensing data. Specifically, we design a channel-aware attention mechanism with a shared special liaison [CLS] token to detect signal patterns in both intra-sensor and inter-sensors. This helps the model to extract more meaningful information considering both time series themselves and the relationships between input sensors. This helps the model to be widely compatible with various sensors settings. NormWear is pretrained on a diverse set of physiological signals, including PPG, ECG, EEG, GSR, and IMU, from various public datasets. Our model shows exceptional generalizability across 11 public wearable sensing datasets, spanning 18 applications in mental health, body state inference, vital sign estimation, and disease risk evaluation. It consistently outperforms competitive baselines under zero-shot, partial-shot, and full-shot settings, indicating broad applicability in real-world health applications.

Figures

Figures reproduced from arXiv: 2412.09758 by the authors.

**Figure 1.** The role of our framework. Several icons from Freepik (n.d.); Zhang et al. (2024a).) 2023; Wimmer & Rekabsaz, 2023) attempt to address this issue by training a generic encoder that can handle type-agnostic series, they remain limited to processing only univariate series. Because of this constraint, these previous works fail to account for the heterogeneity of multivariate input data; specifically, they do not captur… view at source ↗

**Figure 2.** Overview of the pretrain pipeline. 3.3 Share-weighted Encoder Rather than concatenating tokens from all channels into a single long sequence and processing them with a full attention transformer, we treat each channel of the multivariate signal as an independent input stream. Although all channels share the same transformer backbone, the forward pass is executed separately for each one. This design allows the model … view at source ↗

**Figure 3.** Memory stream inspired temporal fusion mechanism for representation alignment. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (13 more)

**Figure 5.** Figure 5: Summary of adaptation performance and module-level performance contributions. Details of ablation study results are presented in Appendix C. 5 Conclusion and Discussion Conclusion. In this work, we mainly propose a foundation model for wearable physiological signals. N…

**Figure 6.** Figure 6: Distribution of sensor signals used for pretraining. Left: Distribution by sensor modality. Right: Distribution by type of physiological information [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: Visualization of original time series (left), CWT transformation image with structured masking (middle), and reconstructed time series (right). as suggested in the MAE framework, due to the high masking ratio. (An end-to-end example of the input and output of this pret…

**Figure 9.** Figure 9: Critical Difference Diagram. E Supplementary Metrics Normwear’s performance is summarized in [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]

**Figure 10.** Figure 10: Impact of scaling the pretraining dataset on downstream tasks. The y-axis represents the average accuracy across tasks, while the x-axis denotes the size of the pretraining dataset in terms of the number of samples. In addition to demonstrating that NormWear outperf…

**Figure 11.** Figure 11: Visualization of runtime complexity when scaling up the number of channels or the sequence length. G.1 All-Attention For the approach of conducting self-attention by concatenating all the patches, we arrive the Big-O complexity expression as follows: • We denote C as …

**Figure 12.** Figure 12: Nonlinear dynamic analysis on the waveforms extract at different layers of our model. [PITH_FULL_IMAGE:figures/full_fig_p031_12.png]

**Figure 13.** Figure 13: Feature visualization. I.4 T-SNE plot among classes In this section, we present T-SNE plots of NormWear’s embeddings across different classes to provide insights into their structure and assess their suitability for sample similarity-based information retrieval. It is…

**Figure 14.** Figure 14: Visualization of example signal embeddings. [PITH_FULL_IMAGE:figures/full_fig_p032_14.png]

**Figure 15.** Figure 15: Visualization of relevance scores(left) and importance scores (right) for a single channel EEG sample from the Epilepsy dataset under five task-specific questions. The background color follows a yellow-to-red scale, where darker regions indicate higher scores. To und…

**Figure 16.** Figure 16: Recency score generated by a decay function. The sample is selected from the Epilepsy dataset. 33 [PITH_FULL_IMAGE:figures/full_fig_p033_16.png]

**Figure 17.** Figure 17: Uncurated random samples on Phyatt scalogram, using a NORMWEAR trained in our training set. The masking ratio is 80%. 34 [PITH_FULL_IMAGE:figures/full_fig_p034_17.png]

**Figure 18.** Figure 18: Uncurated random samples on WESAD scalogram, using a NORMWEAR trained in our training set. The masking ratio is 80%. Note that the IMU data are not in the training set and, in general, NORMWEAR is able to reconstruct this with high accuracy. 35 [PITH_FULL_IMAGE:figur…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

DeepFeature: LLM-Empowered Context-aware Feature Generation for Wearable Biosignals
cs.AI 2025-12 conditional novelty 6.0 of 10

An LLM-based pipeline that generates context-aware features from raw wearable biosignals and iteratively refines them via performance feedback reports AUROC gains of 4-10% over baselines.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages · cited by 1 Pith paper

[1]

Calculate dot product: attn(xi,j, xk,l) =xT i,jxk,l =⇒ O(2d)

work page
[2]

Softmax over attention scores: exp(attn(xi,j ,xk,l)) N =⇒ O(1)

work page
[3]

Weighted average: xi,j + attn(xi,j, xk,l) · xk,l =⇒ O(2d) end for end for end for end for a complexity of: CX i=1 LX j=1 CX k=1 LX l=1 2d = CX i=1 LX j=1 CX k=1 L · 2d = CX i=1 LX j=1 C · L · 2d = O(d · (L · C)2) (4) where in the case of multi-head attention, the dot product still has the complexity of O(2d), and because the number of heads is a constant,...

work page 2021
[8]

N = exp(attn(xi,1)), =⇒ O(L) for k in [2, 3, ..., L] do

work page
[9]

1), 2), 3)

In this diagram, our proposed model, NormWear, is well separated from the others, indicating its statistical superiority over the competitive baselines. 26 Ours/Baselines Stats Chronos CLAP TFCNormWear- activityP < .01 P < .01 P < .01 P < .01NormWear- eeg P < .01 P < .01 P < .01 P < .01NormWear- risk P < .01 P < .01 P < .01 P < .01NormWear- vital P < .01 ...

work page
[10]

Calculate attn(xi,1, xj,k), =⇒ O(2d)

work page
[11]

Softmax over all-attention scores, exp(attn(xi,1,xj,k)) N , =⇒ O(1)

work page
[12]

Feature Associations

Weighted average: xi,1 + xj,k, =⇒ O(2d) end for end for end for with the same notion in the previous subsection. The total complexity is O(C 2 · L · 2d) +O(C 2 · L) +O(C 2 · L · 2d) =O(d · L · C 2) (6) G.3 [CLS]-Attention This is the approach that we adopted for the final version of our proposed foundation model. Only the embedding corresponding to the [C...

work page 2008

Show all 13 references

[13]

Deeper Layers Capture Higher-Order Complexity. • For signals such as GSR, EEG, and ACC, deeper layers show lower self-correlation (DFA (Hu et al., 2001)) and higher unpredictability (persistence entropy), indicating a transition to representations that are less periodic and mo...

2001
[14]

In contrast, PPG and ECG signals, dominated by regular heart activity, exhibit more stable patterns across layers

Modalities with Simpler Dynamics. In contrast, PPG and ECG signals, dominated by regular heart activity, exhibit more stable patterns across layers. This aligns with their simpler waveform structures and less complex dynamics compared to signals related to neural and physical ...
[2017]

Muzammil, M

doi: 10.6084/m9.figshare.5202739.v1. Muzammil, M. Finetuning endevsols/tinyllama-2.5t-clinical model on clinical dataset., 2021. URL https://huggingface.co/muzammil-eds/tinyllama-2.5T-Clinical-v2 . Narayanswamy, G., Liu, X., Ayush, K., Yang, Y ., Xu, X., Liao, S., Garrison, J....

2021 arXiv
[2023]

The subject is presently {activity_label}

ISSN 2398-6352. doi: 10.1038/s41746-023-00840-9. URL https://doi.org/10.1038/ s41746-023-00840-9 . Van der Maaten, L. and Hinton, G. Visualizing data using t-sne.Journal of machine learning research, 9(11), 2008. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., ...

2024 arXiv
[2024]

These results suggest that wearable signals do contain demographic information

have used learned representations to infer demographic labels. These results suggest that wearable signals do contain demographic information. In Table 14, we wanted to investigate that NormWear does not extract only demographic information (e.g. age, sex, height, etc. dependi...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Calculate dot product: attn(xi,j, xk,l) =xT i,jxk,l =⇒ O(2d)

work page

[2] [2]

Softmax over attention scores: exp(attn(xi,j ,xk,l)) N =⇒ O(1)

work page

[3] [3]

Weighted average: xi,j + attn(xi,j, xk,l) · xk,l =⇒ O(2d) end for end for end for end for a complexity of: CX i=1 LX j=1 CX k=1 LX l=1 2d = CX i=1 LX j=1 CX k=1 L · 2d = CX i=1 LX j=1 C · L · 2d = O(d · (L · C)2) (4) where in the case of multi-head attention, the dot product still has the complexity of O(2d), and because the number of heads is a constant,...

work page 2021

[4] [8]

N = exp(attn(xi,1)), =⇒ O(L) for k in [2, 3, ..., L] do

work page

[5] [9]

1), 2), 3)

In this diagram, our proposed model, NormWear, is well separated from the others, indicating its statistical superiority over the competitive baselines. 26 Ours/Baselines Stats Chronos CLAP TFCNormWear- activityP < .01 P < .01 P < .01 P < .01NormWear- eeg P < .01 P < .01 P < .01 P < .01NormWear- risk P < .01 P < .01 P < .01 P < .01NormWear- vital P < .01 ...

work page

[6] [10]

Calculate attn(xi,1, xj,k), =⇒ O(2d)

work page

[7] [11]

Softmax over all-attention scores, exp(attn(xi,1,xj,k)) N , =⇒ O(1)

work page

[8] [12]

Feature Associations

Weighted average: xi,1 + xj,k, =⇒ O(2d) end for end for end for with the same notion in the previous subsection. The total complexity is O(C 2 · L · 2d) +O(C 2 · L) +O(C 2 · L · 2d) =O(d · L · C 2) (6) G.3 [CLS]-Attention This is the approach that we adopted for the final version of our proposed foundation model. Only the embedding corresponding to the [C...

work page 2008