{"id":"8ca93ffb-e4a4-4267-abcf-6f73c8d69914","arxiv_id":"2412.09758","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A masked-autoencoder foundation model for arbitrary multivariate wearable signals, using wavelet scalograms and channel-aware fusion, is shown to transfer to 18 health tasks with competitive average performance.","lead":"This paper introduces NormWear, a pretrained model that learns reusable patterns from wearable sensors like heart, brain, and motion monitors, then adapts to new health tasks with little or no extra data. The authors report strong average performance across 18 health tasks, but their own tables show several tasks where baseline models do better.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Architecture selection on the same 18 downstream tasks (Appendix C) invalidates the claim of consistent outperformance; no held-out tasks or separate validation set exist.","rationale":"I agree with the reader's assessment, and the most load-bearing part of it is the absence of a validation set outside the 18 downstream tasks. The paper presents itself as a foundation model, whose value is generalization to unseen tasks and configurations; yet the very tasks used to demonstrate generalization are the same tasks on which the masking strategy, fusion scheme, and input representation were selected (Section 3.4, Appendix C, Tables 8–10). This makes the reported 'consistent outperformance' (Section 4.3, Figure 4, Table 2) a selected-maximum statistic, not a predictive claim. The discrepancy between Table 2 and Table 13 (WESAD 76.1 vs 89.6 under nominally similar full-shot linear probing) illustrates that the evaluation protocol itself is unstable; if the model's advantage were robust, such a large swing should not occur. The paper's Appendix D statistical tests do not address this because they compare the fixed, already-selected model to baselines; they do not account for the multiple comparisons inherent in the ablation selection, nor for the fact that the selection criterion was the test set. I therefore concur with the reader's REJECT verdict. A constructive path would be to pre-register a single architecture or perform architecture selection on a disjoint set of tasks, and then evaluate on the remaining tasks.","tokens_in":32565,"tokens_out":6271,"duration_ms":63149,"concrete_test":"Run the full 18-task linear-probing protocol for every variant compared in Appendix C (raw input vs CWT, unstructured vs structured masking, no-fusion vs mean-pooling vs [CLS]-attention), all using the same pretraining data and checkpoint size. Then compute the macro-average margin over TF-C and CLAP for each variant. If the selected [CLS]-attention/CWT/structured-masking variant is the only one with a positive margin, the reported advantage is attributable to selection. Stronger test: randomly split the 18 tasks into two halves; use one half to select the best variant (as in Appendix C), then evaluate that variant on the held-out half and compare with baselines. If the held-out margin is no longer positive or is below the reported 3.9–6.1% range, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that NormWear 'consistently outperforms competitive baselines' under zero-shot, partial-shot, and full-shot settings—rests on an evaluation in which the model's own design choices were selected using the same downstream tasks that are later reported as evidence. Section 3.4 states that the masking strategy was chosen by observing which one 'yields the best performance for the downstream tasks,' and Appendix C (Tables 8–10) reports ablations over fusion schemes, masking strategies, and input representations, each evaluated on the same 18 downstream tasks used in Tables 1, 2, and 11. No separate validation set or nested selection procedure is described. Consequently, the reported margins over TF-C, CLAP, Chronos, and statistical baselines are not unbiased estimates of a fixed model's generalization; they are the result of selecting the best-performing variant on the test tasks. This is compounded by the internal inconsistency between Table 2 and Table 13 (e.g., WESAD 76.1 vs 89.6), which shows that the reported numbers are highly sensitive to the evaluation protocol. If the architecture had been frozen a priori, the 'consistent outperformance' claim would be much better supported; as written, the evidence is consistent with selection bias rather than a generalizable advantage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces NormWear, a pretrained foundation model for multivariate wearable physiological signals. The model uses CWT-based scalogram tokenization, a shared-weight per-channel Transformer encoder, a channel-aware [CLS]-token fusion mechanism, and a masked-autoencoder reconstruction objective. A separate text-alignment module (MSiTF) is trained with contrastive and distance losses to enable zero-shot inference. The authors evaluate on 11 public datasets spanning 18 tasks under zero-shot, partial-shot, and full-shot linear probing, and report average improvements over statistical, time-series SSL, spectral, and modality-specific baselines. The paper also includes ablations, complexity analyses, scaling experiments, and feature visualizations, with code and cleaned data released publicly.","tokens_in":32827,"tokens_out":6263,"duration_ms":62534,"significance":"If the performance claims were supported, NormWear would be a valuable contribution to wearable sensing: arbitrary multivariate channel configurations, modality-agnostic tokenization, and text-aligned zero-shot inference are practically important, and the breadth of the evaluation across 18 tasks is commendable. The release of code and data is a concrete reproducibility asset, and the complexity derivations in Appendices G and H are useful. However, the central claim of consistent and peak performance is contradicted by the paper's own tables, and the evaluation protocol has a model-selection flaw that undermines the generalization claim. The strengths of the system design do not compensate for the unsupported headline result.","major_comments":[{"comment":"The claim that NormWear 'consistently achieves peak performance across all task groups' is contradicted by the reported full-shot results. NormWear is below at least one baseline on DriverFatigue (74.292 vs Chronos 76.722 and Modality-Specific 80.430), GAMEEMO (54.937 vs TF-C 56.275), PPG-BP (CV A) (70.625 vs Modality-Specific 73.125 and Statistical 71.250), and PPG-BP (CVD) (51.773 vs TF-C 58.674 and Chronos 58.31). The abstract's 'consistently outperforms' is therefore not supported by the manuscript's own table.","section":"Table 2 and Section 4.3"},{"comment":"The full-shot linear-probing results are not stable across evaluation protocols. For NormWear, WESAD is 76.060 in Table 2 but 89.585 in the 5-fold subject-stratified protocol of Table 13; GAMEEMO is 54.937 vs 67.863; DriverFatigue is 74.292 vs 68.971; and UCI-HAR is 98.954 vs 98.179. Both tables are described as linear probing on the same tasks, so the discrepancies indicate that different checkpoints, split definitions, or preprocessing are being compared, or that the metrics are highly split-sensitive. The headline margins in Figure 4 and Appendix D are therefore not a stable basis for the superiority claim.","section":"Table 2 vs Table 13"},{"comment":"The model's design choices—masking strategy, fusion scheme, and input representation—are selected by downstream performance on the same 18 tasks used in the main evaluation, with no held-out tasks or separate validation set. Section 3.4 states that temporal-and-scale masking 'yields the best performance for the downstream tasks,' and Tables 8–10 report ablations on WESAD, UCI-HAR, DriverFatigue, and the other evaluation tasks. Consequently, the reported margins over baselines are post-selection numbers and cannot be read as evidence of a generalizable advantage of the fixed NormWear architecture.","section":"Section 3.4 and Appendix C"},{"comment":"The zero-shot evaluation is not on unseen label semantics. The MSiTF alignment component is trained on natural-language templates derived from the same kinds of labels as the downstream zero-shot tasks (stress, emotion, activity), and the templates are augmented with GPT-3.5 variations. As a result, the zero-shot retrieval in Table 1 tests whether the pretrained alignment transfers known concepts to new sensor datasets, not whether the model generalizes to novel tasks. The Conclusion's limitation paragraph acknowledges the limited semantic corpus, but the abstract and Section 4.3 still draw the stronger conclusion of 'exceptional generalizability.'","section":"Section 3.5 and Appendices B.5/B.6"},{"comment":"The evaluation uses a single 80/20 subject-stratified split on datasets with as few as 47, 26, 15, and 12 subjects, and the main tables report no variance or repeated-split statistics for this protocol. The large differences between Table 2 and Table 13 show that split choice materially changes the results. The paper needs to report repeated stratified splits or use the 5-fold protocol for the headline tables before any superiority claim can be evaluated.","section":"Appendix B.1 and Table 4"},{"comment":"The permutation tests and Conover post-hoc comparisons are applied to the selected model after the architecture has been chosen on the same tasks, so the reported p<.01 values do not test the generalization claim. In addition, the claim that NormWear is statistically superior to all baselines is inconsistent with the point estimates in Table 2, which show losses on several tasks. The statistical analysis needs to be rerun under a validation protocol that separates model selection from evaluation.","section":"Appendix D"}],"minor_comments":[{"comment":"There are numerous typos and inconsistent notations, including 'publicibly avaliable' (Section 1), 'herustic' (Section 3.1), 'afrer' (Section 3.4), and inconsistent capitalization of 'NormWear'/'NORMWEAR'/'NORM WEAR'.","section":"Throughout"},{"comment":"The section heading 'Share-weighted Encoder' should read 'Shared-weight Encoder' to match the terminology used in the text.","section":"Section 3.3"},{"comment":"The text says 'Table 8 presents the statistical test results,' but the cited material is Figure 8; the cross-reference should be corrected.","section":"Appendix D"},{"comment":"The '#Samp (hours)' column mixes segment counts and hours in a confusing way; the paper should clarify whether numbers such as 230,962(385) mean 230,962 segments from 385 hours, and reconcile the 4,294-hour and 14,943-hour totals with the abstract's description.","section":"Table 5"},{"comment":"Figure 4 applies a softmax transformation to raw scores, which visually amplifies differences; although this is acknowledged in Appendix B.8, the main-text discussion of '3.9% improvement' in Section 4.3 should state explicitly that the numbers refer to macro-averaged raw scores, not the transformed values in the radar-style plots.","section":"Appendix B.8 and Figure 4"}],"recommendation":"reject","confidential_remarks":"For the editor: the manuscript's code release and breadth of evaluation are attractive, but the central evaluation claim is contradicted by the reported tables and the model-selection protocol does not support a generalizable superiority claim. I would not encourage resubmission unless the evaluation is redone with a held-out validation protocol and the claims are substantially revised."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, NormWear is a real system: CWT-based tokenization, channel-independent MAE pretraining, [CLS]-attention fusion, and text-aligned zero-shot retrieval combined for the first time for multivariate wearable signals, trained on 2.5M segments and evaluated on 18 health tasks. The engineering is substantial and the code is released. Second, the central claim—\"consistently outperforms\" all baselines under zero-shot, partial-shot, and full-shot—does not survive contact with the paper's own tables.\n\nThe combination is genuinely new relative to the cited time-series and biosignal foundation models. The complexity analyses in Appendix G are thoughtful, and the demographic analysis in Table 14 is a nice check that the representations are not just encoding age or sex.\n\nThe big problem is evaluation. Section 3.4 says the masking strategy was chosen by observing which one \"yields the best performance for the downstream tasks,\" and Appendix C ablates fusion, masking, and input representation on the same 18 tasks used for final reporting. No separate validation set or nested selection is described. That means the reported margins are not unbiased estimates of a fixed model's generalization; they look like the result of selecting the best variant on the test tasks. Second, Table 13's 5-fold results diverge sharply from Table 2—WESAD goes from 76.1 to 89.6. The paper does not explain this. Third, the zero-shot evaluation is not fully unseen: the label-to-sentence templates used at inference overlap semantically with the pretraining text augmentation, so the alignment module has effectively seen these label concepts. Fourth, the \"consistent outperformance\" is contradicted at task level: NormWear loses to TF-C or modality-specific baselines on DriverFatigue, GAMEEMO, PPG-BP CVD, and some vital-sign tasks.\n\nNone of this means the system is useless. The architecture is plausible and the pretraining is large. But as written, the central claim is load-bearing and it fails. The honest conclusion is that this is a promising framework with an evaluation protocol that needs to be redone: hold out a set of tasks for architecture selection, report a single pre-registered split, and reconcile the k-fold numbers.\n\nI would send it to peer review because there is enough substance here to warrant serious refereeing, but I would expect reviewers to demand a corrected evaluation before acceptance. For your own work, I would not cite the performance claims, though the architecture might be worth citing as related work.","headline":"Substantial wearable foundation model, but the evaluation protocol undermines the headline claim of consistent outperformance.","tokens_in":33367,"tokens_out":2881,"would_cite":false,"duration_ms":28534,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A wearable-signal foundation model claims to beat task-specific baselines on 18 health tasks.","keywords":["wearable sensing foundation model","multivariate physiological signals","continuous wavelet transform","channel-aware attention","representation alignment","zero-shot transfer","masked autoencoder","health monitoring"],"falsifier":"Run a nested subject-stratified cross-validation on all 18 tasks, fixing all architecture choices on a separate validation set before touching the test folds, and compare macro-average AUC-ROC against TF-C, CLAP, and Chronos; if the margin collapses or reverses, the transfer claim is wrong. The paper's own tables already provide a concrete spot to check: WESAD full-shot linear probing is reported as 76.1 in Table 2 but 89.6 in the 5-fold subject-stratified cross-validation of Table 13, a gap far larger than the claimed margins over baselines.","tokens_in":32357,"feed_emoji":"🩺","tokens_out":8515,"duration_ms":84767,"temperature":0.7,"pith_summary":"NormWear is a foundation model for multivariate wearable physiological signals. The paper tries to establish that one pretrained model, using a continuous-wavelet-transform tokenizer and a channel-aware transformer, can produce reusable representations for arbitrary combinations of sensors such as PPG, ECG, EEG, GSR, and IMU. The authors evaluate it on 11 unseen datasets covering 18 applications in mental health, physical state inference, vital sign estimation, and disease risk evaluation. They report that it consistently outperforms statistical, spectral, self-supervised time-series, and modality-specific baselines under zero-shot, partial-shot, and full-shot linear probing. If the claim holds, a single model and unified preprocessing pipeline could replace bespoke per-sensor systems in wearable health monitoring.","feed_headline":"One pretrained model takes on 18 wearable health tasks","feed_subtitle":"NormWear transfers across arbitrary sensor layouts and claims top scores in zero-shot, partial-shot, and full-shot tests.","key_machinery":"The load-bearing machinery is a channel-aware masked autoencoder over continuous wavelet transform scalograms, followed by sensor-semantic alignment. A CWT with Mexican-hat wavelets at scales 1 to 64 turns each raw signal and its first two derivatives into three stacked scalograms, giving a modality-agnostic tokenization that preserves time and frequency information. A shared-weight transformer processes each channel independently, and a [CLS]-attention fusion layer exchanges information between channels with complexity $O(d \\cdot C^2)$ instead of full cross-attention; because self-attention is permutation-invariant, the encoder accepts any number and order of channels. Structured masking along the scale and time axes, with an expected mask ratio of 0.8, forces reconstruction of the original signal through a lightweight decoder. For zero-shot inference, the MSiTF aggregation module weights patches by relevance to a text query, recency, and a Gumbel-sampled importance gate, then aligns the fused embedding to sentence embeddings from a frozen clinical language model through contrastive and distance losses.","core_discovery":"The central discovery claimed is generalizable sensor-agnostic representation learning for wearable physiological signals. NormWear is pretrained on around 2.5 million augmented segments, roughly 14,943 hours of PPG, ECG, EEG, GSR, PCG, and IMU data, with a masked-autoencoder objective. Each channel is tokenized into an RGB-like stack of continuous wavelet transform scalograms of the raw series and its first and second derivatives, processed independently by a shared-weight encoder, and fused through self-attention over per-channel [CLS] tokens. After pretraining, a memory-stream-inspired temporal fusion module aligns the signal embeddings with text embeddings so that unseen tasks can be answered by retrieving the closest text label. The paper reports peak performance across all task groups and a 3.9 to 6.1 percent macro-average improvement over the strongest baselines in full-shot linear probing, plus zero-shot and 10%-label transfer results.","pith_inferences":["The uniform 65 Hz resampling and fixed wavelet scales limit the model to roughly the physiological frequency band; extending the same pipeline to audio or high-frequency biosignals would require adaptive scale selection, which the paper does not test.","Zero-shot accuracy likely depends on how labels are phrased; the paper varies templates during pretraining, but a reader could probe robustness by re-running retrieval with synonym prompts on each downstream dataset.","The channel-aware [CLS] fusion is not specific to physiology; the same architecture could be applied to multi-sensor industrial or environmental time series, but that extension is untested.","The paper's demographic analysis suggests signal embeddings and demographic features are complementary, so combining them is a natural step for clinical risk tasks; the occasional performance drops after concatenation mean confounding remains to be studied."],"forward_implications":["If the transfer claim is right, a new wearable health task can be tackled by feeding the sensor stream through the frozen encoder and, in the zero-shot setting, matching its embedding to text labels, with no per-task training or hand-tuned preprocessing.","Linear probing with only 10% of labels should already approach full-label performance on most of the 18 tasks, which matters for clinical datasets with scarce annotations.","A model can be deployed on different devices with different sensor counts or channel orders without retraining, because performance is reported to be stable under channel shuffling.","The fusion cost scaling as $O(d \\cdot C^2)$ rather than $O(d \\cdot (L \\cdot C)^2)$ makes the approach feasible for edge devices as channel count grows.","Larger pretraining corpora should continue to improve downstream accuracy, based on the reported scaling trend from 37k to 2.5M segments."],"supporting_citations":[{"why":"Supplies TF-C, the self-supervised time-frequency baseline that NormWear is compared against in full-shot, partial-shot, and zero-shot tables.","marker":"Zhang et al., 2022"},{"why":"Supplies Chronos, the time-series forecasting LLM baseline that anchors the forecasting-model comparison.","marker":"Ansari et al., 2024"},{"why":"Supplies CLAP, the spectrogram-based audio-language baseline and the main zero-shot comparison, and provides the contrastive language-audio alignment paradigm.","marker":"Wu et al., 2023"},{"why":"Supplies the masked autoencoder reconstruction objective and encoder-decoder design that NormWear's pretraining is built on.","marker":"He et al., 2021"},{"why":"Supplies the transformer self-attention formulation, including the permutation invariance and length flexibility that justify the channel-agnostic [CLS] fusion.","marker":"Vaswani et al., 2023"},{"why":"Provides the practical wavelet analysis that underpins the multi-resolution CWT scalogram tokenization.","marker":"Torrence & Compo, 1998"},{"why":"Provides the vision transformer patch-embedding procedure used to turn scalograms into tokens.","marker":"Dosovitskiy et al., 2020"},{"why":"Suggests using first and second derivatives of the signal series before spectral transformation, used in the three-channel scalogram stack.","marker":"Slapničar et al., 2019"},{"why":"Supplies the memory-stream retrieval mechanism that inspires the relevance, recency, and importance scoring in the MSiTF aggregation module.","marker":"Park et al., 2023"},{"why":"Supplies the frozen clinical language model that encodes query and label sentences for sensor-text alignment.","marker":"Muzammil, 2021"}],"fun_headline_variants":["Wearable foundation model masters 18 health tasks","NormWear: one model fits all wearables, zero-shot ready","Pretrained on 6 signals, NormWear tops 18 applications","Sensor-agnostic NormWear beats baselines across 11 datasets","Zero-shot to full-shot: NormWear leads wearable sensing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the benchmark protocol being a fair test: the model's masking, fusion, and tokenization choices were selected using the same downstream tasks, the held-out split is a single 80/20 subject-stratified split, and several datasets have fewer than 50 subjects, so a small evaluation instability could erase the reported margins.","fun_headline_variants_meta":{"raw":{"variants":["Wearable foundation model masters 18 health tasks","NormWear: one model fits all wearables, zero-shot ready","Pretrained on 6 signals, NormWear tops 18 applications","Sensor-agnostic NormWear beats baselines across 11 datasets","Zero-shot to full-shot: NormWear leads wearable sensing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000715,"raw_usage":{"total_tokens":3224,"prompt_tokens":962,"completion_tokens":2262,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":2172}},"tokens_in":578,"tokens_out":2262,"duration_ms":18000,"temperature":1.0,"reasoning_tokens":2172,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:45:52.857682+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a nested subject-stratified cross-validation on all 18 tasks, fixing all architecture choices on a separate validation set before touching the test folds, and compare macro-average AUC-ROC against TF-C, CLAP, and Chronos; if the margin collapses or reverses, the transfer claim is wrong. The paper's own tables already provide a concrete spot to check: WESAD full-shot linear probing is reported as 76.1 in Table 2 but 89.6 in the 5-fold subject-stratified cross-validation of Table 13, a gap far larger than the claimed margins over baselines.","supporting_citations":[],"review_version":1}