REVIEW 4 major objections 5 minor 27 references
Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Decomposing then fusing sensor signals sets a new best for wearable activity recognition on three public datasets.
desk verdict Solid architecture, shaky comparison table: the SOTA claim needs verification before it can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a two-phase architecture: Modality-Aware Signal Decomposition isolates each sensor, embeds each variable channel separately, and applies depth-wise 1D convolutions so every intra-sensor channel keeps its own temporal kernel; Hierarchical Interaction Fusion then reverses that separation with grouped point-wise convolutions, a Mamba Selective SSM block for global temporal dependencies, and a self-attention layer whose queries, keys, and values are entire per-sensor feature maps, yielding asymmetric inter-sensor correlation scores. This design replaces shared convolutional kernels and symmetric GCN-based sensor fusion, and its ablation shows each stage contributes.
What would settle it
Re-run all baseline models on Opportunity, Realdisp, and Skoda under DecomposeWHAR's exact preprocessing, leave-one-user-out splits, and 800 ms windows, and check whether the accuracy and macro-F1 gaps shrink or vanish; the claim fails if DecomposeWHAR no longer leads. For Realdisp, also run the full 17-user protocol.
Extended reading notes
Core claim
The central claim is that decomposing multi-sensor signals at sensor, variable, and channel levels, then fusing them in reverse hierarchical order, lets the model capture intra-sensor and inter-sensor spatio-temporal relationships better than shared-kernel convolutions or graph-based sensor fusion. On the three datasets, DecomposeWHAR achieves 78.28% accuracy and 72.04% macro-F1 on Opportunity, 96.64% and 96.10% on Realdisp, and 97.61% and 97.24% on Skoda, beating the runner-up by 1.14 to 4.21 accuracy points and up to 6.93 macro-F1 points. The paper also shows that removing the global temporal aggregation or cross-sensor attention stages sharply degrades performance, which supports the claim that both stages are load-bearing.
Load-bearing premise
The results assume the quoted baseline numbers were produced under the same preprocessing, user splits, and window size as DecomposeWHAR; if the protocols differ, especially on Realdisp where only 10 of 17 users are used, the reported margin may partly reflect evaluation differences rather than model quality.
Editorial extensions
If this is right
- DecomposeWHAR becomes the new reference point for multi-sensor wearable activity recognition on Opportunity, Realdisp, and Skoda if the reported numbers reproduce.
- The depth-wise decomposition eliminates the need for shared convolutional kernels across sensor variables, so per-modality temporal features survive until later fusion.
- Inter-sensor relationships no longer depend on a predefined graph; the self-attention module learns them from data and captures asymmetry between sensors.
- The model retains acceptable computational cost, with FLOPs under 600M and competitive inference time on a wearable watch, so the accuracy gains are not purchased with disproportionate compute.
- Ablations show the cross-sensor attention and global temporal aggregation stages carry most of the improvement; removing both degrades macro-F1 by 9.27 to 19.51 points across datasets.
Reading between the lines
- If the decomposition-fusion pattern is what drives the gains, the same sensor-level, variable-level, and channel-level decomposition could transfer to other multivariate time-series tasks such as EEG analysis or industrial sensor monitoring, but the paper does not test this.
- The learned self-attention weights might serve as a diagnostic tool for which body-worn sensors matter most for a given activity, since the attention matrix is asymmetric and inspectable.
- A direct re-implementation of all baselines under identical preprocessing would settle whether the reported margins are model gains or protocol effects, especially given the paper's use of 10 of 17 Realdisp users.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DecomposeWHAR, a multi-sensor wearable human activity recognition model that decomposes input signals at sensor, variable, channel, and temporal levels and then fuses them hierarchically. The decomposition phase uses a modality-specific embedding and depth-wise convolutions to preserve per-variable temporal features; the fusion phase combines channel- and variable-level features, applies a Mamba block for global temporal aggregation, and uses a self-attention mechanism for cross-sensor interaction. The authors evaluate on Opportunity, Realdisp, and Skoda, reporting accuracy and macro-F1 improvements over several baselines, and they include ablations, parameter analyses, and an efficiency comparison.
Significance. If the reported gains are reproducible under a matched evaluation protocol, the paper would represent a meaningful step in multi-sensor WHAR: the architecture is well-motivated, the ablations are informative, and the efficiency measurements on a wearable device are a valuable addition. The authors also release code and provide standard deviations, which are good scientific practices. However, the central SOTA claim currently rests on baseline numbers quoted from a prior paper without demonstrated protocol equivalence, and the printed improvement deltas in Table 1 contain arithmetic inconsistencies. The scientific contribution is therefore plausible but not yet established at the level required for acceptance.
major comments (4)
- [Experiments, 'Experimental Results' and Table 1] The comparison table is not yet sufficient to support the SOTA claim. The text states that results of models other than HARMamba and IF-ConvTransformer are quoted from Miao et al. (2022), but the paper does not establish that Miao et al. used the same data preprocessing, the same 800 ms window with the same overlap, the same user splits, and — for Realdisp — the same 10-user subset. Since the Realdisp section explicitly says only 10 of 17 users are used 'due to incomplete data,' a baseline computed on a different subset would make the reported 4+ point gaps incomparable. The authors should rerun all baselines under the exact protocol used for DecomposeWHAR, or provide a precise per-dataset mapping of the quoted numbers to the exact protocol and user subsets, including evidence that the splits and preprocessing coincide.
- [Table 1, '(Improvement)' rows] The improvement values printed in Table 1 do not consistently match the printed means. For example, the Opportunity macro-F1 improvement for DecomposeWHAR is listed as 6.93%, but the largest visible gap to any runner-up is 72.04 - 67.05 = 4.99 against HARMamba (or 5.91 against DynamicWHAR); the Realdisp accuracy improvement is listed as 4.21%, but 96.64 - 92.58 = 4.06 against DynamicWHAR. Several other deltas are off by 0.1-0.2 points. These values must be recomputed and the reference runner-up for each cell stated explicitly.
- [Ablation Study, Table 2] The text claims that 'Replacing the CSI module with a GCN (GCN as CSI) proposed in DynamicWHAR led to a decline in performance,' but Table 2 contains no 'GCN as CSI' row. This ablation result is therefore not reported and the claim is unsupported. Either add the corresponding row with numbers and standard deviations, or remove the claim from the text.
- [Experimental Settings, Skoda protocol] For Skoda, the paper describes an 80/10/10 holdout split but does not specify how the split is performed. Because Skoda contains a single subject, a random split without temporal ordering constraints can place adjacent windows from the same activity instance in both training and test sets, inflating accuracy. The authors should describe the split procedure (e.g., contiguous segment boundaries, seed, or leave-one-session-out) and justify that it matches the protocol used for the quoted baselines.
minor comments (5)
- [Experimental Settings] There is a typo: 'Readisp' should be 'Realdisp'.
- [Equation (1)] The expression 'T = L S' should be written as T = floor(L/S) or otherwise clarified, since S is the stride and the output length should be an integer.
- [Figure 3 and Computational Efficiency paragraph] The figure caption says FLOPs are not shown 'due to significant discrepancies,' while the text states that DecomposeWHAR, HARMamba, and DynamicWHAR have FLOPs under 600M. Please reconcile these statements and, if FLOPs are omitted, report them in a table or appendix so the efficiency claim is verifiable.
- [Table 1] The '(Improvement)' line after the HARMamba row is unexplained: it is unclear what baseline the HARMamba improvements are computed against and why the values are negative for some metrics. This row should be removed or clearly labeled.
- [Notation] The abbreviation 'MSE' for Modality-Specific Embedding is easy to confuse with Mean Squared Error; a different acronym (e.g., 'MSEmb') would improve readability.
Circularity Check
No significant circularity; the model is evaluated on held-out benchmark splits and its components are standard, independently motivated building blocks.
full rationale
The paper's central claim is empirical: DecomposeWHAR achieves higher accuracy and macro-F1 than prior methods on Opportunity, Realdisp, and Skoda. This claim is supported by training the model on training folds and evaluating on held-out users or a held-out Skoda split, using cross-entropy loss directly on the target labels. There is no step in the derivation where a fitted parameter is renamed as a prediction or where a component is defined in terms of the outcome it is supposed to explain. The architectural pieces—modality-specific embedding, depth-wise convolution, point-wise fusion, Mamba-based global aggregation, and self-attention cross-sensor interaction—are each described by explicit equations and are standard building blocks drawn from external literature; none of them imports a uniqueness theorem or an unverified self-citation to force the design. The comparison baselines are quoted from Miao et al. (2022) or implemented by the authors, but this is ordinary benchmarking practice rather than circular reasoning, even though protocol mismatches could affect the validity of the SOTA comparison. The only circularity-adjacent concern is that hyperparameters such as kernel size, patch size, and output dimension were tuned using performance on the Opportunity dataset (Figure 5), which may inflate reported numbers on that benchmark; however, this is a model-selection and reproducibility concern, not a case where the reported result is equivalent to its input by construction. Therefore, the paper is not circular in any load-bearing sense.
Assumptions & free parameters
free parameters (6)
- Output channels D of MSE =
64
- LTE kernel size Kdw =
3
- Patch size P =
L/3 (one third of time steps)
- Number of attention heads =
8
- Learning rate per dataset =
0.001 (Opportunity, Realdisp), 0.0001 (Skoda)
- Batch size =
64 (Opportunity, Skoda), 128 (Realdisp)
assumptions (3)
- domain assumption The three benchmark datasets (Opportunity, Realdisp, Skoda) are used as standard and their labels and sensor readings are reliable.
- domain assumption The evaluation protocols (leave-one-user-out for Opportunity and Realdisp, 80/10/10 holdout for Skoda) are appropriate and consistent with the baselines quoted from Miao et al. (2022).
- standard math Depth-wise separable convolutions, Mamba, and self-attention are effective building blocks for time-series feature extraction, as claimed in prior work.
Cite this review
Pith. "Pith review of Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition." pith.science (2026). https://pith.science/paper/2W5DSI2L
@misc{pith2026250110917,
author = {Pith},
title = {Pith review of: Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/2W5DSI2L}},
note = {Machine review of arXiv:2501.10917}
}
read the original abstract
Wearable Human Activity Recognition (WHAR) is a prominent research area within ubiquitous computing. Multi-sensor synchronous measurement has proven to be more effective for WHAR than using a single sensor. However, existing WHAR methods use shared convolutional kernels for indiscriminate temporal feature extraction across each sensor variable, which fails to effectively capture spatio-temporal relationships of intra-sensor and inter-sensor variables. We propose the DecomposeWHAR model consisting of a decomposition phase and a fusion phase to better model the relationships between modality variables. The decomposition creates high-dimensional representations of each intra-sensor variable through the improved Depth Separable Convolution to capture local temporal features while preserving their unique characteristics. The fusion phase begins by capturing relationships between intra-sensor variables and fusing their features at both the channel and variable levels. Long-range temporal dependencies are modeled using the State Space Model (SSM), and later cross-sensor interactions are dynamically captured through a self-attention mechanism, highlighting inter-sensor spatial correlations. Our model demonstrates superior performance on three widely used WHAR datasets, significantly outperforming state-of-the-art models while maintaining acceptable computational efficiency.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Ranasinghe, D. C. 2021. Attend and Discriminate: Beyond the State-of-the-Art for Human Activity Recognition Using Wearable Sensors. Proceedings of the ACM on Interactive,
work page 2021
-
[2]
Ahmad, N.; and Leung, H.-f. 2024. HyperHAR: Inter-sensing Device Bilateral Correlations and Hyper- correlations Learning Approach for Wearable Sensing De- vice Based Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1:1–1:29
work page 2024
-
[3]
Attal, F.; Mohammed, S.; Dedabrishvili, M.; Chamroukhi, F.; Oukhellou, L.; and Amirat, Y . 2015. Physical human ac- tivity recognition using wearable sensors. Sensors, 31314– 31338. Ba˜nos, O.; Damas, M.; Pomares, H.; Rojas, I.; T ´oth, M. A.; and Amft, O. 2012. A benchmark dataset to evaluate sensor displacement in activity recognition. In UbiComp
work page 2015
-
[4]
Chen, L.; Zhang, Y .; Miao, S.; Zhu, S.; Hu, R.; Peng, L.; and Lv, M. 2022. SALIENCE: An unsupervised user adap- tation model for multiple wearable sensors based human ac- tivity recognition. IEEE Transactions on Mobile Computing, 5492–5503
work page 2022
-
[5]
Chollet, F. 2017. Xception: Deep learning with depthwise separable convolutions. In CVPR
work page 2017
-
[6]
Dao, T.; and Gu, A. 2024. Transformers are SSMs: Gener- alized Models and Efficient Algorithms Through Structured State Space Duality. In ICML
work page 2024
-
[7]
Donghao, L.; and Xue, W. 2023. ModernTCN: A Modern Pure Convolution Structure for General Time Series Analy- sis. In ICLR
work page 2023
-
[8]
Y .; Halloran, S.; and Pl ¨otz, T
Hammerla, N. Y .; Halloran, S.; and Pl ¨otz, T. 2016. Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables. In IJCAI
work page 2016
Show all 27 references
-
[9]
Han, J.; He, Y .; Liu, J.; Zhang, Q.; and Jing, X. 2019. Graph- ConvLSTM: Spatiotemporal learning for activity recogni- tion with wearable sensors. In GLOBECOM
2019
-
[10]
D.; and Pl ¨otz, T
Kwon, H.; Wang, B.; Abowd, G. D.; and Pl ¨otz, T. 2021. Approaching the real-world: Supporting activity recognition training with virtual imu data. Proceedings of the ACM on
2021
-
[11]
Wan, Y . 2024. HARMamba: Efficient Wearable Sensor Hu- man Activity Recognition Based on Bidirectional Selective SSM. arxiv:2403.20183
2024 arXiv
-
[12]
Mahmud, S.; Tanjid Hasan Tonmoy, M.; Kumar Bhaumik, K.; Mahbubur Rahman, A.; Ashraful Amin, M.; Shoyaib, M.; Asif Hossain Khan, M.; and Ahsan Ali, A. 2020. Human activity recognition from wearable sensor data using self- attention. In ECAI 2020
2020
-
[13]
Miao, S.; Chen, L.; Hu, R.; and Luo, Y . 2022. Towards a Dynamic Inter-Sensor Correlations Learning Framework for Multi-Sensor-Based Wearable Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 130:1–130:25. M¨unzner...
2022
-
[14]
Stiefelhagen, R.; and D¨urichen, R. 2017. CNN-based sensor fusion techniques for multimodal human activity recogni- tion. In ISWC
2017
-
[15]
S.; and Pl¨otz, T
Murahari, V . S.; and Pl¨otz, T. 2018. On attention models for human activity recognition. In ISWC
2018
-
[16]
Mutegeki, R.; and Han, D. S. 2020. A CNN-LSTM approach to human activity recognition. In ICAIIC. Ord´o˜nez, F. J.; and Roggen, D. 2016. Deep Convolu- tional and LSTM Recurrent Neural Networks for Multi- modal Wearable Activity Recognition. Sensors, 115
2020
-
[17]
Roggen, D.; Calatroni, A.; Rossi, M.; Holleczek, T.; F¨orster, K.; Tr ¨oster, G.; Lukowicz, P.; Bannach, D.; Pirkl, G.; Fer- scha, A.; et al. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In INSS
2010
-
[18]
Stiefmeier, T.; Roggen, D.; Ogris, G.; Lukowicz, P.; and Tr¨oster, G. 2008. Wearable activity tracking in car manu- facturing. IEEE Pervasive Computing, 42–50
2008
-
[19]
Tonmoy, M. T. H.; Mahmud, S.; Mahbubur Rahman, A.; Ashraful Amin, M.; and Ali, A. A. 2021. Hierarchical self attention based autoencoder for open-set human activ- ity recognition. In PAKDD
2021
-
[20]
N.; Kaiser, Ł.; and Polosukhin, I
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017. At- tention is all you need. Advances in neural information pro- cessing systems. In NIPS
2017
-
[21]
Williams, A
Wang, C.; Gao, Y .; Mathur, A.; De C. Williams, A. C.; Lane, N. D.; and Bianchi-Berthouze, N. 2021. Leveraging activity recognition to enable protective behavior detection in con- tinuous data. Proceedings of the ACM on Interactive, Mo- bile, Wearable and Ubiquitous Technologies, 1–27
2021
-
[22]
Yan, S.; Xiong, Y .; and Lin, D. 2018. Spatial temporal graph convolutional networks for skeleton-based action recogni- tion. In AAAI
2018
-
[23]
N.; San, P
Yang, J.; Nguyen, M. N.; San, P. P.; Li, X.; and Krish- naswamy, S. 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In IJCAI
2015
-
[24]
Yuan, H.; Sun, Q.; Fu, X.; Ji, C.; and Li, J. 2024. Dynamic Graph Information Bottleneck. In WWW
2024
-
[25]
Li, J. 2023. Environment-aware dynamic graph learning for out-of-distribution generalization. In NIPS
2023
-
[26]
Zhang, Y .; Wang, L.; Chen, H.; Tian, A.; Zhou, S.; and Guo, Y . 2022. IF-ConvTransformer: A Framework for Human Activity Recognition Using IMU Fusion and ConvTrans- former. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 88:1–88:26
2022
-
[27]
Zhou, Y .; Hefenbrock, M.; Huang, Y .; Riedel, T.; and Beigl, M. 2021. Automatic remaining useful life estimation frame- work with embedded convolutional LSTM as the backbone. In ECML PKDD
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.