LastAct: Trajectory-Guided Latest-Activity Localization for Real-Time Smart-Home Activity Recognition
Pith reviewed 2026-06-28 22:50 UTC · model grok-4.3
The pith
Projecting sensor events onto a home floorplan enables better localization of the latest activity in mixed windows for smart-home recognition.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
LastAct forms layout-aligned trajectory image sequences by projecting sensor events onto the home floorplan, identifies contaminated windows with a lightweight gate, and estimates the last transition with a boundary localizer to apply boundary-guided masking that emphasizes post-boundary evidence while suppressing stale context, yielding competitive or superior performance on pure windows and substantial Macro-F1 gains on cross and mixed windows across four public datasets.
What carries the argument
The layout-aligned trajectory image sequence combined with boundary localizer for guided masking.
If this is right
- Sliding-window inference can be used without severe degradation from boundary contamination.
- Real-time smart-home applications gain robustness under mixed-activity protocols.
- Precomputed template cache enables efficient reuse without repeated rendering.
- Spatial context is explicitly modeled rather than treating sensor IDs independently.
Where Pith is reading between the lines
- Similar trajectory projections might improve other sequential sensor tasks like anomaly detection.
- Integration with video or wearable data could further enhance boundary accuracy if floorplans are available.
- The approach suggests that spatial layout is a key missing element in many ambient sensor pipelines.
Load-bearing premise
Mapping sensor events onto a home floorplan produces trajectories that preserve sufficient spatial continuity for reliable boundary localization without extra sensor metadata.
What would settle it
A controlled experiment showing no Macro-F1 improvement on mixed windows when the trajectory projection and boundary localizer are removed would falsify the central claim.
Figures
read the original abstract
Human Activity Recognition (HAR) from ambient sensors enables smart-home applications such as health monitoring and assisted living. In realistic deployments, however, sensor events arrive as a continuous stream and activity boundaries are unknown. Sliding-window inference therefore produces many windows that straddle transitions and contain mixed activities, creating boundary contamination that violates the pre-segmented instance assumption used by most benchmarks and models. Moreover, many pipelines under-use spatial context by treating sensor IDs as independent tokens. We present LastAct, a trajectory-centric framework for streaming smart-home HAR that targets the most recent activity under mixed windows while explicitly modeling spatial structure. LastAct projects sensor events onto the home floorplan to form a layout-aligned trajectory image sequence that preserves spatial continuity. A lightweight gate identifies contaminated windows, and a boundary localizer estimates the last transition to enable boundary-guided masking that emphasizes post-boundary evidence and suppresses stale context. For efficiency, we reuse a precomputed layout-aligned template cache to avoid repeated rendering. Empirically, across four public smart-home datasets under near-realistic mixed-activity protocols, LastAct achieves competitive or superior performance on pure windows and yields substantial Macro-F1 gains on cross/mixed windows, demonstrating improved robustness under near-realistic sliding-window regimes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents LastAct, a framework for real-time smart-home activity recognition from ambient sensor streams. It addresses the issue of mixed-activity windows in sliding-window inference by projecting sensor events onto a home floorplan to create trajectory image sequences, identifying contaminated windows with a gate, and localizing the last activity boundary to apply masking. The central empirical claim is that this yields competitive performance on pure windows and substantial Macro-F1 improvements on mixed windows across four public datasets.
Significance. Should the performance claims be substantiated with proper baselines and ablations, this work would offer a meaningful advance in handling realistic streaming conditions for HAR, where spatial layout is explicitly modeled rather than treated as independent tokens. The reuse of precomputed layout templates for efficiency is a positive practical aspect that could aid deployment.
major comments (3)
- [§3.2] §3.2: The method for mapping sensor events to layout-aligned trajectory images is presented without quantitative experiments testing robustness to sensor placement inaccuracies or reliance on precise vs. coarse metadata. This is a load-bearing assumption for the trajectory-guided localization's effectiveness, as errors in projection could disconnect trajectories and negate the spatial continuity benefit.
- [Experimental results] Experimental results: The abstract reports gains on four datasets but the manuscript provides insufficient detail on the specific baselines used, statistical significance of the Macro-F1 improvements, error bars, or ablations isolating the contribution of the trajectory projection and boundary localizer. This prevents full assessment of the central claim.
- [Results tables] Results tables (if present): Comparisons under the mixed-activity protocols should include clear definitions of 'pure', 'cross', and 'mixed' windows along with dataset characteristics to allow reproduction and verification of the reported gains.
minor comments (2)
- [Abstract] Abstract: Consider naming the four public datasets explicitly to provide immediate context for readers familiar with the field.
- [Notation] Notation: Ensure consistent use of terms like 'Macro-F1' and clarify if any hyperparameters are involved in the gate or localizer.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback. We address each major comment below and will incorporate revisions to strengthen the manuscript's clarity and empirical support.
read point-by-point responses
-
Referee: [§3.2] The method for mapping sensor events to layout-aligned trajectory images is presented without quantitative experiments testing robustness to sensor placement inaccuracies or reliance on precise vs. coarse metadata. This is a load-bearing assumption for the trajectory-guided localization's effectiveness, as errors in projection could disconnect trajectories and negate the spatial continuity benefit.
Authors: We agree this is a valid concern, as the effectiveness of layout-aligned trajectories depends on metadata quality. The manuscript relies on the floorplan metadata provided with each public dataset, which is standard in the field. In revision, we will add a dedicated sensitivity analysis subsection that quantifies performance under controlled perturbations to sensor positions and coarse vs. precise layouts, using the existing datasets where possible. revision: yes
-
Referee: Experimental results: The abstract reports gains on four datasets but the manuscript provides insufficient detail on the specific baselines used, statistical significance of the Macro-F1 improvements, error bars, or ablations isolating the contribution of the trajectory projection and boundary localizer. This prevents full assessment of the central claim.
Authors: We acknowledge that additional experimental details are needed for full assessment. The revised manuscript will expand the experimental section with: explicit enumeration and references for all baselines, error bars on Macro-F1 scores, statistical significance testing (e.g., paired t-tests across runs), and targeted ablations that separately disable the trajectory projection and boundary localizer components while reporting their isolated contributions. revision: yes
-
Referee: Results tables (if present): Comparisons under the mixed-activity protocols should include clear definitions of 'pure', 'cross', and 'mixed' windows along with dataset characteristics to allow reproduction and verification of the reported gains.
Authors: We will update the results section and associated tables to provide explicit operational definitions of 'pure', 'cross', and 'mixed' windows (including the sliding-window parameters used to generate them) and add a summary table of dataset characteristics such as number of sensors, activities, instances, and total duration for each of the four datasets. revision: yes
Circularity Check
No circularity; method is procedural with external empirical validation
full rationale
The paper describes a trajectory-projection pipeline, gate, and boundary localizer as a new framework evaluated on four public datasets under mixed-activity protocols. No equations, fitted parameters renamed as predictions, or self-citation chains appear in the abstract or described sections. Claims rest on comparative Macro-F1 results rather than reducing to inputs by definition or construction. The floorplan projection step is an explicit modeling choice, not a self-definitional loop.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Muhammad Raisul Alam, Mamun Bin Ibne Reaz, and Mohd Alauddin Mohd Ali. 2012. A review of smart homes—Past, present, and future.IEEE transactions on systems, man, and cybernetics, part C (applications and reviews)42, 6 (2012), 1190–1203
2012
-
[2]
Luca Arrotta, Gabriele Civitarese, and Claudio Bettini. 2022. Dexar: Deep explainable sensor-based activity recognition in smart-home environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies6, 1 (2022), 1–30
2022
-
[3]
Parviz Asghari, Elnaz Soleimani, and Ehsan Nazerfard. 2020. Online human activity recognition employing hierarchical hidden Markov models. Journal of Ambient Intelligence and Humanized Computing11 (2020), 1141–1152
2020
-
[4]
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271(2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[5]
Ahsanul Bari, Fahmid Al Farid, Md Tanjil Sarker, Sarina Mansor, Hezerul Abdul Karim, Md Roman Bhuiyan, and Hasanul Bannah. 2024. Advancements in Multi-View Human Activity Recognition for Ambient Assisted Living. In2024 Multimedia University Engineering Conference (MECON). IEEE, 1–6
2024
-
[6]
Damien Bouchabou, Sao Mai Nguyen, Christophe Lohr, Benoit Leduc, and Ioannis Kanellos. 2021. Fully convolutional network bootstrapped by word encoding and embedding for activity recognition in smart homes. InInternational Workshop on Deep Learning for Human Activity Recognition. Springer, 111–125
2021
-
[7]
Damien Bouchabou, Sao Mai Nguyen, Christophe Lohr, Benoit LeDuc, and Ioannis Kanellos. 2021. A survey of human activity recognition in smart homes based on IoT sensors algorithms: Taxonomies, challenges, and opportunities with deep learning.Sensors21, 18 (2021), 6037
2021
-
[8]
Damien Bouchabou, Sao Mai Nguyen, Christophe Lohr, Benoit LeDuc, and Ioannis Kanellos. 2021. Using language model to bootstrap human activity recognition ambient sensors based in smart homes.Electronics10, 20 (2021), 2498
2021
-
[9]
Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors.ACM Computing Surveys (CSUR)46, 3 (2014), 1–33
2014
-
[10]
Marie Chan, Daniel Estève, Christophe Escriba, and Eric Campo. 2008. A review of smart homes—Present state and future challenges.Computer methods and programs in biomedicine91, 1 (2008), 55–81
2008
-
[11]
Hui Chen, Charles Gouin-Vallerand, Kévin Bouchard, Sébastien Gaboury, Mélanie Couture, Nathalie Bier, and Sylvain Giroux. 2023. Leveraging self-supervised learning for human activity recognition with ambient sensors. InProceedings of the 2023 ACM Conference on Information Technology for Social Good. 324–332
2023
-
[12]
Junxin Chen, Wei Wang, Bo Fang, Yu Liu, Keping Yu, Victor CM Leung, and Xiping Hu. 2023. Digital twin empowered wireless healthcare monitoring for smart home.IEEE Journal on Selected Areas in Communications41, 11 (2023), 3662–3676
2023
-
[13]
Liming Chen and Chris D Nugent. 2019. Sensor-based activity recognition review. InHuman activity recognition and behaviour analysis: for cyber-physical systems in smart environments. Springer, 23–47
2019
-
[14]
Xi Chen, Julien Cumin, Fano Ramparany, and Dominique Vaufreydaz. 2024. Towards llm-powered ambient sensor based multi-person human activity recognition. In2024 IEEE 30th International Conference on Parallel and Distributed Systems (ICPADS). IEEE, 609–616
2024
-
[15]
Diane J Cook, Aaron S Crandall, Brian L Thomas, and Narayanan C Krishnan. 2012. CASAS: A smart home in a box.Computer46, 7 (2012), 62–69
2012
-
[16]
Julien Cumin, Grégoire Lefebvre, Fano Ramparany, and James L Crowley. 2017. A dataset of routine daily activities in an instrumented home. In International conference on ubiquitous computing and ambient intelligence. Springer, 413–425
2017
-
[17]
Devleena Das, Yasutaka Nishimura, Rajan P. Vivek, Naoto Takeda, Sean T. Fish, Thomas Plötz, and Sonia Chernova. 2023. Explainable Activity Recognition for Smart Home Systems.ACM Trans. Interact. Intell. Syst.13, 2, Article 7 (May 2023), 39 pages. doi:10.1145/3561533
-
[18]
Tim De Ryck, Maarten De Vos, and Alexander Bertrand. 2021. Change point detection in time series data using autoencoders with a time-invariant representation.IEEE Transactions on Signal Processing69 (2021), 3513–3524
2021
-
[19]
Sourish Gunesh Dhekane, Shivam Tiwari, Manan Sharma, and Dip Sankar Banerjee. 2022. Enhanced Annotation Framework for Activity Recognition Through Change Point Detection. In2022 14th International Conference on COMmunication Systems & NETworkS (COMSNETS). 397–405. doi:10.1109/COMSNETS53615.2022.9668475
-
[20]
Ha Manh Do, Minh Pham, Weihua Sheng, Dan Yang, and Meiqin Liu. 2018. RiSH: A robot-integrated smart home for elderly care.Robotics and Autonomous Systems101 (2018), 74–92
2018
-
[21]
Shirin Enshaeifar, Ahmed Zoha, Andreas Markides, Severin Skillman, Sahr Thomas Acton, Tarek Elsaleh, Masoud Hassanpour, Alireza Ahrabian, Mark Kenny, Stuart Klein, et al. 2018. Health management and pattern analysis of daily living activities of people with dementia using in-home sensors and machine learning techniques.PloS one13, 5 (2018), e0195605
2018
-
[22]
Yazeed Yasin Ghadi, Mouazma Batool, Munkhjargal Gochoo, Suliman A Alsuhibany, Tamara Al Shloul, Ahmad Jalal, and Jeongmin Park. 2022. Improving the ambient intelligence living using deep learning classifier.Comput. Mater. Contin73, 1 (2022), 1037–1053
2022
-
[23]
Alireza Ghods and Diane J Cook. 2019. Activity2vec: Learning adl embeddings from sensor data with a sequence-to-sequence model.arXiv preprint arXiv:1907.05597(2019)
work page internal anchor Pith review Pith/arXiv arXiv 2019
-
[24]
Munkhjargal Gochoo, Tan-Hsu Tan, Shing-Hong Liu, Fu-Rong Jean, Fady S Alnajjar, and Shih-Chia Huang. 2018. Unobtrusive activity recognition of elderly people living alone using anonymous binary sensors and DCNN.IEEE journal of biomedical and health informatics23, 2 (2018), 693–702
2018
-
[25]
Bruna Maria Vittoria Guerra, Emanuele Torti, Elisa Marenzi, Micaela Schmid, Stefano Ramat, Francesco Leporati, and Giovanni Danese. 2023. Ambient assisted living for frail people through human activity recognition: state-of-the-art, challenges and future directions.Frontiers in neuroscience 17 (2023), 1256682
2023
-
[26]
Harish Haresamudram, Irfan Essa, and Thomas Ploetz. 2024. Towards learning discrete representations via self-supervision for wearables-based human activity recognition.Sensors24, 4 (2024), 1238
2024
-
[27]
Harish Haresamudram, Irfan Essa, and Thomas Plötz. 2021. Contrastive Predictive Coding for Human Activity Recognition.Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.5, 2, Article 65 (June 2021), 26 pages. doi:10.1145/3463506
-
[28]
Md Milon Islam, Ashikur Rahaman, and Md Rashedul Islam. 2020. Development of smart healthcare monitoring system in IoT environment.SN computer science1 (2020), 1–11
2020
-
[29]
Eleni Kanasi, Srinivas Ayilavarapu, and Judith Jones. 2016. The aging population: demographics and the biology of aging.Periodontology 200072, 1 (2016), 13–18
2016
-
[30]
Alexander Karpekov, Sonia Chernova, and Thomas Plötz. 2026. VISAR: Visualization and Interpretation of Sensor-based Activity Recognition for Smart Homes. InCompanion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing(Finland)(UbiComp Companion ’25). Association for Computing Machinery, New York, NY, USA, 1685–1688. doi:10...
- [31]
-
[32]
Narayanan C Krishnan and Diane J Cook. 2014. Activity recognition on streaming sensor data.Pervasive and mobile computing10 (2014), 138–154
2014
-
[33]
Fatemeh Sadat Lesani, Faranak Fotouhi Ghazvini, and Hossein Amirkhani. 2021. Smart home resident identification based on behavioral patterns using ambient sensors.Personal and Ubiquitous Computing25 (2021), 151–162
2021
-
[34]
Ying Li, Junsheng Wu, Weigang Li, Aiqing Fang, and Wei Dong. 2023. Temporal–spatial dynamic convolutional neural network for human activity recognition using wearable sensors.IEEE Transactions on Instrumentation and Measurement72 (2023), 1–12
2023
-
[35]
Daniele Liciotti, Michele Bernardini, Luca Romeo, and Emanuele Frontoni. 2020. A sequential deep learning application for recognising human activities in smart homes.Neurocomputing396 (2020), 501–513
2020
-
[36]
Hui Liu, Yale Hartmann, and Tanja Schultz. 2022. A Practical Wearable Sensor-based Human Activity Recognition Research Pipeline.. InHealthinf. 847–856
2022
-
[37]
Alan Mazankiewicz, Klemens Böhm, and Mario Bergés. 2020. Incremental real-time personalization in human activity recognition using domain adaptive batch normalization.Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies4, 4 (2020), 1–20
2020
-
[38]
Gadelhag Mohmed, Ahmad Lotfi, and Amir Pourabdollah. 2020. Employing a deep convolutional neural network for human activity recognition based on binary ambient sensor data. InProceedings of the 13th ACM international conference on pervasive technologies related to assistive environments. 1–7
2020
-
[39]
Mustafa A Mustafa, Alexandros Konios, and Matias Garcia-Constantino. 2021. IoT-based activities of daily living for abnormal behavior detection: Privacy issues and potential countermeasures.IEEE Internet of Things Magazine4, 3 (2021), 90–95
2021
-
[40]
Qin Ni, Timothy Patterson, Ian Cleland, and Chris Nugent. 2016. Dynamic detection of window starting positions and its implementation within an activity recognition framework.Journal of biomedical informatics62 (2016), 171–180
2016
-
[41]
George A Oguntala, Yim Fun Hu, Ali AS Alabdullah, Raed A Abd-Alhameed, Muhammad Ali, and Doanh K Luong. 2021. Passive RFID module with LSTM recurrent neural network activity classification algorithm for ambient-assisted living.IEEE Internet of Things Journal8, 13 (2021), 10953–10962
2021
-
[42]
Jiho Park, Kiyoung Jang, and Sung-Bong Yang. 2018. Deep neural networks for activity recognition with multi-sensor data in a smart home. In2018 IEEE 4th World Forum on Internet of Things (WF-IoT). IEEE, 155–160
2018
-
[43]
Thomas Plötz et al. 2024. Using Graphs to Perform Effective Sensor-Based Human Activity Recognition in Smart Homes.Sensors24, 12 (2024), 3944
2024
-
[44]
Daniele Riboni, Claudio Bettini, Gabriele Civitarese, Zaffar Haider Janjua, and Rim Helaoui. 2016. SmartFABER: Recognizing fine-grained abnormal behaviors for early detection of mild cognitive impairment.Artificial intelligence in medicine67 (2016), 57–74
2016
-
[45]
Deepika Singh, Erinc Merdivan, Sten Hanke, Johannes Kropf, Matthieu Geist, and Andreas Holzinger. 2017. Convolutional and recurrent neural networks for activity recognition in smart environment. InTowards Integrative Machine Learning and Knowledge Extraction: BIRS Workshop, Banff, AB, Canada, July 24-26, 2015, Revised Selected Papers. Springer, 194–205
2017
-
[46]
Deepika Singh, Erinc Merdivan, Ismini Psychoula, Johannes Kropf, Sten Hanke, Matthieu Geist, and Andreas Holzinger. 2017. Human activity recognition using recurrent neural networks. InMachine Learning and Knowledge Extraction: First IFIP TC 5, WG 8.4, 8.9, 12.9 International Cross-Domain Conference, CD-MAKE 2017, Reggio, Italy, August 29–September 1, 2017...
2017
-
[47]
Jordan Tewell, Dympna O’Sullivan, Neil Maiden, James Lockerbie, and Simone Stumpf. 2019. Monitoring meaningful activities using small low-cost devices in a smart home.Personal and Ubiquitous Computing23 (2019), 339–357
2019
-
[48]
Dipanwita Thakur and Suparna Biswas. 2022. Online change point detection in application with transition-aware activity recognition.IEEE Transactions on Human-Machine Systems52, 6 (2022), 1176–1185
2022
-
[49]
Megha Thukral, Sourish Gunesh Dhekane, Shruthi K Hiremath, Harish Haresamudram, and Thomas Ploetz. 2025. Layout-Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST).Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies9, 1 (2025), 1–38
2025
-
[50]
Md Zia Uddin, Weria Khaksar, and Jim Torresen. 2018. Ambient sensors for elderly care and independent living: a survey.Sensors18, 7 (2018), 2027
2018
-
[51]
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748 [cs.LG] https://arxiv.org/abs/1807.03748
work page internal anchor Pith review Pith/arXiv arXiv 2019
-
[52]
Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. 2019. Deep learning for sensor-based activity recognition: A survey.Pattern recognition letters119 (2019), 3–11
2019
-
[53]
Nawel Yala, Belkacem Fergani, and Anthony Fleury. 2015. Feature extraction for human activity recognition on streaming data. In2015 International symposium on innovations in intelligent systems and applications (INISTA). IEEE, 1–6
2015
-
[54]
Surong Yan, Yixing Liao, Xiaoqing Feng, and Yanan Liu. 2016. Real time activity recognition on streaming sensor data for smart environments. In 2016 International Conference on Progress in Informatics and Computing (PIC). IEEE, 51–55
2016
-
[55]
Jiancong Ye, Hongjie Jiang, and Junpei Zhong. 2023. A graph-attention-based method for single-resident daily activity recognition in smart homes. Sensors23, 3 (2023), 1626
2023
-
[56]
Vincent Gbouna Zakka, Zhuangzhuang Dai, and Luis J Manso. 2024. Action recognition for privacy-preserving ambient assisted living. In International Conference on AI in Healthcare. Springer, 203–217
2024
-
[57]
Pure” denotes single-activity windows; “10–30
Junhao Zhao, Zishuai Liu, Ruili Fang, Jin Lu, Linghan Zhang, and Fei Dou. 2026. CARE: Contrastive Alignment for ADL Recognition from Event-Triggered Sensor Streams. InProceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom). A Appendix A.1 Implementation Details Our proposed framework is implemented in PyTorch an...
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.