{"id":"f95b38ac-dc0e-492b-8978-1bfca4458a61","arxiv_id":"2502.17472","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A 24-class HAR model runs on an ISPU sensor in under 850 bytes of stack memory with 85% accuracy.","lead":"The authors show a 24-class human activity recognizer that runs in under 850 bytes of memory on an intelligent sensor chip. It reaches 85% accuracy on their own dataset, aiming to make gesture recognition work directly on the sensor without sending data to the cloud.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 85% accuracy is not a clean held-out estimate: the top-20% feature set and the 24-class taxonomy are both chosen using the full dataset, so the 0.82KB configuration is selected on the test data.","rationale":"The reader and I identify the same soft spot: data-dependent feature and class selection makes the headline accuracy an optimistic, post-selection estimate. I considered other potential concerns, including the underdocumented stack-memory measurement, the inconsistency between 31 and 24 initial classes, and the unavailable dataset/code. These are real limitations, but they are not demonstrably fatal, and the memory measurement is at least plausible for an XGBoost model with 16 features. The feature-selection and class-selection leakage is the most load-bearing because it directly undermines the quantitative claim that the 850-byte model achieves 85% accuracy in a way that would transfer to new data. A nested cross-validation replication is concrete and decisive. Since the reader already conditioned the verdict on exactly this issue, my stress-test does not change the recommended verdict.","tokens_in":9987,"tokens_out":3757,"duration_ms":42102,"concrete_test":"After releasing the dataset, rerun the pipeline with nested cross-validation: (1) start from the original 31 classes before any model fitting; (2) inside each training fold only, compute XGBoost feature importances and select the top 20%; (3) apply class-discard and merge decisions using only training-fold model behavior; (4) evaluate the resulting model on the held-out folds and report mean plus/minus standard deviation. Compare this nested-CV accuracy to the reported 84.7%. If it falls materially below 84.7%, for example by more than two points, or if variance across folds is large, the '85% accuracy under 850 bytes' claim is not supported as a clean held-out result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is '24-class HAR model achieving 85% accuracy while operating within an 850-byte stack memory limit.' The accuracy half rests on Table 1's XGBoost* row (84.7%), whose features are 'the top 20% most important features, as determined from the feature importance matrix of a model trained on the entire dataset' (Table 1 footnote). Using the full dataset to choose features before reporting an accuracy estimate violates the held-out principle: the test folds influence which 16 of 78 features are used, so the 84.7% figure is optimistic for new users or new recordings. The problem is compounded by Section 5's incremental class injection: classes are discarded ('Brushing Teeth', 'Driving') or merged ('Writing', 'Using Computer') based on observed model confusion on the same data, so the 24-class problem itself is selected post hoc. The paper's own Section 7 documents these data-driven class decisions. The reported accuracy is therefore for a taxonomy and feature set optimized on the evaluation data, not for a pre-specified 24-class task. The memory claim (0.82KB stack) may be correct for that specific compiled configuration, but the headline accuracy attached to it is not a valid unbiased estimate. The paper does not yet provide dataset/code or a nested evaluation protocol, so this cannot be verified from the manuscript alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a human-activity-recognition (HAR) pipeline for STMicroelectronics ISPUs. The authors collect 12.5 hours of accelerometer/gyroscope data, preprocess it (cleaning, downsampling to 26 Hz, 1.5 s windows, 78 statistical features), and compare 1D ResNet, LightGBM, XGBoost, a basic neural network, and Neuton AutoML. The headline result is an XGBoost model with 84.7% accuracy and a 0.82 KB stack footprint, obtained after selecting the top 20% features and after a class-selection procedure that reduces 31 initial gestures to 24. The paper claims this demonstrates in-sensor deployment of a 24-class HAR model under 850 bytes of stack memory.","tokens_in":10374,"tokens_out":5002,"duration_ms":53592,"significance":"The engineering contribution is timely: it shows that gradient-boosted trees can be compiled to ISPU-compatible C and run with a sub-kilobyte stack, and the proposed preprocessing and feature set may be useful for other ultra-constrained HAR tasks. The paper also contains a useful comparison of several model families on the same pipeline. However, because the feature subset and the 24-class label set are both selected using the full dataset, the reported 84.7% accuracy is not a clean held-out estimate of the deployed system's accuracy. The memory claim for the resulting configuration may be valid, but the accuracy attached to it is not yet credible. No code, binary, or dataset URL is provided, so the central numbers cannot be independently verified.","major_comments":[{"comment":"The reported XGBoost* accuracy of 84.7% and the associated 0.82 KB stack footprint are obtained after selecting the top 20% features using the feature importance of a model trained on the entire dataset. This makes the feature subset a function of the evaluation data, so the 84.7% figure cannot be read as an unbiased estimate of the accuracy of the deployed configuration. Please provide a nested evaluation (feature selection performed inside each training fold only) or a separate validation set, and report the resulting accuracy for the final fixed configuration.","section":"Table 1, footnote * and Section 6"},{"comment":"The 24-class label set is not pre-specified: 'Brushing Teeth' and 'Driving' are discarded and 'Writing'/'Using Computer' are merged based on observed confusions during incremental class injection on the full dataset. Consequently, the reported accuracy is for a taxonomy selected post hoc, not for a fixed 24-class task. The evaluation should treat class-set selection as part of the model-selection procedure, for example by fixing the taxonomy before evaluation or by using nested validation over the class-selection rule.","section":"Sections 5 and 7"},{"comment":"The text reports '5-fold cross-validation' but does not specify whether the folds split by window, by recording session, or by participant. If windows from the same recording appear in both training and test folds, temporally correlated samples will inflate accuracy. Please describe the split and, if participant-level generalization is intended, report leave-one-participant-out results and per-class accuracy or a confusion matrix.","section":"Section 6"},{"comment":"The headline 'under 850 bytes' refers only to the stack footprint of one XGBoost configuration; the same row reports a 13 Kb program footprint, and Section 3 notes that the ISPU has no ROM and requires a host MCU to load the program. The paper should state the full memory budget (data memory, program memory, and host overhead) and provide the compiled artifacts, since the memory claim is as central as the accuracy claim.","section":"Section 3 and Table 1"}],"minor_comments":[{"comment":"The first paragraph says the data collection encompassed '31 distinct classes', then says only 24 will be used; the second paragraph immediately says the dataset comprises '24 distinct classes'. This contradiction should be resolved.","section":"Section 4.1"},{"comment":"The heading 'EXPIREMENTS' and the repeated 'Expirement 1/2/3' and 'Experement 3' are typos and should be corrected to 'Experiments'.","section":"Section 5"},{"comment":"Units are used inconsistently across the table and text: 'KB', 'Kb', and 'Bytes' appear interchangeably (e.g., 0.82KB, 13Kb, 21Kb). Define the units and use them consistently, and verify that '13Kb' is not meant to be '13KB'.","section":"Table 1"},{"comment":"The abstract and conclusion claim '85% accuracy' while Table 1 reports 84.7%. Reconcile the exact number or round consistently.","section":"Abstract / Section 8"},{"comment":"The class 'hand-still' is mentioned in the driving-class discussion but is not listed among the 24 classes in Section 4.1; clarify which class this refers to.","section":"Section 7"},{"comment":"The paper states the dataset is 'publicly available' but provides no URL or repository. Add a data availability statement and, if possible, link to the code or compiled ISPU binaries to support reproducibility.","section":"Abstract / Data availability"}],"recommendation":"major_revision","confidential_remarks":"The paper is borderline in its current form because the headline accuracy is not a valid held-out estimate: feature selection and class-set selection both use the full dataset. I recommend asking the authors for a nested or pre-registered evaluation protocol, a clear participant/window split, and data/code availability before considering acceptance. The relation to the existing 8 KB on-sensor learning work cited as [11] should also be clarified to establish novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the short version of this paper is credible for the engineering footprint but not for the accuracy number. A 24-class XGBoost classifier with 5,389 parameters running on the ISPU with a sub-850-byte stack is a real thing they likely did. The 84.7% accuracy, however, is not a clean held-out estimate.\n\nWhat's new: they push a tree ensemble into a much smaller stack than the 8KB on-sensor learning work they cite, and they show a performance-memory tradeoff with a basic NN at 92.47% and 4.2KB stack. The preprocessing pipeline is reasonable (downsample to 26Hz, 1.5s windows, 78 statistical features), and the UMAP plots suggest the classes separate. The dataset, if it actually gets released, would be useful.\n\nSoft spots: the reported accuracy is optimistic because the top-20% feature set was chosen using a model trained on the entire dataset (Table 1 footnote), and the 24-class taxonomy itself was pruned and merged based on model behavior on the same data (Section 7). Those are post-hoc configuration choices, so the '85%' is not a valid estimate for a pre-specified task. The memory claim is also under-specified: stack footprint alone ignores the program memory for the tree traversal code, and the 0.5mA power figure appears to be the sensor's idle current rather than measured inference current. Dataset and code are announced but not available, and the writing is sloppy (duplicated references, typos, inconsistent class counts between sections).\n\nWho this is for: tinyML practitioners who want a data point on what fits on the ISPU. It's a workshop-level extended abstract, not a rigorous study. But the core hardware constraint question is legitimate, and the authors are transparent about their pipeline, so it deserves a proper review rather than a desk reject. I would ask them to redo evaluation with feature selection inside CV folds, report total memory (data + program), and actually release the dataset and code. Then the engineering claim would be solid and the accuracy number could be trusted.","headline":"A credible sub-850-byte ISPU deployment demo, but the 85% accuracy is a post-hoc estimate, not a held-out result.","tokens_in":10800,"tokens_out":2898,"would_cite":false,"duration_ms":31851,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 24-class human activity recognition model can run directly inside an intelligent sensor's processing unit using less than 850 bytes of stack memory while holding about 85 percent accuracy.","keywords":["Human Activity Recognition","TinyML","Intelligent Sensor Processing Unit","on-sensor inference","incremental class injection","statistical feature extraction","IMU accelerometer gyroscope","memory-constrained models"],"falsifier":"Retrain the same pipeline on the released dataset with classes fixed in advance and feature importance computed separately inside each cross-validation training fold; if the held-out accuracy drops materially below 84.7%, the headline number is inflated by selection on the full data.","tokens_in":9793,"feed_emoji":"🏃","tokens_out":9401,"duration_ms":91571,"temperature":0.7,"pith_summary":"This paper tries to show that a full 24-class human activity recognition (HAR) system can be embedded directly inside an intelligent sensor processing unit (ISPU), the small digital core built into a 6-axis accelerometer-gyroscope chip. The authors report a model that classifies gestures such as writing, cleaning, sports, and kitchen tasks at roughly 85% accuracy while using under 850 bytes of stack memory and drawing about 0.5 mA. They achieve this by resampling raw 104 Hz sensor streams to 26 Hz, cutting them into 1.5-second windows, extracting 78 statistical features, and then growing the classifier one class at a time so overlapping classes are merged or discarded. If the claim holds, activity recognition can happen at the sensor itself, with no data sent to a microcontroller or cloud, which would reduce power, latency, and privacy risk in wearables and smart environments. The paper also releases a 12.5-hour, 24-gesture dataset for further work.","feed_headline":"24-class activity model fits in 850 bytes of sensor memory","feed_subtitle":"On-sensor inference at 0.5 mA keeps data local and makes activity recognition practical in wearables.","key_machinery":"The mechanism that carries the argument is a preprocessing-plus-training pipeline whose load-bearing piece is incremental class injection. Raw 104 Hz six-axis data are downsampled to 26 Hz, windowed at 1.5 seconds, and reduced to 78 statistical features per window (max, min, mean, standard deviation, range, zero-crossing rate, and similar quantities on each axis). Training then starts from a small class set and adds classes one at a time, measuring overlap against already-trained classes and either merging or dropping each candidate class; this is what lets the model reach 24 classes while staying under the 2 kB memory target. The deployed model is a decision-tree ensemble restricted to the 16 most important features, and its stack footprint of about 0.82-0.85 kB is the concrete evidence for the headline claim.","core_discovery":"The central discovery, stated on the paper's own terms, is that a 24-class activity classifier can be deployed on an ISPU within an 850-byte stack memory budget without giving up usable accuracy. The final model is a gradient-boosted decision-tree ensemble (an XGBoost model) trained on only the top 20% most important statistical features, derived from accelerometer and gyroscope windows; this reduces stack usage to 0.82 kB while preserving 84.7% accuracy on the preprocessed data. The authors attribute the scalability to an incremental class injection procedure: adding classes sequentially, checking overlap, and either merging or discarding a new class depending on how confused the current model is. With this procedure the system grew from 12-16 classes to 24, and the released dataset records 24 distinct gestures over 750 minutes with 2-5 participants per class. They also report that the model runs in real time, uses about 0.5 mA, and keeps raw data on the sensor, which is the practical payoff of the whole exercise.","pith_inferences":["Because the feature-importance ranking and the final class selection were computed using the full dataset (Table 1 footnote), the 84.7% figure is likely an optimistic estimate of generalization; re-selecting features inside each training fold would give a cleaner number.","The comparison against the 1D ResNet baseline stacks an unconstrained 1M-parameter network against the memory-limited models; a same-budget or quantized ResNet would be a more direct test of what is lost by the feature-based approach.","The 24-class set itself was chosen after looking at model behavior on the full data, so the 'scaled to 24 classes' claim is about a curated 24-class problem; fixing classes before seeing the data would test whether the injection strategy truly scales to any 24 classes.","One testable extension is to apply the same downsampling-windowing-feature pipeline to other periodic signals, such as ECG or vibration, and measure whether a similarly tiny gradient-boosted classifier can match larger models."],"forward_implications":["A 24-class activity classifier can be embedded directly in the sensor die, so an IMU can output activity labels instead of raw time series, cutting host-MCU load.","Raw acceleration and gyroscope data never have to leave the sensor, strengthening privacy in healthcare and home monitoring.","At about 0.5 mA, always-on activity recognition becomes plausible for coin-cell or energy-harvesting wearables.","The released 24-gesture dataset gives the tinyML community a public benchmark for ultra-constrained HAR.","If the class-injection method transfers, other multi-class sensor tasks such as vibration diagnostics or gesture recognition could fit under 8 kB program and 2 kB data limits."],"supporting_citations":[{"why":"Defines ISPU-class sensors and their memory constraints, the deployment target the whole paper is built around.","marker":"[29]"},{"why":"Demonstrates on-sensor learning and classification under 8 KB memory; the prior result this work extends to 24 classes.","marker":"[11]"},{"why":"Shows an earlier ISPU deployment for fault detection with large power savings, supporting the claimed practical gains.","marker":"[25]"},{"why":"Supplies the downsampling/morphology rationale and the ResNet implementation used as the accuracy baseline.","marker":"[4]"},{"why":"XGBoost is the algorithm of the final deployed model; its memory footprint is the one being measured.","marker":"[10]"},{"why":"LightGBM provides the comparison gradient-boosting model whose accuracy brackets the reported result.","marker":"[20]"},{"why":"UMAP is used to visualize the preprocessed feature clusters, evidence that preprocessing makes small models viable.","marker":"[23]"},{"why":"Documents the target sensor's 8 kB data memory and ISPU core, fixing the hard constraint of 850 bytes.","marker":"[30]"}],"fun_headline_variants":["24-class activity AI fits in 850 bytes on sensor","On-chip HAR: 24 classes, 850-byte stack, 0.5 mA","Sensor runs 24-class HAR in 850 bytes of memory","850-byte stack runs 24-class activity recognition","HAR on sensor: 24 classes, 850-byte memory, 85% acc"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported 85% accuracy reflects how the model will behave on genuinely new recordings, even though feature-importance selection and the final choice of 24 classes were made using the entire dataset, including the evaluation data.","fun_headline_variants_meta":{"raw":{"variants":["24-class activity AI fits in 850 bytes on sensor","On-chip HAR: 24 classes, 850-byte stack, 0.5 mA","Sensor runs 24-class HAR in 850 bytes of memory","850-byte stack runs 24-class activity recognition","HAR on sensor: 24 classes, 850-byte memory, 85% acc"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000577,"raw_usage":{"total_tokens":2728,"prompt_tokens":957,"completion_tokens":1771,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":1674}},"tokens_in":573,"tokens_out":1771,"duration_ms":13269,"temperature":1.0,"reasoning_tokens":1674,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T20:44:38.794477+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same pipeline on the released dataset with classes fixed in advance and feature importance computed separately inside each cross-validation training fold; if the held-out accuracy drops materially below 84.7%, the headline number is inflated by selection on the full data.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the target sensor's 8 kB data memory and ISPU core, fixing the hard constraint of 850 bytes."}],"review_version":1}