{"id":"502f723d-5953-476e-88e0-77734e55d193","arxiv_id":"1908.06134","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An RL agent on a wearable selects low- or high-power features online; extending the Advise algorithm to multiple, reliability-estimated feedback sources improves the power/accuracy trade-off on the SPHERE dataset.","lead":"This paper proposes a reinforcement learning method that lets a wearable device choose which sensor features to compute, balancing battery use against activity recognition accuracy. It adds a way to use feedback from multiple room sensors, and estimates how trustworthy each feedback source is on the fly.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim of a superior power/accuracy trade-off rests on an underspecified single-run evaluation: no train/test split, no number of seeds, and no episode-generation protocol are reported, so the 'only approach' conclusion is not yet established.","rationale":"I read the paper in good faith. The algorithmic contribution is a reasonable extension of Advise to multiple trainers with online consistency estimation, and the EM derivation in Section 3.2.1 is broadly consistent with the described generative model. No internal contradiction in the method itself is apparent. However, the paper's strongest claim is the empirical demonstration of a superior power/accuracy trade-off. For that claim, the most load-bearing condition is that the evaluation is statistically valid and reproducible. That condition is not met: Section 4 does not state how many seeds were used, whether curves are averaged or single runs, how classifiers were trained, or how training and test data were separated. The 20,000-episode learning curves also imply an episode-generation mechanism that is never specified for a dataset of only ten participants. The reader's identified weakness about corrupted trainers is real and secondary: the conclusion asserts robustness to unreliable feedback without testing it, and a corrupted-trainer ablation would be needed to support the online consistency contribution. But even if the consistency estimator worked perfectly, the headline comparison to baselines would still be unverifiable without proper statistical reporting. Therefore the verdict remains CONDITIONAL, and my analysis does not move it; the manuscript needs the missing experiments and reproducibility details before the central claim can be accepted.","tokens_in":7744,"tokens_out":8064,"duration_ms":85783,"concrete_test":"Re-run the full protocol on the SPHERE dataset with a leave-one-participant-out split, 10 independent seeds per condition, and a documented episode-generation procedure (for example, sliding windows with non-overlapping training and evaluation segments). Report the mean and 95% confidence interval for total reward, error rate, and power consumption for Multi-Trainers, Q-Learning, Random, and Fixed Low. The central claim stands only if Multi-Trainers' reward distribution dominates the other methods across seeds and on held-out participants; additionally, a corrupted-trainer ablation (for example, flipping the PIR trainer's feedback with probability 0.4) should be run to test the consistency estimator directly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: Section 4.2 concludes that Multi-Trainers is the only method that maintains low error while reducing power. But the evaluation section does not report error bars, the number of independent runs, how the low/high feature classifiers were trained, or how the data were split between training and evaluation. The episodic setup described in Section 4.1 (20-minute episodes, 5-second steps) combined with learning curves extending to 20,000 episodes is difficult to reconcile with the stated dataset size of ten scripted participants unless episodes reuse data; the reuse or resampling procedure is not described. Without this information, the observed advantage over Q-Learning and the baselines could be within-run noise or an artifact of in-sample optimization, and the comparison cannot be reproduced. The paper additionally claims robustness to unreliable feedback, but no experiment with deliberately corrupted trainers is run, so the online consistency estimator's core benefit is not isolated. Both issues bear on the headline result, but the missing statistical grounding is the more load-bearing problem for the strongest claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an RL-based online feature-selection method for wearable activity recognition in which an on-wearable agent chooses between a low-power and a high-power feature set at each decision step. It extends the Advise algorithm with two mechanisms: online EM-based estimation of each trainer's consistency level and combination of multiple feedback sources (PIR and RGB-D classifiers). The method is evaluated on the SPHERE challenge dataset against Random, Fixed Low, and Q-Learning baselines. The authors conclude that the proposed Multi-Trainers method is the only approach that maintains a low error rate while reducing power consumption.","tokens_in":7974,"tokens_out":13468,"duration_ms":135554,"significance":"The proposed extension is relevant to embedded activity recognition: it attacks a real power/accuracy trade-off, builds on a public dataset, and provides explicit pseudocode for the online consistency estimator and the adaptive learning rate. The EM derivation in Eqs. (4)-(7) is internally consistent, and the reward function is defined explicitly, which are strengths for reproducibility. If the empirical claim were established, the method would be a useful lightweight alternative to deep RL approaches. However, the current evaluation is not statistically grounded: single learning curves without error bars, no train/test split, no data-reuse protocol, and no ablation for unreliable trainers. The headline conclusion is therefore plausible but not yet supported. In my view the paper is publishable after a major revision that supplies a rigorous evaluation protocol and additional experiments.","major_comments":[{"comment":"The central claim that Multi-Trainers is the only method that maintains error rate while reducing power is based on single learning curves with no error bars, no number of independent runs, no significance tests, and no numerical summary table. It is therefore impossible to tell whether the advantage over Q-Learning and the baselines is larger than within-run variance. Please run multiple random seeds or multiple agent/classifier initializations, report confidence intervals or interquartile ranges over episodes, state whether the curves show training performance or held-out evaluation, and provide a clear train/test split protocol.","section":"Section 4.2, Figures 2-5"},{"comment":"The episodic protocol is not reconciled with the dataset. An episode is 20 minutes with 5-second steps, i.e., 240 steps, and the learning curves extend to 20,000 episodes, which is 4.8 million decisions. The SPHERE subset used here is ten scripted participants, which cannot provide that many unique transitions unless episodes are generated by replay or resampling; the paper does not describe the episode-generation procedure, whether agents are trained per participant or across all participants, or how the low/high feature classifiers used in the state and in feedback were trained. Without this information, the learning curves could reflect in-sample optimization and the experiments are not reproducible. Please specify the data-reuse policy, the participant/classifier split, and the number of unique episodes.","section":"Section 4.1 and 4.2"},{"comment":"The conclusion states that the method was robust to unreliable feedback, but no experiment with deliberately corrupted trainers is reported. The feedback in Section 4.1 is defined as agreement between the wearable classifier and the host PIR/RGB-D classifiers, not agreement with ground-truth labels, so the assumed link between the trainers' consistency and the latent optimal action in Eq. (7) is not tested. Moreover, the use of the agent's own policy as the prior P_Q1 in Eq. (7) makes the consistency estimate self-referential, and there is no comparison to a single-trainer Advise variant with fixed C, so the online consistency estimator's contribution is not isolated. To support the robustness claim, please include an ablation that corrupts one or both trainers (e.g., random or inverted feedback) and separately ablates the online consistency estimator.","section":"Section 4.1 and Section 5"}],"minor_comments":[{"comment":"The title contains a typo: 'FEA TURE' should be 'FEATURE'.","section":"Title"},{"comment":"The same symbol C is used for the state-action consistency estimate and the recursively averaged consistency; please use distinct symbols (e.g., C_s,a and C_bar) to avoid confusion.","section":"Algorithm 2 and Eq. (8)"},{"comment":"The Require line of Algorithm 2 is incomplete in the manuscript ('C, ˜Q and ˜H persistent variables (C initialized 0.5, ˜Q and'); the pseudocode should be completed.","section":"Algorithm 2"},{"comment":"In Eqs. (10)-(13), Q(s) is a sum of absolute Q-values and H(s) is a feedback count, but the text calls these 'accuracy' values; this is misleading. Please describe them as heuristic confidence measures and state explicitly that the adaptive learning rate is a heuristic.","section":"Section 3.2.1, Eqs. (10)-(13)"},{"comment":"Equation (14) should state the conditional-independence assumption under which multiplying the trainers' policies is Bayes-optimal, and should mention the prior assumptions implicit in combining probabilities this way.","section":"Section 3.2.2, Eq. (14)"},{"comment":"The reward parameters lambda = 1.0 and Ptgt = 16.7mC are fixed without sensitivity analysis; since lambda defines the trade-off between error and power, please add a short sensitivity study or justify the choice.","section":"Section 4.1, Eq. (15)"},{"comment":"The power consumption numbers for the low and high feature sets are not sourced in the paper; please cite the power model or describe how the mC values were obtained. Also state whether the RL agent's own computation and EM updates are included in the power budget or assumed negligible.","section":"Section 4.1"},{"comment":"The explanation that Q-Learning fails because 'the error rate results have higher variance compared to the averaged error rate difference' is not backed by any variance analysis; please present it as a hypothesis or support it with data.","section":"Section 5"},{"comment":"There is a duplicated word in Section 2: 'the transmission energy cost of the raw data is is expensive' should be corrected.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is real and worth taking seriously. The authors extend the Advise algorithm in two concrete ways: supporting multiple feedback sources and estimating each trainer's consistency online via an EM procedure with an adaptive learning rate. That is a legitimate algorithmic contribution, not a re-labeling, and the EM derivation in Eqs. 4-7 is internally plausible. The application to on-wearable feature selection, with the goal of balancing power and accuracy in activity recognition, is also a sensible and practically motivated problem. I agree with the reader that the novelty is genuine and the central algorithmic mechanism holds up on inspection.\n\nThe soft spot is the empirical evaluation, and it is load-bearing. The paper reports single learning curves with no error bars, no multiple seeds, no significance tests, and no explicit train/test split. More troubling, the episodic setup is hard to reconcile with the dataset: 20-minute episodes at 5-second steps over 20,000 episodes means millions of decisions, but the SPHERE challenge dataset has only ten scripted participants. The paper never describes how episodes are generated or how data are reused, so the 'only approach' conclusion could easily be within-run noise or in-sample optimization. I do not think the stress-test note is wrong here; these omissions are real and they directly affect the strongest claim.\n\nA second, smaller issue is the robustness claim. The paper says the method is robust to unreliable feedback, but no experiment with deliberately corrupted trainers is run, so the online consistency estimator's main benefit is not isolated. That is a minor-to-moderate gap, not fatal, but it should be flagged.\n\nThe self-referential coupling in the consistency estimator (using the agent's own policy as the EM prior) is a mild concern, but I do not see it as a circularity problem because the comparison against external baselines and a public dataset would still carry weight if the evaluation were properly grounded.\n\nWho is this for? Researchers working on RL for resource-constrained wearable systems, and anyone building on Advise-style policy shaping. The paper deserves a serious referee because the algorithmic idea is sound and the problem is practical, but the empirical section needs major revision before the claim can be accepted. I would send it to review with a strong request for repeated runs, error bars, a clear episode-generation protocol, and an explicit train/test split.","headline":"A genuine algorithmic extension to Advise (multi-trainer, online consistency) is undercut by an evaluation that is statistically underspecified, so the headline claim is plausible but not yet established.","tokens_in":8486,"tokens_out":1493,"would_cite":false,"duration_ms":16447,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A wearable RL agent learns which features to compute and cuts power use without raising error rate.","keywords":["reinforcement learning","feature selection","activity recognition","wearable sensors","power consumption","multiple feedback","policy shaping","SPHERE dataset"],"falsifier":"Feed one trainer deliberately inverted or random feedback during training while keeping the other trainer honest; if the estimated consistency for the corrupted trainer does not drop and the policy is not correspondingly down-weighted, the claimed robustness to unreliable feedback is false. The paper reports no such experiment.","tokens_in":7567,"feed_emoji":"⌚","tokens_out":6994,"duration_ms":62517,"temperature":0.7,"pith_summary":"This paper sets out to show that a wrist-worn activity-recognition device can learn, while in use, which of two feature sets to compute and transmit: a cheap low-power set or a more expensive high-power set. The learning agent uses reinforcement learning and receives feedback from two ambient sensors in the home, and the algorithm also estimates online how reliable each feedback source is. On the SPHERE challenge dataset, the proposed multi-trainer method is reported to be the only approach among those tested that keeps the activity-recognition error rate low while reducing power consumption. The payoff, if the result holds, is that wearables in smart-home health monitoring could extend battery life by adapting their computation to context rather than using a fixed feature set.","feed_headline":"Reinforcement learning cuts wearable power use without raising errors","feed_subtitle":"On the SPHERE smart-home dataset, the multi-trainer method beat fixed, random, and plain Q-learning on the power/accuracy trade-off.","key_machinery":"The load-bearing mechanism is the consistency-weighted policy-shaping rule inherited from the Advise algorithm. Each trainer $n$ contributes a factor $\\left(C^{[n]}\\right)^{\\Delta^{[n]}(s,a)}$, where $C^{[n]}$ is that trainer's estimated probability of giving correct feedback and $\\Delta^{[n]}(s,a)$ is the accumulated difference between positive and negative feedback for state $s$ and action $a$; the factors are multiplied across trainers and combined proportionally with the Q-learning policy $\\pi_R(s,a)$. The consistency levels are estimated by an expectation-maximization procedure with a hidden variable indicating whether the action is optimal, then averaged over state-action pairs with an adaptive learning rate. This machinery lets unreliable feedback be down-weighted automatically and lets the agent treat the host's ambient sensors as teachers rather than requiring a known reliability value.","core_discovery":"The paper's central claim is that incorporating multiple sources of feedback into the on-device reinforcement-learning agent, together with an online estimate of each source's consistency, yields a feature-selection policy that achieves a better power/accuracy trade-off than the alternatives. The agent's state is a discrete tuple of elapsed time, current power consumption, and the output of the low-energy classifier, and its action selects either the low-energy feature group or the high-energy feature group. Feedback from PIR and RGB-D classifiers on the host is shaped into a trainer policy using the Advise method, extended so that the consistency level of each trainer is estimated on the fly rather than set beforehand. The resulting policy combines the trainers' product-of-consistency terms with the underlying Q-learning policy, and the reported learning curves show faster learning and a higher final reward than plain Q-learning, with lower power than random selection and lower error than fixed low-power features.","pith_inferences":["A natural robustness test the paper does not run is to corrupt one trainer's feedback deliberately; if the consistency estimator works as intended, the corrupted trainer's influence should shrink and the final trade-off should stay close to the uncorrupted case.","The two-action split into low- and high-energy feature groups is a coarse discretization; the same machinery could be applied to more feature subsets, at the cost of a larger action space and slower convergence.","The feedback signal relies on agreement between the wearable classifier and the host classifiers, so the method is most portable to settings where an independent, reasonably accurate reference exists; in a home without PIR/RGB-D infrastructure, a single on-device classifier would have to supply its own feedback."],"forward_implications":["Activity-recognition wearables can run a lightweight RL agent that switches between cheap and expensive feature sets, so power savings do not require giving up recognition accuracy.","The online consistency estimator removes the need to hand-tune the trust placed in each feedback source; the agent learns that the RGB-D trainer is slightly more consistent than the PIR trainer.","The discrete state representation and simple Q-learning are light enough for a low-power microcontroller, unlike deep-RL feature-selection approaches.","The reward function, which quadratically penalizes power relative to a target and linearly penalizes error, gives system designers a direct knob for trading battery life against accuracy.","Because the method is evaluated per episode with a fixed target power, the learned policy can be re-tuned when the battery budget or activity set changes."],"supporting_citations":[{"why":"Supplies the activity-recognition dataset used for evaluation, including accelerometer, PIR, and RGB-D data.","marker":"[1]"},{"why":"Demonstrates the energy savings of on-board feature extraction on the same wearable platform, motivating the feature-group power costs.","marker":"[5]"},{"why":"Provides the Advise algorithm, whose single-trainer feedback shaping this work extends to multiple trainers.","marker":"[6]"},{"why":"Provides the EM algorithm used to estimate each trainer's consistency level.","marker":"[12]"},{"why":"Justifies multiplying probabilities from independent sources to combine multiple trainers.","marker":"[13]"},{"why":"Gives the Q-learning algorithm used as the underlying RL method and baseline.","marker":"[14]"}],"fun_headline_variants":["Multi-feedback RL balances wearable power and accuracy","RL feature selection trims wearable power, keeps accuracy","On-device RL picks features to cut power, not accuracy","Reinforcement learning tunes wearable sensors for power"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that agreement between the wearable's classifier and the host's PIR/RGB-D classifiers indicates whether the chosen feature set was the right action, and that the online consistency estimator will down-weight feedback when this proxy is wrong.","fun_headline_variants_meta":{"raw":{"variants":["Multi-feedback RL balances wearable power and accuracy","RL feature selection trims wearable power, keeps accuracy","On-device RL picks features to cut power, not accuracy","Reinforcement learning tunes wearable sensors for power"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1273,"prompt_tokens":884,"completion_tokens":389,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":326}},"tokens_in":500,"tokens_out":389,"duration_ms":3634,"temperature":1.0,"reasoning_tokens":326,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:54:52.905510+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed one trainer deliberately inverted or random feedback during training while keeping the other trainer honest; if the estimated consistency for the corrupted trainer does not drop and the policy is not correspondingly down-weighted, the claimed robustness to unreliable feedback is false. The paper reports no such experiment.","supporting_citations":[{"cited_title":"This is evidenced by the vast amount of studies undertaken utiliz- ing accelerometers [2, 3], perhaps the most commonly used device for detecting ADL","cited_arxiv_id":null,"evidence_quote":"Supplies the activity-recognition dataset used for evaluation, including accelerometer, PIR, and RGB-D data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates the energy savings of on-board feature extraction on the same wearable platform, motivating the feature-group power costs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Advise algorithm, whose single-trainer feedback shaping this work extends to multiple trainers."},{"cited_title":"The SPHERE Challenge: Activity Recognition with Multimodal Sensor Data","cited_arxiv_id":"1603.00797","evidence_quote":"Provides the EM algorithm used to estimate each trainer's consistency level."},{"cited_title":"The consistency level estima- tion uses two sources, information from the underlying RL algorithm (P Q 1 andP Q 0 ) and the trainer’s feedback (h+ s,a and h− s,a)","cited_arxiv_id":null,"evidence_quote":"Justifies multiplying probabilities from independent sources to combine multiple trainers."},{"cited_title":"Large scale population assessment of physical activity using wrist worn accelerometers: The uk biobank study,","cited_arxiv_id":null,"evidence_quote":"Gives the Q-learning algorithm used as the underlying RL method and baseline."}],"review_version":1}