{"id":"bb5d549a-32ec-42a4-94a8-6fe19890f95f","arxiv_id":"2607.15395","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multimodal biosignal model plus a feasibility-gated supervisor cuts false rest-to-move triggers in an assistive robot arm, though the ground-truth rest labels are generated from the very signals it classifies.","lead":"This paper builds a system that reads brain, muscle, and eye signals to decide when an assistive robot arm should move, and it tests the decision on a real robot. The headline safety improvement is measured against automatically generated 'rest' labels that come from the same signals the system reads, so the numbers are only as trustworthy as that labeler.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"All false-activation metrics are measured against HSMM-generated REST/ACTION labels that are never validated against independent ground truth; if the labeler is biased, the headline FP/1k REST and HIL false-start numbers do not transfer to real use.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing concern: the HSMM-generated labels are the ground truth for every false-activation and flapping metric, yet they are derived from the same signal modalities the model consumes and are never validated independently. This is not a minor methodological nitpick; it directly controls the abstract's headline numbers and the safety interpretation of Tables IV–VIII. A concrete annotation study would settle it. The threshold-calibration issue is real but secondary—it affects the fairness of baseline comparisons, not the absolute FP claim. Because the reader already returned CONDITIONAL conditional on exactly this kind of evidence, my read does not change the verdict. I am not raising a new objection; I am affirming the reader's central condition with a specific test that would resolve it. The paper has genuine strengths—LOSO evaluation, sensor-dropout scenarios, HIL replay, and honest statements about limitations—but those strengths do not replace independent label validation.","tokens_in":15182,"tokens_out":4308,"duration_ms":45195,"concrete_test":"Select a random subset of subjects (e.g., 6 of 32) or 10% of trials with recorded video/audio. Have two independent annotators, blinded to the HSMM labels, label REST vs ACTION onset/offset using only task instructions, video, and explicit user intent cues; compute inter-annotator agreement and Cohen's kappa between HSMM and human labels. Then recompute FP/1k REST, flaps/min, and commit coverage for NeuroCommitSSM and the strongest baseline (TCN) using the human labels as ground truth on that subset, holding decision thresholds fixed. If NeuroCommitSSM's low FP and the TCN gap persist, the safety claim transfers; if the ordering flips or FP inflates substantially, the reported numbers are labeler-relative.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—reduced REST false activations and state flapping—is evaluated entirely against an HSMM-generated label set. Section III-A states that \"an HSMM-based labeling pipeline generated ACTION/REST supervision labels from EEG, EMG, eye-tracking activity features, and trial-timing constraints,\" and Section III-D.3 says \"Readiness is supervised from onset-relative timing for ACTION windows.\" The same modality features are consumed by NeuroCommitSSM (e.g., EEG bandpower, EMG RMS/MAV/WL, ET salient events), so the labeler and the model can share systematic artifacts. No independent validation of the HSMM labels against human annotation, video review, or user self-report is reported. If the HSMM mislabels pre-movement muscle activity, gaze shifts, or ambient motion as REST, then the headline numbers (0.75 FP/1k REST in S0, 0.29 in S4, HIL false-start reductions in Table VIII) are relative to the labeler, not to user intent. This does not invalidate the engineering contribution, but it means the safety claim is not yet established for deployment. A secondary related gap is that commit thresholds are calibrated on validation for the proposed method, and the text does not state whether each baseline received equivalent threshold calibration; if only NeuroCommitSSM's thresholds were tuned, the large FP gaps in Tables IV–V could be partly a calibration artifact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces NeuroCommitSSM, a decision-centric framework for assistive robotic manipulation that predicts a continuous commit-readiness score from synchronized EEG, EMG, and eye-tracking, converts it to discrete commit events via dwell/hysteresis filtering, and gates execution through a three-state HOLD–ASSIST–COMMIT supervisor that also evaluates perception and robot feasibility. The method is evaluated on a newly collected 32-participant dataset of five ICF-aligned ADL tasks under leave-one-subject-out cross-validation and seven sensor-dropout scenarios, with additional hardware-in-the-loop replay experiments on a Kinova Gen3 arm. The paper claims that the framework reduces REST false activations and state flapping relative to baselines while preserving task success.","tokens_in":15432,"tokens_out":5676,"duration_ms":50912,"significance":"If the reported safety results are valid, the paper makes a useful contribution by modeling when to execute rather than only what to do, integrating multimodal biosignals with physical feasibility gating, and releasing a synchronized EEG–EMG–ET dataset. The evaluation is extensive (LOSO, sensor dropout, ablations, Holm-corrected paired statistics, HIL), and the public release of code, data, and videos is a clear strength. However, the safety claims rest on a label-generation pipeline that is not independently validated, which limits the conclusions that can be drawn from the headline false-activation numbers.","major_comments":[{"comment":"The headline safety metrics (FP/1k REST in Tables IV–V, false starts in Table VIII) are computed against ACTION/REST labels produced by an HSMM labeling pipeline that consumes the same EEG, EMG, and eye-tracking features used as model inputs (Sec. III-A: 'An HSMM-based labeling pipeline generated ACTION/REST supervision labels from EEG, EMG, eye-tracking activity features, and trial-timing constraints'). Readiness supervision is also derived from onset-relative timing within ACTION windows (Sec. III-D.3). No independent validation of the HSMM labels (e.g., human annotation, video review, user self-report, or protocol-based onset times alone) is presented. Consequently, the false-activation numbers measure agreement with the automatic labeler rather than with user intent; if the labeler confuses ambient EMG or gaze shifts with REST, the safety claims do not transfer. I request either inde","section":"Sec. III-A, III-D.3; Tables IV–V, VIII"},{"comment":"The text states that 'Validation calibrates thresholds (τon, τoff, Ndwell, Ncool) to balance time-to-commit against false commits during REST' for the proposed method, and that baselines use the same commit-event extraction, but it does not state whether baseline thresholds were also calibrated on validation (or on a matched operating point). If only NeuroCommitSSM's thresholds were tuned, the large FP differences in Tables IV–V (e.g., TCN in S4: 99.95 vs. 0.29 FP/1k REST) could be partly a calibration artifact. Please specify the calibration protocol for every baseline, including whether thresholds were selected per method on the validation split and whether any baseline was given an equivalent false-commit budget.","section":"Sec. III-D.3, IV; Tables IV–V"},{"comment":"There is a numerical inconsistency that undermines the ablation table. Table VI reports the full 'Base' model averaged over S1–S3 with FP/1k REST = 7.481±5.056 and flaps/min = 12.283±3.295, while Tables IV–V report the same model (Ours) in S1, S2, S3 with FP/1k REST = 1.23, 0.12, 0.17 (mean ~0.51) and flaps/min = 6.96, 3.06, 4.63 (mean ~4.88). These are incompatible. The discrepancy needs to be explained (e.g., different threshold calibration, different evaluation windows, or a reporting error) and the affected numbers corrected. As written, the ablation results call into question the integrity of the headline FP/1k REST figures.","section":"Table VI vs. Tables IV–V"}],"minor_comments":[{"comment":"'MA V' appears to be a typo for 'MAV'.","section":"Sec. III-B"},{"comment":"'Action-balanced accuracy' is used without an explicit definition; please define it (e.g., mean of true-positive rate and true-negative rate) at first occurrence.","section":"Sec. IV"},{"comment":"The calibration section would benefit from a concrete statement of the validation procedure and the resulting threshold values per scenario; currently it is described in one sentence.","section":"Sec. III-D.3"},{"comment":"The Feasibility-only row (100% false starts, 0% CV-infeasible starts) is understandable but deserves a one-sentence explanation in the text to avoid confusion.","section":"Table VIII"},{"comment":"The caveat that HIL replay does not replace live human-in-the-loop testing is welcome; consider also explicitly noting the small held-out subject count (N=2) in the results section.","section":"Sec. III-E / IV"}],"recommendation":"major_revision","confidential_remarks":"The label-validation gap is the central issue: the safety claims are relative to an automatic labeler that uses the same signals as the model, and no independent validation is provided. The ablation inconsistency between Table VI and Tables IV–V is a red flag that must be resolved. If the authors can add independent label validation or substantially temper the safety claims, and correct the numerical inconsistency, I would support publication. The engineering contribution is solid, but the current evidence is insufficient for the safety claims as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a substantial piece of systems engineering. The commit-readiness formulation, the HOLD-ASSIST-COMMIT supervisor that couples neural readiness with real perception/IK/collision checks, and the first synchronized EEG-EMG-ET ADL dataset are real contributions. The evaluation is serious: LOSO, seven dropout scenarios, ablations, and actual HIL replay on a Kinova Gen3 with paired statistics. Give credit where due.\n\nThe soft spot is the one the abstract hides: the ground truth. ACTION/REST labels come from an HSMM that consumes the same EEG/EMG/ET features the model uses, and 'readiness' is supervised from onset-relative timing inside those ACTION windows. The headline metrics — 0.75 FP/1k REST, 0.29 in S4, the HIL false-start rates — are all relative to that labeler, not to independently verified user intent. The paper never validates the HSMM output against human annotation, video, or self-report. That doesn't invalidate the work; the trial-timing constraints probably make many labels correct. But it means the safety claims are not yet established. A reviewer should ask for at least one independent validation pass.\n\nSecond issue: threshold calibration. The text calibrates τ_on/τ_off/N_dwell/N_cool on validation for NeuroCommitSSM. It says baselines use the same commit-event extraction, but never says whether baselines got equivalent threshold tuning. If they didn't, the 99.95 vs 0.29 FP/1k REST gap is partly a calibration artifact. That needs to be stated or fixed.\n\nTwo smaller items: all 32 subjects are healthy adults, so the assistive claim is untested, and the paper's own conclusion acknowledges this. And the safest policy triples time-to-success (16.8s to 46.3s). That may be an acceptable safety tradeoff, but it is a real usability cost.\n\nWho's this for? People building intent-to-execute interfaces, BCI/assistive robotics, and anyone who wants a multimodal ADL dataset. It deserves a serious referee — the engineering and dataset are publishable — but the revision needs the label validation and calibration details. I'd accept the invite.","headline":"Genuinely useful decision-centric pipeline and dataset, but the headline safety numbers are measured against HSMM labels that are never independently validated — treat the false-activation claim as plausible, not proven.","tokens_in":16065,"tokens_out":2006,"would_cite":true,"duration_ms":19951,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Assistive-arm safety can be improved by deciding when to act, not just what to do: this paper proposes NeuroCommitSSM, a framework that learns a continuous commit-readiness score from EEG, EMG, and eye-tracking, converts it to discrete comm","keywords":["shared autonomy","commit readiness","brain-computer interface","EEG-EMG-eye tracking","assistive manipulation","sensor dropout","leave-one-subject-out","hardware-in-the-loop"],"falsifier":"A human-labeling study on a held-out set of the recorded trials: have annotators or the participants themselves mark true REST/ACTION intervals and intended initiation moments, then recompute FP/1k REST and flaps/min against those labels. If false-commit rates rise substantially versus the HSMM-based labels, the central safety claim is an artifact of the labeling pipeline.","tokens_in":14944,"feed_emoji":"🦾","tokens_out":5513,"duration_ms":47207,"temperature":0.7,"pith_summary":"The paper tries to establish that false activations during rest—moments when the user has no intent to act—can be suppressed by modeling the commit decision itself, rather than treating recognition confidence as a trigger. It proposes NeuroCommitSSM, which fuses EEG, EMG, and eye-tracking into a continuous readiness score in [0,1], converts it to discrete commit events via dwell and hysteresis filtering, and passes those events through a HOLD–ASSIST–COMMIT supervisor that also requires target visibility, inverse-kinematics solvability, and collision-free planning. In leave-one-subject-out tests with 32 participants and seven sensor-dropout scenarios, the authors report 0.75 false commits per 1000 rest windows at full sensing and 0.29 in the EEG-only case, versus roughly 100 for a temporal-convolutional baseline; hardware-in-the-loop replay on a robotic arm shows fewer false starts and aborts without sacrificing success. A sympathetic reader would care because this reframes assistive intent interfaces as safety-critical decision systems, with a released dataset and benchmark as supporting infrastructure.","feed_headline":"A 'when to act' score cuts false robot starts from ~100 to <1","feed_subtitle":"EEG-EMG-eye tracking readiness plus feasibility gates keeps task success high even when sensors drop out.","key_machinery":"Central object: the commit-readiness score c_t in [0,1], learned from EEG–EMG–ET and supervised from onset-relative timing in ACTION windows (disabled in REST). The argument turns on how c_t becomes an event: action-gating by the predicted action probability, a two-state hysteresis filter with separate on/off thresholds and a dwell of N consecutive windows, plus reset/cool-down. The second object is the HOLD–ASSIST–COMMIT (HAC) supervisor, which combines the gated readiness with perception and robot-feasibility cues and requires stable feasibility for a dwell period before entering COMMIT, with abort on loss of feasibility. Together they turn a noisy neural signal into conservative, reversib","core_discovery":"Core claim: when-to-act can be treated as a learnable, supervised target, not a by-product of classification confidence. NeuroCommitSSM learns a continuous readiness score from EEG–EMG–ET, action-gates it, and converts it to a discrete commit event via dwell/hysteresis; the HAC supervisor admits motion only if readiness and physical feasibility (target visible, IK solvable, collision-free) both hold. On LOSO data this gives 0.950 action balanced accuracy and 0.75 false commits per 1000 REST windows with all sensors, and 0.29 in EEG-only mode—versus 99.95 for a TCN baseline—while HIL replay on a Gen3 arm shows lower false starts and aborts at equal success.","pith_inferences":["If readiness labels ultimately derive from the same signals the model consumes (as the HSMM onset-relative supervision suggests), the reported false-commit numbers bound errors relative to the auto-labeler, not to independently verified intent; a human-annotated ground-truth study would be the natural next check.","The same decision structure—dwell/hysteresis on a continuous readiness score plus feasibility gating—could transfer to other intent modalities such as speech, gesture, or gaze-only interfaces, where false activation during rest is equally safety-relevant.","An adaptive dwell/hysteresis conditioned on real-time signal quality (the paper lists this as future work) would likely improve the responsiveness-safety trade-off; one concrete test is to lengthen dwell when EEG signal quality drops, not just when a modality disappears entirely.","The HIL replay method isolates the decision layer from user adaptation; the open question is whether live users, including motor-impaired populations, show the same low false-start behavior when they can adjust their behavior in response to the robot's gating."],"forward_implications":["If the reported numbers hold, assistive interfaces can suppress rest-time false commands by two orders of magnitude relative to simple threshold baselines under cross-subject and sensor-dropout conditions.","Feasibility gating is not just a safety add-on: in the paper's HIL ablation, adding perception and robot feasibility raised success from 40.5% (commit-only) to 97.6% (full HAC) while cutting false starts from 11.9% to 2.4%.","Because commit quality is measured as false commits per 1000 REST windows and state toggles per minute, the framework makes decision stability a first-class evaluation metric rather than an afterthought.","The released synchronized EEG–EMG–ET dataset with a balanced-window LOSO protocol gives other groups a standard way to compare intent-to-commit pipelines.","Low false-commit rates are not achieved by suppressing action entirely: commit coverage remains nonzero across all sensor-dropout scenarios, so safety does not come from simply never triggering."],"fun_headline_variants":["When-to-act score cuts false robot starts from 100 to <1","Neural commit gate slashes false robot starts to under 1","Readiness gate drops false starts from 99.95 to 0.29","EEG-EMG-ET readiness gate: 0.29 vs 99.95 false starts"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the HSMM-derived ACTION/REST segments and onset-relative readiness targets reflect the user's true intent; the paper does not validate the labeler against human annotation or self-report, so its headline safety numbers are relative to that auto-labeler.","fun_headline_variants_meta":{"raw":{"variants":["When-to-act score cuts false robot starts from 100 to <1","Neural commit gate slashes false robot starts to under 1","Readiness gate drops false starts from 99.95 to 0.29","EEG-EMG-ET readiness gate: 0.29 vs 99.95 false starts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000536,"raw_usage":{"total_tokens":2494,"prompt_tokens":910,"completion_tokens":1584,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":1497}},"tokens_in":654,"tokens_out":1584,"duration_ms":13757,"temperature":1.0,"reasoning_tokens":1497,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T23:29:20.552940+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A human-labeling study on a held-out set of the recorded trials: have annotators or the participants themselves mark true REST/ACTION intervals and intended initiation moments, then recompute FP/1k REST and flaps/min against those labels. If false-commit rates rise substantially versus the HSMM-based labels, the central safety claim is an artifact of the labeling pipeline.","supporting_citations":[],"review_version":1}