{"id":"d2552f24-8dfb-4659-b32d-35ff6be9f8e9","arxiv_id":"2502.04398","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A visual analytics tool combines an interval-forest classifier with linked plots so users can predict a reach-to-grasp target early and see which hand-aperture features drive the prediction.","lead":"This paper presents XMTC, a visual analytics tool that combines a time-series ensemble classifier with linked accuracy, confusion-matrix, heatmap, and partial-dependence plots to predict which object a person is reaching for as early as possible. It applies the tool to motion-capture data from reach-to-grasp experiments and reports that the intended object can often be predicted before the hand actually grasps it.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Random-trial split leaks per-user kinematics; the paper's own leave-one-out test on Experiment 1 shows a 10.5-point drop at the recommended window, and no user-independent test is reported for Experiment 3's before-grasp accuracy.","rationale":"The reader and I converge on the same bottleneck. The paper's contribution is primarily an integrated tool plus a case study; the quantitative early-prediction claim is the evidence that the tool's recommended trade-off point is meaningful. That evidence is generated by a random 80/20 trial split. Since each of 29 participants contributes multiple trials and hand kinematics are strongly user-specific, this split can inflate accuracy by letting the model recognize the user rather than the intended object. The paper's own leave-one-out experiment on Experiment 1 is direct evidence: at the recommended window (380), accuracy drops from 71.09% to 60.62%, a 10.5-point decrease, and the median is 62.5%. The fact that the same protocol was not run on Experiment 3, the experiment behind the headline 'before grasp' numbers, makes the central claim unverified under the deployment-relevant protocol. I do not think this invalidates the XMTC tool; the visual analytics contributions and the qualitative confusion-matrix and PDP analyses stand independently. It does mean the central empirical claim should remain conditional until user-independent results for Experiment 3 are available. Minor inconsistencies (e.g., Table 2 counts not matching accuracies for l_cup) and the unspecified stretching procedure are additional robustness issues but are secondary to the evaluation-scheme concern.","tokens_in":17359,"tokens_out":7406,"duration_ms":74886,"concrete_test":"Run a leave-one-participant-out cross-validation on Experiment 3: for each of the 29 participants, train DrCIF on all trials from the other 28 participants for every window size (or at least windows 360 and 640), using the same feature normalization and stretching procedure from Section 4, and evaluate on all trials of the held-out participant. Report mean and standard deviation of accuracy across participants at each window, plus per-class recall. Compare with the random-split values in Figure 12 (73.02% at w=360, 91.01% at w=640). If the mean user-independent accuracy at w=360 is not materially above chance (12.5% for 8 classes) or is substantially below the random-split number, the early-prediction claim as stated is not supported. Repeating this protocol for all windows would also reveal whether the recommended trade-off window shifts under user-independent evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the evaluation protocol. The paper's quantitative claim of good early prediction before grasp rests on accuracy numbers from a random 80/20 split across trials (Sections 6.1 and 6.3). Because the data contain repeated trials per participant (29 adults, two trials each), a random trial-level split places the same user's motion style in both training and test, and DrCIF can exploit user-specific kinematics. For deployment, the intended use is predicting an unseen user's intention, so a user-independent split is required. The paper's own Section 6.4 leave-one-out test on Experiment 1 confirms the risk: at the recommended window 380, mean accuracy drops from 71.09% to 60.62% with sigma 10.19%, and median is 62.5%. Crucially, no equivalent leave-one-user-out test is reported for Experiment 3, the case study that supports the headline 73.02% at window 360 and 91.01% at window 640. Thus the central 'before grasp' accuracy for Experiment 3 is not shown to generalize to new users. Section 6.4 calls this a 'slight performance drop,' but a 10.5-point drop is material; moreover, the recommended window was selected from the random-split test curve, so the comparison may be optimistic. No error bars or repeated random splits are given for the main accuracy curves. The early-prediction claim should be conditional on a user-independent evaluation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents XMTC, an interactive visual analytics tool for early classification of multivariate time series, applied to reach-to-grasp hand kinematics. The approach trains DrCIF models on prefixes of increasing length (moving window with step 10) and provides coordinated views—a temporal accuracy plot, confusion matrix heatmaps, per-series prediction heatmaps, and partial dependence plots—to let analysts choose an early-vs-accurate trade-off and identify difficult classes and influential features. Three real motion-capture experiments are analyzed; in Experiment 3 the authors report 73.02% accuracy at window size 360 (before any object is grasped) and 91.01% at window size 640, with qualitative explanations of confusable classes and feature impacts. A leave-one-user-out test on Experiment 1 is reported as a generalizability check.","tokens_in":17650,"tokens_out":4596,"duration_ms":44062,"significance":"If the accuracy claims held under deployment conditions, the paper would make a useful contribution: it packages an existing strong MTSC classifier (DrCIF) into a visually guided early-prediction workflow, with post-hoc, model-agnostic explanations that transfer to other classifiers. Strengths include real high-frequency motion data, multiple controlled experiments, and the explicit attempt to measure cross-user generalization in Section 6.4. The main methodological question is whether the headline before-grasp accuracies, obtained from a random trial-level split, reflect performance for unseen users; the paper's own leave-one-out numbers show a substantial drop, so the generalizability claim is currently not established for the central case study.","major_comments":[{"comment":"The evaluation protocol is the load-bearing issue. Section 6.1 states that training and testing data are 'randomly selected and stratified'; with 29 participants and two trials each, a trial-level split places the same user's kinematics in both training and test sets, and DrCIF can exploit per-user motion style rather than only object-intention signals. The recommended window-380 model for Experiment 1 achieves 71.09% on this split, but the leave-one-out test in Section 6.4 reports 60.62% (sigma 10.19%, median 62.5%). This 10.5-point drop is material and should not be called 'slight' as in Section 7. The same issue affects Experiment 3, where the headline 73.02% at window 360 and 91.01% at window 640 are reported only on the random split; no leave-one-user-out or other user-independent evaluation is reported for that experiment. The central claim that XMTC can predict the intended object before grasp for unseen users therefore requires a user-independent evaluation for Experiment 3 (and ideally for Experiments 1 and 2), with the recommended trade-off window selected from that evaluation rather than from the random-split test curve.","section":"Sections 6.1, 6.3, 6.4"},{"comment":"The paper claims that DrCIF received the highest weight among the four HIVE-COTE2 classifiers and that using DrCIF alone yields 'the same accuracy' as HIVE-COTE2, but no accuracy comparison is presented. Since DrCIF is the classifier on which all reported results depend, this equivalence is load-bearing. Please provide HIVE-COTE2 versus DrCIF test accuracies for a representative set of window sizes (including the recommended windows in Section 6), or state precisely the criterion used to conclude 'without loss of accuracy'.","section":"Sections 2 and 4"},{"comment":"The stretching of short time series to the required window length is specified only as 'we stretch too short time series accordingly for the training phase' (Section 4). The interpolation or padding method is not described, and it is not stated whether the same stretching is applied to test series and in the leave-one-out test. This matters for early-prediction validity, since the procedure must not introduce information from the future or artificially distort the interval features that DrCIF extracts. Please specify the method and justify that it preserves the early-prediction semantics.","section":"Section 4"},{"comment":"Table 2 contains an impossible count: the l cup row reports a total of 24 test samples but 25 correct predictions for Model 1436. This suggests either a data error or inconsistent test sets across window sizes. Since the accuracy tables are the quantitative basis for the findings, the inconsistency must be resolved and the tables checked for similar issues.","section":"Table 2"}],"minor_comments":[{"comment":"There is a typo, 'classsification', in the final paragraph of Section 7.","section":"Section 7"},{"comment":"The text uses 'Shapely values' but the correct term is 'Shapley values'; also 'Partial Dependency Plots' should be 'Partial Dependence Plots' for consistency with Section 5.","section":"Section 2"},{"comment":"The histograms in the accuracy plots have no y-axis label; adding a label such as 'number of time series' would improve interpretability.","section":"Figures 6 and 12"},{"comment":"The text refers to 'time step' when describing the x-axis of the leave-one-out plots, but the x-axis is the window size; please unify the terminology.","section":"Figures 16 and 17"},{"comment":"The phrase 'two random time series from the class l dice' should read 'two randomly selected time series'.","section":"Section 6.3"},{"comment":"The text first identifies 'r cup and r bottle' as the most difficult classes (based on Table 2) but later says 'r cup and r knife were identified as the most challenging classes'; please reconcile this inconsistency.","section":"Section 6.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's novelty is primarily the visualization workflow and the empirical case study; the classifier itself is existing. The journal fit is reasonable for a visualization or HCI venue, but the evaluation protocol must be corrected before acceptance. The HIVE-COTE2/DrCIF equivalence claim should also be supported by concrete numbers. The authors appear to engage with related work, and the inclusion of a leave-one-user-out test for Experiment 1 is a positive sign; extending it to the decisive Experiment 3 is necessary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: XMTC is a well-built visual analytics tool for early classification of multivariate time series, and the reach-to-grasp case study is a sensible application. But the central quantitative claim—73% accuracy before grasp in Experiment 3—is not yet supported for unseen users. The evaluation uses a random 80/20 split across trials, and because each of the 29 participants contributed multiple trials, the same user's kinematics appear in both training and test. The paper's own leave-one-out test in Section 6.4 shows the cost: at the recommended window 380, mean accuracy drops from 71.09% to 60.62% (σ=10.19%), median 62.5%. That is a material drop, not the \"slight performance drop\" the Discussion calls it. And no equivalent user-independent test is reported for Experiment 3, the source of the headline \"before grasp\" accuracy.\n\nWhat's genuinely new is the integrated tool. The components—DrCIF, PDPs, confusion matrices, the Detoxer-style heatmap—are all published, but XMTC coordinates them around the earliness/accuracy trade-off, and the task analysis with a domain expert shows real design thought. The case studies illustrate how the views support finding confusing classes and impactful features; Experiment 2's negative result is reported honestly. The paper also deserves credit for running a leave-one-out check at all, even if it underweights the result.\n\nSoft spots, in order: (1) The evaluation protocol for all three experiments is the random trial-level split; Section 6.1 says \"randomly selected and stratified\" with no user grouping. Without a user-independent split, the accuracy curves are optimistically biased. (2) No error bars or repeated splits are given for the main accuracy plots, so the reader cannot judge stability. (3) Section 4 says short series are \"stretched accordingly,\" but the stretching procedure is never specified. (4) The claim that DrCIF matches HIVE-COTE2 accuracy is supported only by a weights plot, not an accuracy comparison. (5) The recommended window is picked from the test accuracy curve, a form of test-set tuning that makes the leave-one-out comparison optimistic.\n\nWho gets value: visual analytics and HCI researchers working on early time series prediction, and anyone building predictive interfaces for grasp or gesture. It deserves a serious referee, but the early-prediction claim should be accepted only conditionally, after a user-independent evaluation is added for all experiments, error bars are provided, and the stretching and DrCIF/HIVE-COTE2 comparison are pinned down.\n\nMy take: send it to peer review, but a skeptical reviewer will (rightly) push for the user-independent numbers before believing the headline.","headline":"A useful visual analytics tool, but the before-grasp accuracy claim needs a user-independent evaluation before it can be believed.","tokens_in":18193,"tokens_out":2728,"would_cite":false,"duration_ms":25632,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"XMTC claims that a reach-to-grasp target can be predicted from hand kinematics before the hand touches the object, with an interval-forest classifier reaching 73% accuracy at a 360-step window and a visualization tool exposing why…","keywords":["early time series classification","multivariate time series","reach-to-grasp","hand kinematics","DrCIF","explainable AI","visual analytics","object prediction"],"falsifier":"Run the same windowed DrCIF training on Experiment 3 with a strict user-exclusive split, holding out all trials of a set of participants rather than random trials, and read the accuracy at window 360 and window 640; if before-touch accuracy falls to chance level or well below the leave-one-out mean, the central early-prediction claim does not transfer to unseen users.","tokens_in":17140,"feed_emoji":"🖐️","tokens_out":10366,"duration_ms":98684,"temperature":0.7,"pith_summary":"The paper asks whether a person's intended grasp target can be read from hand motion early enough to be useful, and it answers yes for controlled reach-to-grasp recordings. In the strongest scenario, the classifier reaches 73.02% accuracy at time step 360, before any tested participant has touched the object, and 91.01% at time step 640. The same analysis shows when classes become separable, which objects remain confused, and which finger-aperture features carry the signal. The authors wrap the classifier in XMTC, a visual tool that connects accuracy-over-window plots, confusion heatmaps, per-series prediction heatmaps, and partial dependence plots, so a designer can choose an earliness/accuracy trade-off with evidence in view.","feed_headline":"Grasp target readable from hand motion before touch — 73% accurate","feed_subtitle":"At 360 time steps into the reach, the classifier names the object; the tool shows which finger gaps matter.","key_machinery":"The load-bearing classifier is DrCIF (Diverse Representation Canonical Interval Forest), an ensemble of 200 decision trees that draw random intervals from three representations of the series, namely the original series, first-order differences, and periodograms, and compute 29 statistical features including the catch22 set over each interval, with random dimension and feature subset selection per tree and majority voting at the end. Around this classifier, the paper builds a moving-window protocol: one model per prefix $[0,10], [0,20], \\ldots$ of the time series, so the accuracy curve itself becomes the earliness/accuracy axis. XMTC then maps each analysis task to a coordinated view, using the accuracy plot with series-length histograms, confusion-matrix heatmaps, a per-series class-probability heatmap, and partial dependence plots, so global accuracy, confusion evolution, individual trajectories, and feature impact are inspectable from the same trained models.","core_discovery":"On its own terms, the paper establishes that early object prediction from reach-to-grasp kinematics is not only possible but diagnosable, where the kinematic input is a 12-dimensional time series of aperture vectors: the x, y, and z components of the thumb-to-index, thumb-to-middle, thumb-to-ring, and thumb-to-little-finger distances. The intended object among eight condition classes is predicted from the first $i\\cdot 10$ time steps of this time series, with short series stretched to a common window length for training. Experiment 3 improves from 29.97% at window 10 to 73.02% at window 360, the last step before any object is grasped, and to 91.01% by window 640, stabilizing near 93.19% at the full length. Experiment 1 reaches about 71% at the recommended window 380 and 85.4% at the full 1436-step window, while Experiment 2 stalls near 60%, which the authors attribute to participants not attending to object identity. The explanations identify persistent confusions between similar objects and single out, for example, the x-components of thumb-middle and thumb-ring apertures as the features that lift one class from 74% to 100% accuracy.","pith_inferences":["The authors leave implicit that deployment accuracy for a new user is the user-independent number: their own leave-one-out test drops the recommended Experiment 1 model from 71.09% to a mean of 60.62%, so an actual interface should be evaluated by held-out users, not random trials.","At the 960 Hz motion-capture rate, the 360-step window is 0.375 seconds into the reach; if the before-touch accuracy transfers, a system could begin acting about a third of a second before contact, which is enough to mask typical interaction latency.","A direct extension is to ablate the x-components of thumb-middle and thumb-ring apertures: the partial dependence analysis implies they carry the decisive early signal for at least one class, so a reduced sensor setup might preserve much of the accuracy.","The tool's accuracy-over-window plot is most informative when series lengths vary; in fixed-length multivariate time series, the early axis would need a different operationalization, such as the fraction of the series observed."],"forward_implications":["In reach-to-grasp interfaces, the intended object can be named before the hand touches it, at 73% accuracy under the paper's evaluation in Experiment 3.","A tool user can choose the operating point explicitly: window 360 for maximal earliness, or window 640 for the steady state near 91% accuracy.","Persistent confusions are structural, not random: similar objects such as bottle and cup or knife and pen are the last to separate, so a system can treat those pairs as expected ambiguities.","Feature-impact analysis points to specific aperture components, such as the x-components of thumb-middle and thumb-ring apertures, as the discriminative signal, which can guide sensor choice.","Because the explanation views are global, post-hoc, and model-agnostic, the same visual analysis can be attached to a different classifier without changing the tool's logic."],"supporting_citations":[{"why":"Defines HIVE-COTE 2.0 and DrCIF, the interval-based ensemble that the paper identifies as its dominant classifier and uses throughout.","marker":"[11]"},{"why":"The earlier reach-prediction study using finger motion dynamics whose failure to discriminate synthetic objects motivates the current approach.","marker":"[29]"},{"why":"Categorizes multivariate time series classification algorithms and motivates the meta-ensemble design that the paper reduces to DrCIF.","marker":"[38]"},{"why":"Introduces partial dependence plots, the method XMTC uses to show which features drive each class decision.","marker":"[39]"},{"why":"Supplies the heatmap style for per-time-series prediction evolution that XMTC adapts for individual trajectory inspection.","marker":"[41]"},{"why":"catch22; its canonical time-series features expand DrCIF's candidate feature pool and therefore define the statistics the trees are built on.","marker":"[42]"}],"fun_headline_variants":["Hand motion predicts grasp target before contact, 73% early","Explainable AI reveals which finger gaps signal grasp intent early","Grasp object predicted from hand motion at 73% before touch","Early grasp recognition via hand kinematics: 73% at 360 steps","XMTC: explainable early classification of grasp intent from hand motion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that accuracy measured on a random 20% holdout of trials predicts how the model will do on a person whose movement style was never in training; the paper's own leave-one-out test shows that switching to held-out users drops the recommended Experiment 1 model from 71.09% to an average of 60.62% with a standard deviation near 10 percentage points.","fun_headline_variants_meta":{"raw":{"variants":["Hand motion predicts grasp target before contact, 73% early","Explainable AI reveals which finger gaps signal grasp intent early","Grasp object predicted from hand motion at 73% before touch","Early grasp recognition via hand kinematics: 73% at 360 steps","XMTC: explainable early classification of grasp intent from hand motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000528,"raw_usage":{"total_tokens":2592,"prompt_tokens":1039,"completion_tokens":1553,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":655,"completion_tokens_details":{"reasoning_tokens":1463}},"tokens_in":655,"tokens_out":1553,"duration_ms":11228,"temperature":1.0,"reasoning_tokens":1463,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T00:37:33.458758+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same windowed DrCIF training on Experiment 3 with a strict user-exclusive split, holding out all trials of a set of participants rather than random trials, and read the accuracy at window 360 and window 640; if before-touch accuracy falls to chance level or well below the leave-one-out mean, the central early-prediction claim does not transfer to unseen users.","supporting_citations":[{"cited_title":"Middlehurst, J","cited_arxiv_id":null,"evidence_quote":"Defines HIVE-COTE 2.0 and DrCIF, the interval-based ensemble that the paper identifies as its dominant classifier and uses throughout."},{"cited_title":"Reach prediction using finger motion dynamics,","cited_arxiv_id":null,"evidence_quote":"The earlier reach-prediction study using finger motion dynamics whose failure to discriminate synthetic objects motivates the current approach."},{"cited_title":"Bake off redux: a review and experimental evaluation of recent time series classification algorithms,","cited_arxiv_id":null,"evidence_quote":"Categorizes multivariate time series classification algorithms and motivates the meta-ensemble design that the paper reduces to DrCIF."},{"cited_title":"Greedy function approximation: a gra- dient boosting machine,","cited_arxiv_id":null,"evidence_quote":"Introduces partial dependence plots, the method XMTC uses to show which features drive each class decision."},{"cited_title":"Detoxer: a visual debugging tool with multiscope explanations for temporal multilabel classi- fication,","cited_arxiv_id":null,"evidence_quote":"Supplies the heatmap style for per-time-series prediction evolution that XMTC adapts for individual trajectory inspection."},{"cited_title":"catch22: Canonical time- series characteristics: Selected through highly compar- ative time-series analysis,","cited_arxiv_id":null,"evidence_quote":"catch22; its canonical time-series features expand DrCIF's candidate feature pool and therefore define the statistics the trees are built on."}],"review_version":1}