{"id":"91397d4a-49cd-4049-b9ea-a4abed33c668","arxiv_id":"2412.06596","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A HoloLens 2-based AR system with color-coded trajectory feedback reduced arm-tracking errors in healthy users by about 9 percent compared to no feedback.","lead":"An augmented reality headset with 3D tunnel guidance and real-time color feedback helped healthy users trace arm rehabilitation paths more accurately. The prototype also received moderate usability scores from clinicians, suggesting the approach is worth testing on neurological patients.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No tunnel-only control condition means the reported kinematic improvement cannot be attributed to the real-time feedback; the no-Holo baseline differs in both trajectory visibility and feedback. A feedback-disabled HoloLens condition is required to determine the active ingredient.","rationale":"The reader's conditional verdict is reasonable, but the weakest point is slightly upstream of hand-tracking validation: the experiment does not separate the three-dimensional trajectory guide from the real-time feedback channel. The paper's own framing in §2.2 emphasizes the novelty of real-time performance indication, so the causal role of that channel is central. The missing tunnel-only condition is a direct experimental gap, and the ambiguous order statement in §4.2 raises an additional learning-confound possibility. A feedback-disabled condition would settle the attribution without requiring new tracking hardware, and it is a natural extension of the existing protocol. Since the application-level feasibility result remains plausible, I keep the reader's CONDITIONAL verdict rather than moving to reject.","tokens_in":14342,"tokens_out":5766,"duration_ms":66406,"concrete_test":"Run a randomized within-subject protocol with three conditions: (1) no HoloLens (start/end targets only), (2) HoloLens with the static tunnel but no colour/size error feedback, and (3) HoloLens with active feedback as in the paper. Compute AGREE end-effector RMSE per condition per exercise. If condition 2 is statistically equivalent to condition 3 and both beat condition 1, the improvement is due to trajectory visualization, not real-time feedback; if condition 3 beats condition 2, the feedback channel is the active ingredient. Include session/order as a covariate and randomize condition order, including condition 1.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims 'our application leads to improved kinematic performance,' with the novelty being real-time colour-coded feedback (§2.2). The protocol in §4.2 compares a no-Holo condition in which participants see only start/end targets against HoloLens conditions in which they see the full 3D tunnel plus the dynamic colour/size feedback. Therefore Table 2's drop from 2.28±0.45 to 2.06±0.57 cm (end-effector) bundles at least two interventions: the static trajectory visualization and the real-time feedback. Even a perfectly accurate HoloLens tracker (the reader's hand-tracking concern) would not resolve this, because the tracking quality only affects the feedback channel, not the tunnel's presence. The protocol paragraph also lists the no-headset block before the HoloLens blocks while stating that condition order was randomized; if the no-Holo block always came first, a practice/learning effect could alone produce part of the observed improvement. These issues do not invalidate the application-level finding, but they do mean the central claim about real-time kinematic feedback is not yet causally established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an augmented reality (AR) rehabilitation system built on the HoloLens 2 that projects custom 3D tunnel trajectories and provides real-time, colour- and size-coded feedback based on the HoloLens hand-tracking centroid. The authors report a study with 15 healthy participants who performed four upper-limb exercises while wearing the AGREE exoskeleton, comparing kinematic error in three HoloLens conditions (three confidence intervals, C1-C3) against a no-HoloLens baseline. They also report a usability and acceptability evaluation with 12 clinicians, yielding a SUS score of 67.7 and a TAM 'Willingness to Use' score of 4.4/5. The central claim is that the AR application improves kinematic precision, supported by a significant decrease in end-effector error from 2.28 ± 0.45 cm without HoloLens to 2.06 ± 0.57 cm with HoloLens C1, and by corresponding improvements in joint space.","tokens_in":14556,"tokens_out":4405,"duration_ms":48065,"significance":"If the causal claim survives scrutiny, this paper provides one of the few quantitative demonstrations that AR-based real-time kinematic feedback improves trajectory-tracking precision in upper-limb rehabilitation exercises. A key strength is that the outcome measure (AGREE encoder kinematics) is independent of the feedback device, so the central effect is not an artifact of the paper's own measurement definitions. The clinician evaluation is also informative for the acceptability of the system. However, the central 'real-time feedback' attribution is confounded with the presence of the static trajectory visualization, and the manuscript does not report the claimed hand-tracking validation. These issues are addressable with an additional control condition and a validation subsection, so the significance is conditional on that work.","major_comments":[{"comment":"The comparison in Table 2 contrasts a no-HoloLens condition in which participants see only start and end targets with HoloLens conditions in which they see both the full 3D tunnel and the dynamic colour/size feedback. Therefore the reported error reduction (from 2.28 ± 0.45 cm to 2.06 ± 0.57 cm) cannot be attributed to the real-time feedback channel; it is equally compatible with the static trajectory visualization being beneficial. To support the abstract's claim that the application's real-time kinematic feedback leads to improved performance, the authors need a control condition with HoloLens showing the tunnel but without the colour/size feedback, and that comparison should be reported explicitly.","section":"§4.2, Table 2, Abstract"},{"comment":"The protocol lists 'Without wearing the headset' as the first block and then states that the order of conditions was randomized. This is ambiguous. If the no-HoloLens condition always preceded the HoloLens conditions, practice or familiarization could explain part of the improvement. The authors should state exactly how the four conditions were randomized (e.g., Latin square, random permutation), report whether the no-HoloLens block could occur after a HoloLens block, and, if randomization was incomplete, include order as a covariate in the analysis.","section":"§4.2"},{"comment":"The Wilcoxon signed-rank comparisons are reported only with significance stars, without multiple-comparison correction, exact p-values, effect sizes, or confidence intervals. With 15 comparisons per space (4 exercises plus global, each against three HoloLens conditions), the number of significant results at the 0.05 level in the end-effector rows is not overwhelming: for example, T1-C1, T2-C3, and T3-C1/C2 do not reach significance. The authors should report adjusted p-values (e.g., Bonferroni or FDR) and effect sizes or 95% confidence intervals for the key differences, and rephrase the claim that improvements are demonstrated across all four exercises.","section":"§5.1, Table 2"},{"comment":"The Introduction states that AGREE was used to validate the hand-tracking system, but the manuscript reports no direct comparison of HoloLens hand-centroid positions with AGREE end-effector positions. Since the real-time feedback is computed from the HoloLens tracker, a biased or delayed tracker could mislead the user and change the interpretation of the feedback effect. The authors should either add a dedicated validation subsection with the comparison data (e.g., tracking error over time, correlation, latency) or explicitly remove the claim of validation and discuss the implications for the interpretation of the feedback results.","section":"§3.2 and §1 (Introduction)"}],"minor_comments":[{"comment":"The SUS score is reported as 67.7 ± 12.1 and described as 'OK' and 'in line with' the 68 threshold; 67.7 is actually below 68, so the text should say the score is marginally below the acceptable-usability threshold, not that it reaches it.","section":"§5.2"},{"comment":"The Discussion states 'The Comfort of the device was ranked 4.04 out of 5,' but Table 3 reports 4.45 ± 0.44 for Comfort and 4.04 ± 0.62 for Clarity of the calibration. Please correct this inconsistency.","section":"§5.1, Table 3"},{"comment":"The reference to Burke et al. (2010) appears twice consecutively in the same sentence ('(Burke et al, 2010)(Burke et al, 2010)'). Consolidate the citation and check the reference list for other duplicate entries.","section":"§2.2, References"},{"comment":"In Fig. 3 (RIGHT), the three confidence intervals are described as visibly distinct, but the printed figure does not clearly show the size differences or the green/red transitions; in Fig. 4, the 'thin coloured line' representing the user's path is difficult to see. Please provide higher-resolution or annotated screenshots.","section":"Figures 3 and 4"},{"comment":"Please report the exact number of repetitions per condition (the text says 'five times' only for the no-HoloLens condition; for consistency, state whether the HoloLens conditions also used five repetitions, and how the repetitions were segmented for the temporal normalization in Eq. (2)).","section":"§4.2 (Exercises execution)"}],"recommendation":"major_revision","confidential_remarks":"The core system is interesting and the kinematic outcome is measured with AGREE encoders, which is a real strength. The main obstacle is the confound between the static tunnel visualization and the real-time feedback: the abstract's causal claim cannot be sustained without a feedback-disabled control condition. A well-designed follow-up with one additional condition, plus the missing hand-tracking validation, would substantially strengthen the manuscript. The clinician evaluation is valuable but secondary. I recommend major revision with a request for the control condition, corrected statistics, and the tracking validation data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the system is new and the kinematic improvement is probably real, but the paper does not show that the real-time feedback is the active ingredient. The no-Holo baseline sees only start/end targets, while the HoloLens condition shows the full 3D tunnel plus the colour/size feedback. So the Table 2 drop from 2.28 to 2.06 cm bundles trajectory visualization together with the feedback. A tunnel-only HoloLens condition with feedback disabled is the missing control, and it is not optional. The protocol says the order was randomized, but it also lists the no-headset block first; if that block actually came first, practice alone could explain part of the gain.\n\nWhat the paper does well: the combination of HoloLens 2 hand tracking, custom 3D tunnels, and the AGREE exoskeleton as an independent outcome measure is new, and the outcome is not circular—AGREE encoders don't depend on the HoloLens at all. The healthy-subject data across four exercises and the clinician SUS/TAM feedback give a reasonable feasibility picture. The paper is clearly written and the methods are mostly reproducible in spirit, though code and data are only available \"on reasonable request.\"\n\nSoft spots, in proportion: the missing tunnel-only control is the main one, and the abstract's \"demonstrating\" overreaches because of it. The claimed hand-tracking validation is never actually reported—no direct HoloLens-to-AGREE position comparison appears in the results, only a questionnaire item. The statistics use multiple Wilcoxon tests without correction, and there are no effect sizes or confidence intervals. The healthy sample is small and no patients were tested, which the authors do acknowledge. These are normal feasibility-study limitations; none of them sink the application-level finding.\n\nThis paper deserves a serious referee, not a desk rejection. The right outcome is major revision: add a tunnel-only control condition, report the hand-tracking validation data or soften the claim, and improve the statistical reporting. I would not cite it for the causal claim in its current form, but it is a useful existence proof and a fair target for discussion. Bring it to reading group if you want a good example of why control conditions matter in feedback studies.","headline":"Genuinely novel AR rehab feedback system with a likely real kinematic effect, but the paper never isolates the real-time feedback from the tunnel visualization, so the central causal claim is not yet established.","tokens_in":15075,"tokens_out":1654,"would_cite":false,"duration_ms":21142,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AR feedback lowers hand-path error in rehab reaching","keywords":["augmented reality","upper limb rehabilitation","kinematic feedback","hand tracking","HoloLens 2","exoskeleton","usability","technology acceptance"],"falsifier":"A time-synchronized comparison of the HoloLens 2 hand centroid against the AGREE end-effector position during the same reaching movements would settle it: if the tracking error or latency is comparable to the tunnel's tolerance (1.5 to 5 cm from center), the color feedback could be directing users toward the wrong place.","tokens_in":14175,"feed_emoji":"🥽","tokens_out":10561,"duration_ms":94501,"temperature":0.7,"pith_summary":"The paper sets out to show that an augmented-reality app can make upper-limb rehabilitation exercises more precise by showing the user a 3D trajectory tunnel and updating its color live as the hand deviates from the path. The application runs on the HoloLens 2 headset, tracks the hand directly, and is meant to be used together with exoskeleton robots like AGREE, which currently give support but little feedback. In a test with 15 healthy adults performing four reaching exercises, trajectory error in end-effector space fell from $2.28 \\pm 0.45$ cm without feedback to $2.06 \\pm 0.57$ cm with the widest tunnel, a statistically significant difference, and similar reductions appeared in joint space. Twelve clinicians rated the system acceptable on usability and high on willingness to use. If these results hold in patients with neurological disorders, the same exoskeleton hardware could deliver the real-time corrective feedback that therapists currently lack.","feed_headline":"AR feedback lowers hand-path error in rehab reaching","feed_subtitle":"Color-coded 3D tunnel guidance cut reaching error from 2.28 to 2.06 cm in a 15-person trial","key_machinery":"The central object is the trajectory tunnel: a 3D holographic tube made of spheres that represents the path the user should follow, with three selectable confidence intervals ($10$ cm, $6.5$ cm, and $3$ cm diameters) that set how much deviation is tolerated before the feedback turns red. The feedback loop is driven by the Euclidean distance between the HoloLens 2 hand-tracking centroid and the closest via-point of the tunnel; the spheres shrink and darken in green as the distance decreases, and the path actually followed is drawn as a thin line after the task. The AGREE exoskeleton supplies the independent kinematic measurement: its joint encoders run at 5 kHz, and its forward-kinematics model gives the end-effector position used to compute the reported root-mean-square error. The tunnels can be generated from stored trajectory databases or from therapist demonstrations recorded with the headset cameras.","core_discovery":"The central claim is that giving users concurrent, color-coded 3D feedback on hand-path error improves kinematic precision during exoskeleton-assisted arm exercises. The application projects a custom trajectory as a tunnel of spheres with three selectable diameters ($10$ cm, $6.5$ cm, $3$ cm), corresponding to increasingly strict confidence intervals. During execution, HoloLens 2 computes the Euclidean distance between the tracked hand centroid and the nearest via-point of the trajectory, and the spheres shrink and turn green as the error decreases. The measured effect, averaged over four exercises and five repetitions per subject, is a drop in end-effector error from $2.28 \\pm 0.45$ cm in the no-HoloLens condition to $2.06 \\pm 0.57$ cm in the widest-interval condition, with statistically significant differences in both end-effector and joint space. The authors interpret this as evidence that the AR feedback improves accuracy of movement execution, which matters because accurate execution is thought to prevent maladaptive plasticity.","pith_inferences":["The paper does not isolate learning from guidance; without a retention test after feedback is removed, the improved error could reflect following a visible guide rather than a durable change in motor control. A reasonable next experiment is to re-measure the same exercises with feedback switched off after a training block.","The paper says the HoloLens 2 hand-tracking centroid is compared with the nearest via-point, but it does not directly validate that centroid against the AGREE encoders; a sample-by-sample comparison would either close that gap or reveal a bias that could make the tunnel teach a shifted path.","The clinician questionnaire suggests that perceived ease of use, not perceived usefulness, is the main barrier to adoption; the ease-of-use score was the lowest category (3.50/5) and it had the weakest correlation with willingness to use, so future work should focus on simplifying interaction."],"forward_implications":["If the central claim is correct, exoskeleton-based rehabilitation can gain a real-time visual channel that shows both patient and therapist how far the hand is from the prescribed path, without altering the robot's hardware.","Because the error reduction was consistent across four different exercises and did not show an exercise-condition interaction, the feedback should generalize to a variety of reaching and drawing tasks rather than one specific movement.","The joint-space results suggest that following the holographic tunnel improves or preserves shoulder and elbow coordination instead of trading hand accuracy for worse posture.","The reported SUS score of 67.7 is just below the standard 'acceptable' threshold of 68, so usability improvements would be needed before broad clinical rollout, but the clinicians' willingness-to-use score of 4.38/5 indicates the core value is recognized.","The authors' stated expectation is that patients with neurological disorders, who rely more on extrinsic feedback, may show larger gains than healthy subjects; that is the natural next test."],"supporting_citations":[{"why":"Supplies the AGREE exoskeleton whose encoders and kinematic model measure the end-effector and joint positions used for all error comparisons.","marker":"Dalla Gasperina et al, 2023"},{"why":"Describes AGREE's trajectory generation system, which stores the desired trajectories used as the reference for the RMSE comparisons.","marker":"Gasperina et al, 2022"},{"why":"Provides the evidence base that augmented visual feedback improves motor learning and motivates the 3D tunnel design for multidimensional arm movements.","marker":"Sigrist et al, 2013"},{"why":"Meta-analysis showing AR's potential in stroke rehabilitation, used to justify AR over VR for the feedback system.","marker":"Phan et al, 2022"},{"why":"Establishes that stroke patients often lack intrinsic feedback, the clinical rationale for concurrent augmented feedback.","marker":"Thikey et al, 2012"},{"why":"Supplies the claim that precise movement execution prevents maladaptive plasticity, the clinical goal the system serves.","marker":"Takeuchi and Izumi, 2013"},{"why":"Provides the System Usability Scale scoring that the clinicians' 67.7 result is measured against.","marker":"Bangor et al, 2008"},{"why":"Defines the Technology Acceptance Model categories used for the clinicians' acceptability evaluation.","marker":"Marangunić and Granić, 2015"}],"fun_headline_variants":["AR feedback trims hand-path error in rehab reaching","HoloLens 2 AR cuts rehab arm error","Color-coded AR tunnels improve kinematic precision","AR feedback reduces reaching error in 15-person trial","Real-time AR feedback improves movement precision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The feedback loop assumes the HoloLens 2's hand tracking gives an accurate, low-latency estimate of where the hand actually is, and the paper does not directly compare the headset's tracked position with the AGREE encoder position during the same movement.","fun_headline_variants_meta":{"raw":{"variants":["AR feedback trims hand-path error in rehab reaching","HoloLens 2 AR cuts rehab arm error","Color-coded AR tunnels improve kinematic precision","AR feedback reduces reaching error in 15-person trial","Real-time AR feedback improves movement precision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001398,"raw_usage":{"total_tokens":5707,"prompt_tokens":1050,"completion_tokens":4657,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":666,"completion_tokens_details":{"reasoning_tokens":4585}},"tokens_in":666,"tokens_out":4657,"duration_ms":34999,"temperature":1.0,"reasoning_tokens":4585,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:29:17.396996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A time-synchronized comparison of the HoloLens 2 hand centroid against the AGREE end-effector position during the same reaching movements would settle it: if the tracking error or latency is comparable to the tunnel's tolerance (1.5 to 5 cm from center), the color feedback could be directing users toward the wrong place.","supporting_citations":[{"cited_title":"Applied Sciences 12(4):1848","cited_arxiv_id":null,"evidence_quote":"Meta-analysis showing AR's potential in stroke rehabilitation, used to justify AR over VR for the feedback system."},{"cited_title":"Trials 13(1):163","cited_arxiv_id":null,"evidence_quote":"Establishes that stroke patients often lack intrinsic feedback, the clinical rationale for concurrent augmented feedback."},{"cited_title":"Stroke Research and Treatment 2013:1--13","cited_arxiv_id":null,"evidence_quote":"Supplies the claim that precise movement execution prevents maladaptive plasticity, the clinical goal the system serves."}],"review_version":1}