{"id":"d08ad2eb-af9e-4fb7-908a-f65f961070ea","arxiv_id":"2507.13602","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Force feedback and force-conditioned ACT training on the low-cost GELLO teleoperator improved success on 3 of 4 manipulation tasks and was preferred by experienced users.","lead":"The authors added force information to GELLO, a low-cost 3D-printed teleoperation system for robot arms, by implementing haptic feedback and by training imitation learning policies on force readings. They report that experienced users preferred the feedback, and that force-conditioned training improved task success in three of four manipulation tasks, though no statistical tests are reported for the learning results.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II's success-rate improvements rest on 3 seeds with large variance; without significance tests, the 3/4 improvement may be sampling noise, undermining the central claim.","rationale":"The reader's weakest_assumption focuses on the reliability of the Franka external-torque estimate for both haptic reflection and policy input. That is a legitimate technical concern, and I agree it affects both contributions. However, the more load-bearing issue for the paper's central quantitative claim is the statistical strength of Table II. Even if the external-torque estimates were perfectly calibrated, the reported success-rate deltas from 3 seeds with the shown standard deviations would not reach conventional significance; the data do not distinguish a real effect of force input from seed-level noise. Conversely, even an imperfect torque signal can improve policy success if it carries useful contact information, as the Drawer Opening recovery behavior suggests. The real-robot demonstrations and randomized initial conditions are positive aspects, but they cannot compensate for the lack of per-seed reporting and significance testing. The user-preference claim is explicitly qualitative, so I do not treat the small experienced-user subgroup as a fatal flaw. With this statistical concern unresolved, the appropriate verdict remains conditional acceptance, matching the reader's verdict.","tokens_in":9082,"tokens_out":11102,"duration_ms":139009,"concrete_test":"Obtain the per-seed success counts underlying Table II (3 seeds x 15 rollouts per condition, for each of the four tasks) and run a paired permutation test or Wilcoxon signed-rank test on the seed-level success rates. If the two-sided p-value for Drawer Opening or Whiteboard Erasing exceeds 0.05, the claim that force inputs improve success on a majority of tasks is not supported. If per-seed data are unavailable, re-run the 3-seed/45-rollout evaluation for Drawer Opening and Whiteboard Erasing under the same protocol and perform the same test.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The quantitative central claim, that force inputs improved task success on the majority of tasks, rests entirely on the four deltas in Table II: Nut Assembly 0.60->0.42, Door Opening 0.96->1.0, Drawer Opening 0.62->0.93, Whiteboard Erasing 0.24->0.36. Each entry is the mean over only 3 random seeds with 15 rollouts per seed, reported as mean +/- SD across seeds. The SDs are very large: Whiteboard position-only is 0.24 +/- 0.48, and position+force is 0.36 +/- 0.48; Drawer position-only is 0.62 +/- 0.48 versus 0.93 +/- 0.25. No significance test of any kind is reported for the IL comparisons. With n=3, a paired test would need a t-statistic above about 4.3 for p<0.05. For the Drawer Opening difference of 0.31, the approximate standard error of the difference is about 0.31, giving t about 1.0; for Whiteboard Erasing the difference is only 0.12 with an even larger SE. Thus the observed 'improved in 3/4 tasks' outcome is statistically indistinguishable from seed-level chance. The paper itself admits that Whiteboard success is limited by occlusion, but that does not address whether the force-conditioned policy is genuinely better. Without per-seed counts or a test, the reported evidence does not establish that force input improves task success.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the GELLO low-cost teleoperation system with force feedback (force-position control) and with force/torque inputs to ACT imitation learning policies. A 20-participant user study finds no significant NASA-TLX differences but claims qualitative preference from experienced users. IL experiments on four tasks (two simulation, two real-world) report success-rate improvements in three of four tasks when force inputs are added, with the caveat that the baseline and force-conditioned policies are compared on the same demonstrations. The central contribution is a low-cost force augmentation for teleoperation and IL on contact-rich tasks.","tokens_in":9382,"tokens_out":4686,"duration_ms":51598,"significance":"If the reported improvements hold, the paper would provide a low-cost, accessible way to add haptic feedback and force-conditioned policies to GELLO-style teleoperation, with implications for contact-rich imitation learning. The implementation is straightforward and uses standard components (Franka FCI torque estimates, Dynamixel current control, ACT). The paper is honest about non-significant TLX results and includes ethical approval, task randomization, and a clear description of the control architecture. However, the evidence for the main IL claim is statistically weak, and the user-study preference claim is anecdotal; the significance of the contribution is therefore contingent on added statistical support and reproducibility details.","major_comments":[{"comment":"The central claim that force inputs improve task success rests entirely on the differences between the 'Position' and 'Position + Force' rows of Table II, each entry being the mean over 3 seeds with 15 rollouts. No significance test, per-seed breakdown, or confidence interval is reported. For Whiteboard Erasing the improvement is 0.24 vs 0.36 with SDs of 0.43 and 0.48; for Drawer Opening it is 0.62 vs 0.93 with SDs 0.48 and 0.25. With n=3, these differences are well within seed-level sampling noise; the statement in Section III-C that 'performance across seeds was relatively consistent' is contradicted by these large SDs. The authors should report per-seed success counts and a paired test (e.g., Wilcoxon signed-rank or permutation test) or temper the abstract's 'improved task success on the majority of tasks' accordingly.","section":"III-C, Table II"},{"comment":"The user-study section reports no significant TLX differences and then states that 'the few users who were familiar with robots' preferred the force-feedback controller. The abstract elevates this to a claim that 'users with robotics experience preferred our controller.' The number of experienced users, how experience was determined, and the preference distribution are not reported, so this claim is unsupported by the presented data. It should be either quantified (N, proportion, inter-rater agreement) or explicitly labeled as an informal observation.","section":"III-B"},{"comment":"The force-feedback gain kf is a critical free parameter for stability and transparency, but its value, tuning procedure, and the 'additional conversion factor' for the Dynamixel current control are not given. Without these, the force feedback implementation is not reproducible, and the force signals used as IL inputs (tau_ext_f) are not characterized. Fig. 2 shows non-zero current in free motion due to noise in the Franka's external-torque estimate; the paper does not state whether this signal is filtered before being used in (2) or as an ACT input. Please provide the gain values, filtering, and a characterization of the torque signal under contact.","section":"II-B, Eq. (2)"}],"minor_comments":[{"comment":"In the Conclusion, 'teleportation' should be 'teleoperation'.","section":"IV"},{"comment":"The maximum-likelihood objective for BC is written as 'max π(at|st)', but the likelihood of a trajectory is a product over timesteps; please clarify the notation.","section":"II-C"},{"comment":"The statement that 'during free motion the leader will not have any force feedback' is only true if tau_ext_f is exactly zero; Fig. 2 shows noise, so the claim should be qualified.","section":"II-B"},{"comment":"The Whiteboard Erasing success criterion 'majority of the mark is erased' is vague; a more objective metric (e.g., pixel coverage) would strengthen the evaluation.","section":"III-A"},{"comment":"The text reports only the largest Wilcoxon signed-rank result (Physical Demand, W=19.0, p=0.06); please report the W and p values for all six TLX items for completeness.","section":"Table I"},{"comment":"Please specify whether the position-only and position+force policies are trained on the same demonstration set or on separate collections; this affects the interpretation of the comparison.","section":"III-C"}],"recommendation":"major_revision","confidential_remarks":"The paper's contribution is incremental but potentially useful; the main risk is overclaiming from small-sample experiments. I would advise requiring statistical support or a softened claim, and asking for kf and torque-signal details. The qualitative user-study claim should not appear in the abstract in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is a solid engineering contribution: it adds force feedback to the low-cost GELLO leader via standard force-position control, using the Franka's external-torque estimate and Dynamixel current control, and it feeds the same force signal into ACT during training and deployment. That integration is genuinely new for the GELLO platform, and the paper is honest about where it sits relative to Bi-ACT and Comp-ACT. The writing is clear, the method is described in enough detail to be plausible, and the user study is properly randomized even if it came out null.\n\nWhat the paper does well: it implements something non-trivial (force reflection on a <$300 leader with motor and power limits), and it shows a plausible mechanism for improvement in the Drawer task—policies learning to recover from missed grasps using force cues. The authors also acknowledge the failure cases, including the Nut Assembly regression and the Whiteboard occlusion problem. No circularity; the literature is cited appropriately.\n\nThe soft spot is the imitation-learning evidence. Table II shows means over 3 seeds with large standard deviations, and there are no significance tests. The stress-test concern lands: the Drawer Opening delta (0.62 to 0.93) and the Whiteboard delta (0.24 to 0.36) are both within what you'd expect from seed noise given n=3 and 15 rollouts per seed. The paper's claim that 'performance across seeds was relatively consistent' doesn't address this; consistent rates can still give large binomial variance. So the central quantitative claim—that force inputs improved success on most tasks—is descriptive, not inferential. That's not fatal; the authors don't oversell, but the abstract's 'improved task success on the majority of tasks' should be read as 'observed improvement in the means,' not as a demonstrated effect. Also, kf is never reported, no code or data are promised, and the Fig. 2 caption already flags noise in the external-torque estimate, which is exactly the signal being used.\n\nWho is this for? People building low-cost teleoperation rigs for IL data collection, and researchers working on force-conditioned policies for contact-rich manipulation. They will find the integration details and the honest evaluation useful, even if the head-to-head policy numbers are not decisive.\n\nI would send this to peer review. It deserves a serious referee, but I'd ask for per-seed counts, a significance test or effect-size analysis, and the value of kf. If the authors release code, so much the better. This is a conditional accept with revisions, not a rejection.","headline":"A useful engineering extension of GELLO with force feedback and force-conditioned ACT, but the IL comparison is statistically underpowered and the quantitative central claim is not established beyond descriptive means.","tokens_in":747,"tokens_out":897,"would_cite":false,"duration_ms":34255,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By adding force reflection and a force-sensing learning channel to the low-cost GELLO leader, the paper reports improved task success on three of four contact-rich manipulation tasks and preference for haptics among experienced…","keywords":["teleoperation","force feedback","haptic feedback","imitation learning","behavioral cloning","action chunking transformer","contact-rich manipulation","external torque estimation"],"falsifier":"Log the external-torque feature $\\tau^{\\mathrm{ext}}_f$ during the Drawer Opening evaluations and check whether it cleanly separates successful handle grasps from misses; if the free-motion noise floor overlaps the contact-induced values, the claimed 0.62-to-0.93 recovery benefit cannot be attributed to the policy detecting a missed grasp from absent resistance.","tokens_in":8897,"feed_emoji":"🕹️","tokens_out":13500,"duration_ms":135249,"temperature":0.7,"pith_summary":"The paper takes the low-cost GELLO teleoperation handle—a 3D-printed, kinematically scaled leader arm that normally sends only joint positions—and shows that a force channel can be added without making the system expensive or complex. On the human side, the leader's servo motors are driven by a scaled follower-side external torque, so the user feels resistance exactly when the robot contacts the environment. On the learning side, the same torque estimate is fed into an action-chunking transformer policy alongside images and leader joint positions. Reported success rises from 0.62 to 0.93 on drawer opening and from 0.24 to 0.36 on whiteboard erasing, stays near perfect on door opening, and drops from 0.60 to 0.42 on nut assembly. A 20-person user study found no significant workload difference on the standard Task Load Index, but users with robotics experience qualitatively preferred the force-feedback controller.","feed_headline":"Force inputs lift teleoperation success on 3 of 4 tasks","feed_subtitle":"A low-cost leader arm gains haptics and torque-aware policies, letting robots recover missed grasps and erase better.","key_machinery":"The load-bearing mechanism is the force-position (FP) control loop $\\tau^{\\mathrm{ref}}_l = -k_f\\,\\tau^{\\mathrm{ext}}_f$, in which the follower tracks the leader's joint positions while the estimated external torque on the follower is scaled by $k_f$ and applied to the leader's servo motors as a current command. Because GELLO's Denavit-Hartenberg parameters are a kinematically scaled version of the follower's, the reflected joint torque carries the contact information despite the leader's lack of force sensors. The second mechanism is the ACT policy, an action-chunking transformer that predicts short sequences of joint positions; its observation space is extended from images plus leader joint positions to include the external torque vector, while its targets remain the GELLO joint positions. Together the two mechanisms turn a contact event on the follower into a felt resistance for the operator and a training feature that lets a behavior-cloned policy detect and recover from failed contacts.","core_discovery":"The central claim is that a low-cost leader robot with no force sensors can provide meaningful haptic feedback and, more importantly, that the same contact signal improves imitation learning on contact-rich manipulation tasks. The control law is force-position reflection, $\\tau^{\\mathrm{ref}}_l = -k_f\\,\\tau^{\\mathrm{ext}}_f$, where $\\tau^{\\mathrm{ext}}_f$ is the follower's estimated external joint torque and $k_f$ is a scalar gain; the leader feels a scaled version of what the follower feels. When this force vector is added to the observation of an ACT policy, the trained policy can notice the absence of expected resistance—for example, a missed drawer-grip—and re-grasp, which the position-only policy cannot do. The paper reports three of four tasks improving with force, with the only regression being nut assembly, where force sensitivity is not needed and the bottleneck is initial pick-up.","pith_inferences":["The same force-reflection recipe should transfer to any follower arm that exposes an external-torque estimate, potentially making haptic teleoperation a standard add-on rather than a custom build; the paper only demonstrates one follower platform.","A deployment-time ablation—zeroing the force input after training—would separate whether the policy needs force online or only benefits from learning with force; the paper does not run this ablation.","The grasp-recovery behavior suggests force-conditioned imitation may need fewer demonstrations on contact-rich tasks than position-only imitation, but the paper keeps demonstration count fixed.","Because the paper tests a single policy family (ACT), the conclusion that force inputs improve imitation learning is so far specific to action-chunking transformers; diffusion-policy variants remain untested."],"forward_implications":["Force-conditioned ACT policies can recover from failed grasps on their own: drawer opening success rose from 0.62 to 0.93 because the policy recognized a missed grip from absent resistance and re-grasped.","The force channel is not universally helpful: nut assembly success fell from 0.60 to 0.42, so contact-force inputs should be enabled selectively by task.","A low-cost, sensor-free leader can produce usable haptic feedback if its power supply is upgraded and the reflection gain is scaled down; the original hardware disconnects its motors under torque.","User workload ratings were not significantly different between controllers, while experienced teleoperators reported that force feedback gave complementary information to vision.","Seed-to-seed success was consistent in the reported evaluations, indicating the ACT training procedure is stable across random initializations in these settings."],"supporting_citations":[{"why":"Supplies the low-cost, kinematically equivalent leader hardware and joint-position teleoperation baseline that this work extends.","marker":"[13]"},{"why":"Provides the ACT architecture, default hyperparameters, action-chunking training, and temporal-averaging deployment used for the policies.","marker":"[14]"},{"why":"Establishes the stability and scaling constraints on the force-feedback gain in force-position bilateral control.","marker":"[21]"},{"why":"Defines the stability and transparency objectives that frame the teleoperation controller design.","marker":"[15]"},{"why":"Is the prior bilateral-control extension of ACT that incorporates force, used as the design comparison for force-in-imitation-learning.","marker":"[26]"},{"why":"Provides the Task Load Index survey instrument used in the user study to compare perceived workload.","marker":"[37]"},{"why":"Provides the meta-analytic evidence that haptic feedback improves teleoperation task performance, motivating the force-feedback extension.","marker":"[19]"},{"why":"Provides the Nut Assembly and Door Opening simulation task definitions used for policy evaluation.","marker":"[36]"}],"fun_headline_variants":["Force feedback lifts low-cost teleop on 3 of 4 tasks","Haptic force improves imitation learning and teleop","Torque-aware policies help robots re-grasp on teleop","Cheap teleop gains force sense, better on most tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the follower arm's internally computed external-torque signal being a faithful measure of real contact, when the paper's own figure shows non-zero, noisy torque during free motion.","fun_headline_variants_meta":{"raw":{"variants":["Force feedback lifts low-cost teleop on 3 of 4 tasks","Haptic force improves imitation learning and teleop","Torque-aware policies help robots re-grasp on teleop","Cheap teleop gains force sense, better on most tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000433,"raw_usage":{"total_tokens":2164,"prompt_tokens":856,"completion_tokens":1308,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":1237}},"tokens_in":472,"tokens_out":1308,"duration_ms":16263,"temperature":1.0,"reasoning_tokens":1237,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:20:44.149823+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log the external-torque feature $\\tau^{\\mathrm{ext}}_f$ during the Drawer Opening evaluations and check whether it cleanly separates successful handle grasps from misses; if the free-motion noise floor overlaps the contact-induced values, the claimed 0.62-to-0.93 recovery benefit cannot be attributed to the policy detecting a missed grasp from absent resistance.","supporting_citations":[{"cited_title":"GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators,","cited_arxiv_id":null,"evidence_quote":"Supplies the low-cost, kinematically equivalent leader hardware and joint-position teleoperation baseline that this work extends."},{"cited_title":"Learning Fine-grained Bimanual Manipulation with Low-Cost Hardware,","cited_arxiv_id":null,"evidence_quote":"Provides the ACT architecture, default hyperparameters, action-chunking training, and temporal-averaging deployment used for the policies."},{"cited_title":"Design fundamentals of master-slave systems with a force-position bilateral control scheme,","cited_arxiv_id":null,"evidence_quote":"Establishes the stability and scaling constraints on the force-feedback gain in force-position bilateral control."},{"cited_title":"Stability and Transparency in Bilateral Teleopera- tion,","cited_arxiv_id":null,"evidence_quote":"Defines the stability and transparency objectives that frame the teleoperation controller design."},{"cited_title":"Bi-ACT: Bilateral Control-Based Imitation Learning via Action Chunking with Transformer,","cited_arxiv_id":null,"evidence_quote":"Is the prior bilateral-control extension of ACT that incorporates force, used as the design comparison for force-in-imitation-learning."},{"cited_title":"Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research,","cited_arxiv_id":null,"evidence_quote":"Provides the Task Load Index survey instrument used in the user study to compare perceived workload."},{"cited_title":"A meta-analysis of the effects of haptic interfaces on task performance with teleoperation systems,","cited_arxiv_id":null,"evidence_quote":"Provides the meta-analytic evidence that haptic feedback improves teleoperation task performance, motivating the force-feedback extension."}],"review_version":1}