{"id":"c00c5542-db15-477b-8224-bee908f39c21","arxiv_id":"2411.12734","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SWIFT uses CMA-ES to find pen-spinning parameters for a soft robotic hand from real-world trials, reaching 10/10 success on three pens.","lead":"This paper describes SWIFT, a robotic system that learns to spin a pen in a soft robotic hand through real-world trial and error, without a simulator or a model of the pen. It reaches a 100% success rate on three differently weighted pens and can also spin a brush and a screwdriver, showing soft hands can handle fast, dynamic tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Success claims are not independent of human selection: the abstract says 'self-labeled trials' but §V-B says a human labels successes, and brush/screwdriver evaluations use the 'first manually observed success' rather than the optimized policy.","rationale":"The reader's weakest assumption is reset repeatability; I find that plausible given a fixed slot and fixed arm trajectory, and it is a standard experimental precondition, though unmeasured. The more load-bearing issue is that the reported success rates depend on human selection. The paper claims self-supervised learning, but §V-B explicitly says human observers label trials, and §V-A says the brush/screwdriver evaluation uses the first manually observed success instead of the CMA-ES incumbent. This makes the headline 10/10 and 5/10 conditional on a favorable hand-picked sample, not an unbiased assessment of the learned policy. It also undermines the claim that 130 sampled actions suffice. This is an internal inconsistency, not a disagreement with consensus, and it can be checked by re-running the final incumbent. I would keep the conditional verdict: the underlying system may work, but the current evidence does not support the quantified claims until the selection bias is removed and the self-supervision language is reconciled.","tokens_in":9344,"tokens_out":5578,"duration_ms":57252,"concrete_test":"Re-evaluate the brush and screwdriver with the CMA-ES incumbent parameters (final mean and best candidate according to Eq. 1, not the first manually observed success) over 10 trials each, using an automated success criterion from the point-cloud/PCA pipeline. Also recompute Table I for all three pens using the final incumbent. If the incumbent success rates are materially below the reported 10/10 and 5/10, the headline claims are selection-biased. Optionally, rerun the full optimization with no human in the loop, replacing the human stop rule with a fixed 10-generation budget and always evaluating the incumbent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SWIFT autonomously learns a robust spin policy from only real-world, self-labeled trials, reaching 10/10 on three pens and 10/10 and 5/10 on brush and screwdriver after about 130 sampled actions. The weakest load-bearing point is not the reset procedure but the evaluation protocol. The abstract and contribution (2) claim 'self-labeled trials,' yet §V-B states: 'During optimization, the reward function we used only indirectly captures whether a spinning action is successful or not. Thus, a human observer labels trials a success or failure.' For the brush and screwdriver, §V-A says: 'We then chose the first manually observed success for evaluation, rather than directly using the stored value of CMA-ES.' Selecting a sample because a human saw it succeed, then scoring that same sample 10/10 or 5/10, is not an unbiased estimate of the learned policy; it is conditional on a favorable draw. It also means the 130-sample cost does not describe the number of trials needed to obtain a robust policy, only the trial in which the first human-visible success appeared. If the same protocol was used for the pen rows, the 100% claims are similarly inflated. The paper never reports the success rate of the CMA-ES incumbent (final mean or best-by-objective) or of all final-generation candidates, so the reported rates cannot be separated from cherry-picking.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents SWIFT, a system that uses a three-finger soft robotic hand (MOE) attached to a 6-DOF arm to learn a \"Thumbaround\"-style pen spinning skill entirely from real-world trials. The action space is compressed to eight parameters: six servo-target changes, a catch delay, and a horizontal grasp offset. CMA-ES optimizes these parameters using a camera-based proxy reward that combines accumulated rotation around the hand's m3 finger with a fall penalty. The authors report 10/10 success on three visually identical pens with different mass distributions after 130 sampled actions per object, and 10/10 and 5/10 on a brush and a screwdriver respectively. They also report ablations showing that optimizing the grasp offset is important and that parameters tuned for one pen do not transfer to another. The claimed novelty is that a soft hand can learn a fast, dynamic in-hand manipulation skill without simulation, precise object models, or closed-loop high-speed control.","tokens_in":9753,"tokens_out":4297,"duration_ms":44809,"significance":"If the claims hold, this would be a meaningful step for soft robotic manipulation, which has mostly been limited to quasi-static tasks. The paper's strengths are its real-world, trial-and-error optimization approach; a deliberately compact task-specific parameterization; and the public release of videos, data, and code. The system also demonstrates a plausible use of CMA-ES on a physical platform where simulation would be difficult due to soft-body contact dynamics. However, the evaluation protocol currently undermines the headline claims: the paper describes trials as \"self-labeled\" while later stating that a human observer labels success, and the generalization results are based on a manually selected successful sample rather than the optimized policy. These issues are load-bearing for the central contribution of autonomous, self-supervised learning, and they must be resolved before the reported success rates can be interpreted as policy performance.","major_comments":[{"comment":"The abstract and contribution (2) describe the trials as \"self-labeled\" and the process as \"self-supervised autonomous.\" In §V-B, however, the authors state: \"a human observer labels trials a success or failure.\" These statements are in direct tension. If the human labels are used only for the reported success rates and not for optimization, the text must say so explicitly and avoid \"self-labeled.\" If human labels are used to select or update parameters, the process is not autonomous and the contribution claim should be revised. This distinction is load-bearing for the paper's central claim.","section":"Abstract, §V-B"},{"comment":"For the brush and screwdriver, the authors \"chose the first manually observed success for evaluation, rather than directly using the stored value of CMA-ES.\" Selecting a parameter sample because a human observed a success, and then scoring that same sample 10/10 or 5/10, is a selection-biased estimate of the learned policy. It does not measure the performance of the CMA-ES incumbent, the final mean, or the best-by-objective parameter. The paper should report the success rates of the final CMA-ES policy (e.g., the incumbent after the last generation), and clearly separate optimization samples from evaluation trials.","section":"§V-A"},{"comment":"It is not specified whether the 10-trial success rates for the three pens are obtained from the CMA-ES incumbent, from the best-by-objective sample, or from a manually selected successful sample as in the generalization experiments. If the pen rows also use a manually selected favorable trial, the 10/10 claims overstate the reliability of the learned policy. The authors should state the exact parameter selection rule for every row and, ideally, report success rates for both the objective-best and the final mean parameter sets.","section":"Table I, §V-A"},{"comment":"The objective in Eq. (1) is an indirect proxy: it accumulates z-axis rotation while the filtered point cloud exceeds a threshold, and the penalty counts frames with insufficient points. This proxy may score a trial as good even when the pen does not complete a full rotation around the finger or is caught in a way the human would not call a success. Since \"success\" is defined in §III as a full rotation without dropping, the mismatch between the proxy and the human label should be quantified (e.g., agreement rate between high reward and human success), or the objective should be redesigned to directly encode the success criterion. At minimum, the paper should clarify whether the human labels influenced the optimization or only the final evaluation.","section":"§IV-D, §V-B"}],"minor_comments":[{"comment":"There is a typo: \"The sot robot hand consists...\" should be \"The soft robot hand consists...\".","section":"§IV-A"},{"comment":"The sentence \"The camera has setup to have its z-axis pointing...\" is ungrammatical; it should be \"The camera was set up with its z-axis pointing...\".","section":"§IV-D"},{"comment":"The definition of the rotation angle θt_z is not precise: the text says the first principal component is projected onto the x, y, and z planes to compute Euler angles, but it is unclear how the sign and branch of the z-axis rotation are resolved. Please specify the exact convention used.","section":"§IV-D"},{"comment":"The phrase \"130 sampled actions per object\" in the abstract is inconsistent with the generalization protocol, where optimization stops at the first generation with observed successes (reported as generation 4, i.e., about 52 samples). The per-object sample counts should be stated explicitly for the pen experiments and for the brush/screwdriver experiments separately.","section":"§V-A"},{"comment":"The row label \"Initialization ∅\" is confusing: the row reports the success rate of the fixed hand-crafted initialization, not of an empty parameter set. Renaming it to \"Hand-crafted initialization (no optimization)\" would make the table easier to interpret.","section":"Table I"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the journal and addresses an interesting problem. The main issue is not the core method but the evaluation protocol and the mismatch between the \"self-labeled\" claim and the actual human involvement. I believe this is fixable within a revision by reporting unbiased policy success rates and clearly separating the human role in evaluation from the optimization process."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the actual contribution: SWIFT appears to be the first soft end-effector to do a dynamic in-hand pen spin, learned directly on hardware with no simulation and no object model. That is a real step beyond rigid high-speed hands and quasi-static soft reorientation. The parameterization (6 servo deltas, catch delay, grasp offset) is sensible, and the ablations—no grasp optimization, transferring Pen 1's parameters to Pens 2/3—convince me that optimizing the grasp point matters. Shipping code, data, and videos is genuine evidence.\n\nNow the soft spots. The big one is the evaluation protocol, and it lands where the stress-test says. The abstract says \"self-labeled trials,\" but §V-B says a human observer labels trials success/failure because the reward is \"only indirectly\" aligned. That is not self-supervised in the strong sense the abstract sells. Worse, for the brush and screwdriver they explicitly say they \"chose the first manually observed success for evaluation, rather than directly using the stored value of CMA-ES.\" That makes the 10/10 brush and 5/10 screwdriver conditional on a favorable draw, not a measure of the learned policy. For the main pens, it is unclear whether the evaluated parameters are the CMA-ES incumbent (final mean or best-by-objective) or another human-selected sample; the paper never says, and never reports the incumbent's success rate. So the \"100% after 130 actions\" headline is not backed by the evidence as written.\n\nSecond, small numbers: 10 trials per cell, no error bars, no variance. That is normal for hardware papers, but it should be stated as demonstration, not statistical proof.\n\nThird, the reset procedure is manually placing the pen in a slot and using open-loop arm motions. The paper calls this \"repeatable grasps,\" but it is an external assumption; the learned parameters are tied to that setup. That is fine for a proof of concept, but it limits the generalization claim.\n\nNone of this kills the paper. The central result—CMA-ES can find spinning parameters on a real soft hand—is plausible and visually supported. The issues are about the strength of the claims, not the existence of the phenomenon.\n\nWho is this for? Roboticists working on soft manipulation, especially dynamic in-hand tasks. It deserves peer review: yes, send it to serious referees. But the authors should be asked to report the incumbent's success rate, clarify the labeling and selection procedure, and temper the \"self-supervised\" language.","headline":"A genuinely new soft-hand dynamic manipulation result whose headline success rates are not as clean as the abstract implies; the evaluation protocol needs scrutiny before you trust the 10/10 numbers.","tokens_in":10179,"tokens_out":2399,"would_cite":true,"duration_ms":22271,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The SWIFT system shows that a soft three-finger hand can learn to spin a pen dynamically by optimizing eight interaction primitives through real-world trial and error, reaching 100% success on three pens and generalizing to a brush and a…","keywords":["soft robotics","in-hand manipulation","dynamic manipulation","pen spinning","CMA-ES","real-world learning","compliant hands","trial-and-error optimization"],"falsifier":"Run the same 130-sample optimization on visually identical pens while shifting the manual reset slot by 5–10 mm along the pen's length, and check whether the 10/10 success rate collapses; if it does, the learned skill depends on the fixed reset geometry rather than on a general dynamic manipulation capability.","tokens_in":9150,"feed_emoji":"🖊️","tokens_out":9670,"duration_ms":79567,"temperature":0.7,"pith_summary":"This paper tries to establish that a soft, compliant robotic hand can perform fast dynamic in-hand manipulation—specifically spinning a pen around a finger like the human 'Thumbaround' trick—by learning from real-world trial and error. The proposed system, SWIFT, searches over only eight action parameters (six servo targets, a catch delay, and a grasp location) with CMA-ES, using a camera-based reward that rewards rotation and penalizes drops. No simulator, object model, or prior knowledge of the pen's mass or center of mass is required. After 130 sampled actions per object, the optimized parameters spin three differently weighted pens successfully in 10 out of 10 trials each, and the same approach spins a brush 10/10 and a screwdriver 5/10. If correct, this shows that compliance need not be a barrier to high-speed manipulation, and that low-dimensional primitive search can replace accurate modeling in contact-rich tasks.","feed_headline":"A soft robot hand learns pen spinning in 130 tries","feed_subtitle":"No simulator, no object model: an eight-parameter search spins three pens 10/10 and a brush 10/10.","key_machinery":"The carrying mechanism is the reduction of pen spinning to an eight-parameter primitive space: six servo-target changes $s \\in \\mathbb{R}^6$ (two per finger on the tendon-driven MOE soft hand, each finger actuated by two servos pulling four tendons), a catch delay $d$, and a grasp displacement $g$ along the pen's length. These parameters are executed open-loop and scored by $r = r_{\\mathrm{rot}} - \\lambda p_{\\mathrm{fall}}$, where the reward accumulates the pen's rotation around the camera's $z$-axis while the pen is tracked, and the penalty counts frames where the pen is displaced from the fingers. CMA-ES, a gradient-free evolution strategy, updates the parameter distribution from each batch of real-world trials, so the search itself carries the reasoning that previous systems put into simulation or analytic models. The result is a small search space that still includes the grasp point, which the ablations show is necessary for reliable spinning across different weight distributions.","core_discovery":"The central claim is that dynamic pen spinning, usually reserved for high-speed rigid hands with analytic models or simulation-trained policies, can be learned by a soft three-finger hand from scratch in the real world. The paper demonstrates this with the SWIFT system: it optimizes an eight-dimensional action $(s,d,g)$, where $s$ are six servo angle changes, $d$ is a catch delay, and $g$ is a horizontal grasp offset along the pen. Trials are self-labeled through a camera-based objective that rewards rotation of the pen's major axis around the $z$-axis and penalizes frames where the pen is not near the fingers. After about 130 sampled actions per object, the optimized parameters succeed on all three pens (10/10 each), and the same procedure produces 10/10 on a brush and 5/10 on a screwdriver. The author's interpretation is that compliance, rather than being a limitation for fast manipulation, lets the system interact safely and repeatably while the search finds the timing and contact that make the spin work.","pith_inferences":["Inference: the optimized grasp displacement $g$ likely tracks each pen's center-of-mass offset; a direct test would be to check whether $g$ changes in the same direction as the COM positions shown for the three pens.","Inference: the same eight-parameter recipe could be applied to other fast in-hand tricks (coin flip, pencil twirl, finger pass) by changing the rotation axis in the reward and reusing CMA-ES, provided the task can be captured by a similarly small primitive family.","Inference: the manual reset and human success labels are currently hidden supervision; replacing the fixed slot with visual servoing and deriving a binary success signal from the drop penalty would make the loop fully autonomous and reveal how much of the 130-sample efficiency depends on the reset procedure.","Inference: if this scaling holds, precise dynamic models and high-speed closed-loop control are not prerequisites for fast soft manipulation; the hand's compliance can absorb the errors of open-loop primitives."],"forward_implications":["With the full eight-parameter optimization, the system reaches 10/10 success on three visually identical pens that differ in total mass and center-of-mass offset, after about 130 sampled actions per pen.","Removing the grasp-location parameter from the search reduces success to at most 7/10 on one pen and 0/10 on the other two, so searching over where to grasp is a necessary part of the discovered solution.","Action parameters found for one pen transfer poorly to another (pen 1's solution gives 7/10 on pen 3 and 0/10 on pen 2), so each object needs its own short optimization run.","The same method spins a brush successfully in 10/10 trials and a screwdriver in 5/10 trials, indicating the approach is not limited to symmetric, uniformly weighted cylinders.","Because everything is learned from real-world interactions with a camera-based reward, the approach sidesteps the sim-to-real gap that prior pen-spinning systems had to overcome."],"supporting_citations":[{"why":"provides the MOE soft hand, the tendon-driven three-finger end-effector whose compliance makes the real-world trials safe and repeatable.","marker":"[28]"},{"why":"supplies CMA-ES, the gradient-free optimizer that searches the eight action parameters from batches of real trials.","marker":"[30]"},{"why":"prior analytic-model pen spinning with a high-speed rigid hand and vision; the contrast motivating a soft, model-free approach.","marker":"[11]"},{"why":"prior simulation-based reinforcement learning for pen spinning whose sim-to-real gap motivates learning directly in the real world.","marker":"[12]"},{"why":"provides SAM 2 segmentation masks used to isolate the pen in each RGB-D frame so the rotation reward can be computed.","marker":"[29]"}],"fun_headline_variants":["Soft hand spins pens 100% with 130 real-world tries","No sim, no model: soft hand masters pen spin in 130 tries","Soft robot hand learns pen spinning via trial-and-error","Dynamic pen spin learned by soft hand in 130 attempts","SWIFT: soft hand spins pens after 130 self-labeled tries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole loop assumes every trial starts from a repeatable grasp, which is achieved by a human manually placing the pen in a fixed slot and the arm following a fixed motion sequence; if that setup variation is larger than the differences the optimizer is meant to find, the learned parameters will not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Soft hand spins pens 100% with 130 real-world tries","No sim, no model: soft hand masters pen spin in 130 tries","Soft robot hand learns pen spinning via trial-and-error","Dynamic pen spin learned by soft hand in 130 attempts","SWIFT: soft hand spins pens after 130 self-labeled tries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000222,"raw_usage":{"total_tokens":1471,"prompt_tokens":978,"completion_tokens":493,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":405}},"tokens_in":594,"tokens_out":493,"duration_ms":14578,"temperature":1.0,"reasoning_tokens":405,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:11:29.024524+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 130-sample optimization on visually identical pens while shifting the manual reset slot by 5–10 mm along the pen's length, and check whether the 10/10 success rate collapses; if it does, the learned skill depends on the fixed reset geometry rather than on a general dynamic manipulation capability.","supporting_citations":[{"cited_title":"Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation,","cited_arxiv_id":null,"evidence_quote":"supplies CMA-ES, the gradient-free optimizer that searches the eight action parameters from batches of real trials."},{"cited_title":"Dynamic manipulation like normal- type pen spinning by a high-speed robot hand and a high-speed vision system,","cited_arxiv_id":null,"evidence_quote":"prior analytic-model pen spinning with a high-speed rigid hand and vision; the contrast motivating a soft, model-free approach."},{"cited_title":"Lessons from learning to spin “pens","cited_arxiv_id":null,"evidence_quote":"prior simulation-based reinforcement learning for pen spinning whose sim-to-real gap motivates learning directly in the real world."}],"review_version":1}