{"id":"5c7b3faa-0335-41cd-8cde-816674ffdf30","arxiv_id":"2608.07045","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":11,"one_line_summary":"C2Dex converts monocular human videos into executable dexterous robot manipulation trajectories by using stable object-side contacts as a shared representation for reconstruction and retargeting, achieving 57.78% and 26.67% trajectory success on DexYCB and TACO.","lead":"C2Dex turns ordinary videos of human hands manipulating objects into instructions a robot hand can follow, by first finding stable contact points on the object and then using those points to guide both video reconstruction and robot hand control. It reports much higher simulated success rates than prior methods on two benchmarks and shows real robot replays on eight everyday tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central success rates rest on stable contacts computed in a coordinate frame supplied by ProxyPose; the paper admits pose drift corrupts this frame but provides no sensitivity test, so the headline is unverified under its own stated failure mode.","rationale":"Reader's weakest assumption is object pose accuracy; I concur and sharpen it. The stable contact map is not an auxiliary input but the coordinate frame in which the paper's core invariant lives. The authors themselves list pose drift as a limitation, so this is not an external quibble; the manuscript is missing the experiment that would bound the impact of that limitation. The controlled substitution experiment above isolates upstream pose accuracy from the rest of the pipeline because only the object-pose source changes. A stronger version would also compare using frame-wise contacts with ground-truth object poses, but the single most decisive missing evidence is the pose perturbation curve. Other concerns (no code, no error bars, sampling procedure) are real but secondary; they affect reproducibility rather than the internal logic. The verdict remains CONDITIONAL: the method is coherent and the ablations support the role of stable contacts, but the central quantitative claim cannot be accepted as stated until pose sensitivity is quantified.","tokens_in":12320,"tokens_out":5726,"duration_ms":54412,"concrete_test":"On the 45 DexYCB sequences, re-run the full pipeline with object poses set to (a) DexYCB ground-truth object poses and (b) ground-truth poses with controlled drift (e.g., 0.5 cm and 1 deg per frame; 1 cm and 2 deg per frame), keeping all other components fixed. Recompute stable contact maps, HOI optimization, retargeting, and end-to-end ATE success. If the success rate in (b) drops toward the 17.78% baseline, or if (a) produces a substantially higher rate than the reported 57.78%, then the headline result is strongly conditioned on upstream object pose accuracy. As a secondary check, plot per-sequence success against ProxyPose object ATE to confirm that failures concentrate where pose drift is high.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Load-bearing concern: the central claim (57.78% vs 17.78% end-to-end success) depends on stable object-side contacts recovered in canonical object space and shared by the HOI optimizer (Eq. 5) and retargeter (Eq. 11). These contacts (Eq. 1) are computed by transforming frame-wise ray-cast observations through ProxyPose object poses; if those poses drift, the canonical-space aggregation is corrupted by construction. The paper states this in the Conclusion ('pose drift directly corrupts the canonical-space contact aggregation'), but offers no measurement of the effect: no ProxyPose pose-error statistics on the evaluation sequences, no correlation between object pose ATE and success/failure, and no ablation that replaces estimated object poses with ground-truth poses. This matters because Eq. (2) anchors hand vertices to possibly wrong contact targets, Eq. (7) transfers those wrong targets to the dexterous hand, and the RL stage also tracks the estimated object trajectory. A self-consistent but drifted object trajectory can therefore satisfy all internal losses while still failing the ground-truth ATE criterion, or conversely the high success rate may only hold when ProxyPose happens to be accurate. The admitted failure mode is precisely the load-bearing assumption; without a sensitivity analysis the headline gain is not yet established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript presents C2Dex, a pipeline that takes a monocular human video and produces a dexterous-hand manipulation trajectory. The key idea is to aggregate frame-wise hand-object contact observations over temporally stable segments in the canonical object frame, producing a 'stable object-side contact map' (Eq. 1). This map is used twice: as a trajectory-level constraint in HOI reconstruction (Eq. 2, Eq. 5) and as explicit retargeting targets for the dexterous hand (Eq. 7, Eq. 11), with Laplacian interaction preservation and residual RL refinement. Experiments on DexYCB and TACO report end-to-end trajectory success rates of 57.78% and 26.67%, versus 17.78% and 10.00% for the strongest baselines; ablations show large drops when the contact-consistency module is removed. Real-robot replay of 24 demonstrations across 8 tasks is reported qualitatively.","tokens_in":12607,"tokens_out":4629,"duration_ms":42932,"significance":"The stable-contact representation is a clean, well-motivated idea, and the dual-use design (the same representation for reconstruction and retargeting) is elegant. The evaluation is structured into four parts and includes an ablation study; the retargeting evaluation correctly isolates retargeting from reconstruction by using ground-truth human trajectories. If the reported success rates are robust, C2Dex would make monocular video a substantially more reliable source of dexterous-manipulation demonstrations. However, the paper currently ships no code, no data splits, no error bars or statistical tests, and no sensitivity analysis for the object-pose-estimation assumption that the authors themselves identify as load-bearing; these gaps currently limit the strength of the claim.","major_comments":[{"comment":"The stable object-side contact map is computed by transforming frame-wise ray-cast contact observations through ProxyPose object poses into canonical object space. The authors state in the Conclusion that 'pose drift directly corrupts the canonical-space contact aggregation,' but the paper provides no measurement of this effect: no ProxyPose pose-error statistics on the evaluation sequences, no correlation between object-pose error and trajectory success, and no ablation that replaces estimated object poses with ground-truth poses. Since Eq. (2) anchors MANO hand vertices to x_stab and Eq. (7) transfers the same contacts to the dexterous hand, a drifted object trajectory can satisfy all internal losses while failing the ground-truth ATE criterion. DexYCB provides ground-truth object poses, so a pose-replacement ablation is feasible on that dataset; I request it, together with pose-error curves, before the headline claim can be considered established.","section":"Sec. III-A(b), Eq. (1), Conclusion"},{"comment":"The evaluation set is described only as '45 manipulation sequences sampled from DexYCB and 30 demonstration sequences sampled from TACO.' The sampling procedure, sequence-length distribution, object/task composition, and inclusion criteria are not given. Because the main result is a count-based success rate (26/45 vs 8/45 in Table I), the choice of sequences can materially change the comparison. Please specify the exact sampling protocol, list the selected sequences, or release the evaluation split.","section":"Sec. IV-A (End-to-End Trajectory Generation)"},{"comment":"No error bars, confidence intervals, or statistical tests are reported for the trajectory success rates. With 45 and 30 trials, the difference between C2Dex and the strongest baseline on TACO (8/30 vs 3/30) is not obviously significant under a simple binomial test. Report binomial confidence intervals for each rate and, if feasible, multiple evaluation seeds or a paired test on the shared sequences.","section":"Table I and Sec. IV-A"}],"minor_comments":[{"comment":"The residual-RL stage is described at a high level, but the PPO hyperparameters, reward weights, simulation timestep, and training duration are not reported; since this stage is part of the end-to-end pipeline, these details are needed for reproducibility.","section":"Sec. III-B(d)"},{"comment":"The authors correctly note that the object-reconstruction metrics (CD, F@5) reflect the complete reconstruction systems rather than the isolated effect of contact-consistent refinement; this caveat should appear in the main text near Table II, not only in the prose, to avoid over-reading the object-side comparison.","section":"Sec. IV-C, Table II"},{"comment":"The real-robot study is explicitly qualitative, which is acceptable for a feasibility claim, but the paper should state how many of the 24 demonstrations were attempted per task and whether any trajectories failed during replay; a short success/failure table would strengthen the claim.","section":"Sec. IV-F"},{"comment":"The normal-consistency threshold gamma_n and the temporal segmentation threshold epsilon_theta are introduced but their values are never given; please report them in the implementation details.","section":"Sec. III-A(b)"},{"comment":"The definitions of E_prec and E_align do not specify how ground-truth contact regions and contact normals are determined for the retargeting evaluation; a precise description is needed for the retargeting results to be reproducible.","section":"Sec. IV-D, Table III"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper is promising but the central claim currently rests on a self-admitted failure mode that is not tested. The requested pose-sensitivity ablation is feasible on DexYCB (ground-truth object poses are available), so I see this as fixable within revision rather than grounds for rejection. I would also encourage the editor to require release of the evaluation split and, if possible, code, since the current 45/30-sequence selection is not described and the headline numbers are count-based."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious look. The core idea is clean and, as far as I can tell, genuinely new: aggregate noisy per-frame contact observations in canonical object space to get stable object-side contacts, then use those same contacts both as trajectory-level constraints for HOI reconstruction and as explicit targets for retargeting to a dexterous hand. That shared representation is the right kind of abstraction, and the ablations support it—removing cross-frame contact consistency drops the DexYCB success rate from 57.78% to 17.78%, so the mechanism is doing real work.\n\nThe method section is well specified. The losses are stated clearly, the evaluation protocol is explicit, and the comparisons with baselines under identical criteria are fair. I also credit the authors for being honest in the conclusion about the limits: severe occlusion, pose drift corrupting the canonical-space aggregation, and no support for finger gaiting. That is the right kind of limitation section.\n\nThe soft spots are the usual ones, plus one specific to this method. First, there is no code, no data splits, no error bars, and no statistical tests. The 45 and 30 sampled sequences are not described, so I can't tell whether they are representative or cherry-picked. The real-robot replay is qualitative, which is consistent with prior work but still weak. Second, and more importantly, the stress-test concern is valid: the stable contacts are computed by transforming frame-wise observations through ProxyPose object poses, and the paper admits that pose drift corrupts this representation. What the paper does not provide is any sensitivity analysis—no ProxyPose pose-error statistics on the evaluation sequences, no correlation between pose error and success, no ablation using ground-truth object poses. That is a real gap. I don't think it is a demonstrated fatal flaw: the end-to-end ATE metric already penalizes a drifted object trajectory, because the simulated object motion is compared to ground truth, so a self-consistent but wrong object trajectory would show up as a failure rather than a false success. But a high success rate without any pose-error numbers leaves the method's robustness unquantified. A ground-truth-pose ablation would settle this, and the authors should be asked to provide one.\n\nThe paper deserves peer review. It has a novel representation, coherent math, and results that are important if they hold. My recommendation: send it out, but require code or a detailed supplement, a description of the evaluation-sequence sampling, error bars, and a pose-drift sensitivity analysis before accepting.","headline":"C2Dex has a genuinely useful new idea—stable object-side contacts shared between HOI reconstruction and retargeting—but the headline numbers rest on an untested pose-drift assumption and the paper ships no code, data, or error bars.","tokens_in":13216,"tokens_out":2094,"would_cite":true,"duration_ms":20589,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"C2Dex claims that recovering stable object-side contacts in canonical object space is the missing link between monocular hand-object reconstruction and executable dexterous retargeting.","keywords":["dexterous manipulation","hand-object interaction","monocular video","contact consistency","retargeting","residual reinforcement learning","imitation learning","canonical object space"],"falsifier":"Perturb the per-frame object pose estimates in the canonical-space aggregation stage by injecting controlled drift (for example, sinusoidal additive errors of increasing amplitude on DexYCB sequences with ground-truth poses) and measure end-to-end trajectory success. If success rates remain at or near the reported 57.78% and 26.67% even when the injected drift displaces canonical contacts by more than about a centimeter, the claim that stable contacts derived from accurate object pose drive the gains would be falsified; the paper predicts a collapse in success as drift grows.","tokens_in":12067,"feed_emoji":"🦾","tokens_out":7147,"duration_ms":56625,"temperature":0.7,"pith_summary":"C2Dex claims that a single shared representation—stable object-side contacts, built by aggregating noisy per-frame contact observations in the object's own coordinate frame—can fix both halves of the video-to-robot pipeline. Guided by these contacts, hand-object reconstruction from monocular video becomes temporally coherent and physically plausible, and the same contacts become explicit targets that let a differently-shaped robot hand preserve the demonstrated grasp. The paper reports end-to-end trajectory success rates of 57.78% on DexYCB and 26.67% on TACO, against at most 17.78% and 10.00% for the strongest baselines under identical criteria, with real-robot replay on eight contact-rich tasks. If true, abundant monocular human videos become a substantially more reliable source of executable dexterous manipulation demonstrations.","feed_headline":"Stable contacts triple dexterous-hand success from video","feed_subtitle":"Anchoring contacts to the object lifts success to 57.78% and 26.67%, versus at most 17.78% and 10.00% for baselines.","key_machinery":"The central mechanism is the stable object-side contact map: per-frame hand-object contact observations, filtered by silhouette overlap and normal compatibility, are transformed into the canonical object frame using the estimated object pose, clustered with DBSCAN within locally stable temporal segments, and summarized by the cluster medoid. This map is the shared interface between the two coupled modules—contact-consistent HOI reconstruction, which uses the map as a trajectory-level loss alongside an SDF penetration penalty, and contact-interaction-preserving retargeting, which uses the same contact targets plus Laplacian optimization over a hand-object interaction graph to keep local geometry intact across embodiments, with residual RL in simulation for physical executability.","core_discovery":"The paper's central claim is that temporally unstable contacts are the root cause of both poor monocular HOI reconstruction and failed cross-embodiment retargeting, and that the cure is to anchor interactions to the object rather than to a particular hand. By projecting frame-wise contact observations into the canonical object frame, clustering them with DBSCAN over locally stable temporal segments, and taking cluster medoids as stable contact points, C2Dex obtains a contact map that is robust to per-frame noise and independent of hand embodiment. This map is used twice: as a trajectory-level contact-consistency loss that pulls the reconstructed human hand into stable contact with the object, and as an explicit retargeting target for a dexterous hand, together with a Laplacian interaction loss that preserves the local hand-object geometry and a residual-RL stage that makes the trajectory executable in simulation. The result, the paper argues, is that monocular human videos can yield executable robot trajectories at success rates several times those of prior video-to-dexterity pipelines.","pith_inferences":["The canonical-object-space contact map is a compact, embodiment-independent interaction descriptor, so it could plausibly serve as a training target or reward signal for other robot-learning setups, such as contact-aware skill segmentation or grasp affordance prediction, beyond the trajectory-retargeting pipeline studied here.","Because the representation lives on the object, it should transfer across robot hands without re-annotation; a natural stress test would be to evaluate C2Dex with two or more different target hands and check whether the same recovered contacts yield similar success rates.","Pose drift is the stated soft spot; jointly optimizing object pose along with hand articulation in the HOI stage, rather than keeping the object trajectory fixed, is a direct extension that would address the failure mode the paper flags.","The method currently targets stable contact phases; extending the clustering to handle frequent contact switching could open up in-hand manipulation and finger gaiting, which the paper lists as out of scope."],"forward_implications":["If the central claim holds, monocular internet-scale human videos become a practical demonstration source for dexterous manipulation rather than just visualization content.","End-to-end trajectory success on DexYCB would be roughly three times that of prior video-to-dexterity pipelines, making the video-to-robot route competitive with teleoperation and mocap collection for many tasks.","Reconstructed hand-object interactions become physically stable: simulation displacement drops to 1.57 cm from 4.55 cm and maximum penetration falls to 3.99 mm from 22.92 mm, so downstream uses of reconstructed HOI would inherit less physically implausible data.","Retargeting to a different hand embodiment preserves which object regions are grasped and how, not just finger positions, which is what matters for contact-rich tasks like hanging, sweeping, wiping, and dropping.","Ablations show that removing the cross-frame contact consistency collapses success to baseline levels (17.78% on DexYCB, 10.00% on TACO), indicating the stable-contact representation itself is doing the work."],"supporting_citations":[{"why":"Baseline video-to-dexterity pipeline that C2Dex must beat on end-to-end trajectory generation; supplies the strongest comparison numbers and the real-robot replay protocol.","marker":"[8]"},{"why":"Second baseline video-to-dexterity pipeline for end-to-end trajectory evaluation on DexYCB and TACO.","marker":"[9]"},{"why":"Supplies the residual-policy learning framework used to refine and execute the trajectory in simulation.","marker":"[10]"},{"why":"Dyn-HaMR supplies the initial per-frame MANO hand articulation and global transformations that the contact-consistent optimization refines.","marker":"[29]"},{"why":"SAM 3D reconstructs the canonical object mesh used for ray casting, silhouette comparison, and signed-distance collision penalties.","marker":"[31]"},{"why":"ProxyPose provides the per-frame 6D object pose; its accuracy is the load-bearing assumption for canonical-space contact aggregation.","marker":"[32]"},{"why":"DBSCAN clusters the canonical-frame contact observations within stable temporal segments to form the stable contact map.","marker":"[33]"},{"why":"Defines the interaction graph whose Laplacian coordinates are reused to preserve hand-object geometry during retargeting.","marker":"[34]"},{"why":"Applies the interaction-mesh Laplacian idea to robot-object spatial relationships; C2Dex adapts it for hand-object retargeting.","marker":"[35]"}],"fun_headline_variants":["Stable object contacts triple dexterity from monocular video","C2Dex stabilizes hand-object contacts, tripling robot success","Anchor contacts to object, not hand, for dexterous control","Monocular video to dexterous robot: stable contacts triple success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the per-frame object poses estimated by the upstream pose tracker are accurate enough that transforming contact observations into the canonical object frame yields clusters that correspond to real physical contacts; the paper itself states that pose drift directly corrupts the canonical-space contact aggregation.","fun_headline_variants_meta":{"raw":{"variants":["Stable object contacts triple dexterity from monocular video","C2Dex stabilizes hand-object contacts, tripling robot success","Anchor contacts to object, not hand, for dexterous control","Monocular video to dexterous robot: stable contacts triple success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000737,"raw_usage":{"total_tokens":3335,"prompt_tokens":1032,"completion_tokens":2303,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":648,"completion_tokens_details":{"reasoning_tokens":2228}},"tokens_in":648,"tokens_out":2303,"duration_ms":15110,"temperature":1.0,"reasoning_tokens":2228,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:49:18.341530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Perturb the per-frame object pose estimates in the canonical-space aggregation stage by injecting controlled drift (for example, sinusoidal additive errors of increasing amplitude on DexYCB sequences with ground-truth poses) and measure end-to-end trajectory success. If success rates remain at or near the reported 57.78% and 26.67% even when the injected drift displaces canonical contacts by more than about a centimeter, the claim that stable contacts derived from accurate object pose drive the gains would be falsified; the paper predicts a collapse in success as drift grows.","supporting_citations":[{"cited_title":"Do as I Do: Dexterous Manipulation Data from Everyday Human Videos","cited_arxiv_id":"2606.19333","evidence_quote":"Baseline video-to-dexterity pipeline that C2Dex must beat on end-to-end trajectory generation; supplies the strongest comparison numbers and the real-robot replay protocol."},{"cited_title":"Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the residual-policy learning framework used to refine and execute the trajectory in simulation."},{"cited_title":"ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation","cited_arxiv_id":"2607.06555","evidence_quote":"Defines the interaction graph whose Laplacian coordinates are reused to preserve hand-object geometry during retargeting."}],"review_version":1}