{"id":"2a72b01c-d1ad-4b87-b40d-65f9a7766ce2","arxiv_id":"2602.06226","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A feed-forward diffusion model jointly completes 2D occluded masks and 3D voxel geometry, trained on a new 400K-clip synthetic dataset, reconstructing hand-held objects from monocular video in ~1 minute and outperforming optimization-based baselines on HO3D and HOT3D.","lead":"ForeHOI reconstructs the complete 3D shape of an object from an ordinary monocular video of a hand using it, producing a mesh in about one minute without preprocessing. Trained only on a new 400,000-clip synthetic dataset, it reports higher benchmark scores than slower optimization-based methods, though the evaluation sets are small and hand-curated.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim rests on 6 hand-curated HOT3D clips with object-size cropping and no error bars; sim-to-real generalization to full daily HOI video is not robustly established.","rationale":"The method is well-specified: the dual-branch 2D-mask/3D-voxel completion, bidirectional cross-attention, and synthetic dataset are coherent, and the ablations (Tab. 3) internally support the design choices. I found no mathematical circularity in the losses or the evaluation protocol. The most important risk is evidential rather than formal: the headline 'state-of-the-art on daily HOI video' leans on a 6-clip HOT3D set that was manually curated for ease, and on a modest HO3D margin with no uncertainty quantification. The authors themselves concede in Future Work that the diffusion core 'poses inherent challenges for precise reconstruction'; this tempers but does not invalidate the claim. A fixed evaluation protocol on a larger, automatically selected subset would resolve whether the observed margins are real or an artifact of clip/crop choice. This is a request for a more representative benchmark, not an accusation of cherry-picking. The release of code/data is promised, making the proposed test feasible. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":16915,"tokens_out":6644,"duration_ms":72747,"concrete_test":"Run the released model (or a faithful reimplementation) on all HOT3D test clips where a tracked object appears for at least 10 frames, using a fixed automatic crop from the official hand/object tracking boxes instead of manual curation. Report per-sequence CD/F@5/F@10 and the mean with 95% CI across at least 3 diffusion seeds. If the unfiltered HOT3D mean F@5 is below ~40% or overlaps EasyHOI's CI, the Tab 2 HOT3D column cannot support the SOTA claim. Also report HO3D per-sequence margins over MagicHOI to rule out selection within the 14 sequences.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — SOTA geometry reconstruction from daily monocular HOI videos with ~100x speedup (Tab 2) — is anchored to a non-representative real-world benchmark. Sec 4.2 states that the HOT3D evaluation uses only 6 short clips, selected after 'carefully preprocess[ing]' the video, including 'sometimes cropping the video frames so that the object won't be too small.' These are exactly the easier cases for a synthetic-trained model (large, centered objects, 5–15 frames of interaction). On HO3D, the advantage over MagicHOI is modest (CD 0.79 vs 0.86; F@5 68.95 vs 64.53), with no per-sequence breakdown or seed variance reported, even though Sec 3.3 uses stochastic diffusion sampling. The large HOT3D margin over EasyHOI (F@5 60.5 vs 18.46) is the main evidence for 'significantly outperforming', yet MagicHOI and HOLD are N/A on HOT3D, so the strongest video-based competitor is absent. Thus the load-bearing premise — synthetic-only training transfers to uncurated daily HOI video — is not tested by a 6-clip hand-picked subset.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ForeHOI, a feed-forward diffusion-based model that reconstructs 3D object geometry from monocular hand-object interaction videos. The method encodes input frames with DINOv2 and WiLoR hand features, then runs a dual-branch diffusion transformer that jointly denoises a 3D object voxel latent and per-frame 2D complete object masks, with bidirectional cross-attention between the branches. The model is trained exclusively on a new synthetic dataset of 400K clips built from GraspXL grasps, Objaverse meshes, and textured MANO hands rendered in Blender. At inference, the voxel latent is upsampled with a masked structured-latent flow, and per-frame object poses are recovered by a render-and-match procedure using VGGT and Mast3R. The paper claims state-of-the-art reconstruction quality on HO3D and HOT3D, with roughly a 100x speedup over optimization-based baselines, and reports ablations supporting the hand-feature and 2D-mask-completion components.","tokens_in":17047,"tokens_out":4599,"duration_ms":53441,"significance":"If the claims are robust, this is a significant contribution: it replaces hour-scale per-object optimization with a feed-forward network, introduces a large-scale synthetic HOI video dataset, and provides evidence that joint 2D mask and 3D shape completion improves reconstruction under occlusion. The ablations in Table 3 are informative, and the dataset release would be valuable to the community. However, the headline SOTA claim rests on a very small and hand-curated HOT3D evaluation, an unspecified train/test hand-feature pipeline, and quantitative results without variance or per-sequence breakdowns. These gaps currently prevent the paper from fully supporting its advertised generalization to 'daily HOI videos.'","major_comments":[{"comment":"The HOT3D evaluation is based on 6 short clips that are 'carefully preprocessed,' including manual cropping so that the object 'won't be too small.' This is exactly the regime where a synthetic-trained model is most likely to succeed, and it contradicts the abstract's claim of eliminating pre-processing and reconstructing from 'daily' videos. Moreover, MagicHOI and HOLD, the strongest video-based competitors, are N/A on HOT3D, so the large margin over EasyHOI, a single-image method, does not establish video-based SOTA. Please provide per-sequence results on all evaluated HOT3D clips, report results on uncurated or automatically selected clips, and include any available comparisons with MagicHOI/HOLD on HOT3D.","section":"Sec. 4.2 / Table 2"},{"comment":"The training-time hand feature pipeline is unclear. Sec. 3.1 says hand features are encoded with 'a state-of-the-art hand pose estimation model [49]' (WiLoR), but Sec. 6.2 describes decoding input hand features into a hand mesh through WiLoR's optimization process. It is not stated whether WiLoR is run on the synthetic renderings during training, whether its weights are frozen, or whether the model instead uses ground-truth hand parameters from GraspXL. If training uses ground-truth hand features and inference uses WiLoR predictions, the hand prior at test time is domain-shifted and includes estimation error. This directly affects the validity of the '+ hand feats' ablation in Table 3. Please specify the exact hand-feature supervision at train and test time, and, if there is a mismatch, quantify its effect.","section":"Sec. 3.1 / Sec. 6.2"},{"comment":"The reported reconstruction metrics lack any indication of variance. The geometry branch is a stochastic conditional flow matching model, so results can depend on sampling seed, yet the table gives only point values. The HO3D improvement over MagicHOI is modest (CD 0.79 vs 0.86 cm; F@5 68.95 vs 64.53). Without per-sequence breakdowns or error bars over multiple sampling runs, it is impossible to assess whether this difference is meaningful. Please report per-sequence scores and means over at least a few seeds.","section":"Table 2, Sec. 3.3"},{"comment":"The object pose comparison is not apples-to-apples. Dynhor is evaluated using only its first-step pose because its refinement is not open-sourced, and HOLD is initialized from SfM poses, which are known to be unreliable in this setting. The resulting pose numbers (RPE 1.42 cm, 2.64 deg vs 4.25 cm, 5.25 deg for Dynhor) may overstate the advantage. Either obtain the final Dynhor output from the authors, or clearly state this limitation next to Table 1.","section":"Table 1"}],"minor_comments":[{"comment":"The loss term contains a typo: `(ϵ−x 0])` has an unmatched bracket; should be `(ϵ - x_0)`.","section":"Eq. (2)"},{"comment":"The caption says 'HOI dataset' but the second block reports on HOT3D. Use the dataset name consistently.","section":"Table 2 caption"},{"comment":"The claim of 'eliminating the need for any pre-processing steps' is too strong given the manual curation, undistortion, and cropping of HOT3D clips described in Sec. 4.2. Please soften or explicitly separate training/inference preprocessing from evaluation-protocol preprocessing.","section":"Abstract vs Sec. 4.2"},{"comment":"Wording errors: 'Our method outperform all the baseline method' should be 'Our method outperforms all baseline methods.' Also, 'the resulted textured mesh has a high degree of randomness, thus influences' should be 'thus influencing.'","section":"Sec. 4.4"},{"comment":"The paper does not specify the number of diffusion sampling steps, the GPU used for inference-time timing, or the breakdown of the 1.1 min (e.g., voxel denoising, SLat decoding, pose refinement). Please include these details.","section":"Throughout"},{"comment":"Reference formatting appears corrupted in places (e.g., [1] ends with '1', [54] with '1', [43] with '1'). Please regenerate the bibliography.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper has a valuable dataset and a plausible architecture, but the central 'SOTA from daily HOI videos' claim is currently anchored to a 6-clip hand-curated HOT3D evaluation and an underspecified hand-feature training protocol. I did not find mathematical circularity; the concerns are empirical and evidential. With a more rigorous evaluation protocol and clarification of the hand-feature pipeline, this could become a strong acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the pipeline: a feed-forward diffusion model that jointly completes 2D object masks and 3D voxels, conditioned on hand features, trained on a large synthetic HOI dataset. That gets object geometry in about a minute versus hours for optimization baselines, and the design choices are well supported by ablations. The synthetic dataset itself is a real asset — first HOI video dataset at this scale, built on GraspXL and Objaverse, with pose and depth annotations. The pose refinement via render-and-match is also clever, and the pose numbers (ATE 0.13 vs 0.24 for Dynhor) are strong. I found no mathematical circularity; the targets are supervised from ground truth, and the authors are honest about limitations, including the Dynhor baseline handicap and the inherent precision limits of diffusion in their future work section.\n\nThe soft spots are mostly about evidence, not method. The HOT3D evaluation uses 6 hand-curated clips, and the authors admit to cropping frames so the object won't be too small. That is exactly the kind of selection that favors a synthetic-trained model, and it makes the big margin over EasyHOI (F@5 60.5 vs 18.46) hard to trust. MagicHOI and HOLD are N/A on HOT3D, so the strongest video competitor is absent. On HO3D, the improvement over MagicHOI is modest (CD 0.79 vs 0.86; F@10 93.7 vs 91.9) and there are no error bars anywhere, despite stochastic diffusion sampling. The sim-to-real transfer claim is load-bearing: training only on synthetic data and transferring to real video is the whole point, but 14 HO3D sequences plus 6 curated HOT3D clips is a thin test. The ablation showing their dataset beats generic multi-view data is suggestive but not a substitute for a broader real-data evaluation. Code and data are promised but not verifiable here, and the loss-balance beta is never reported.\n\nOverall, the central claim — feed-forward reconstruction under severe hand occlusion — holds up as a proof of concept, and the speedup is real. The method is coherent and the ablation logic is sound. But the \"significantly outperforming\" headline is not yet supported by the real-data benchmarks as presented. This is worth serious peer review, but the authors should be pushed to evaluate on more representative HOT3D sequences or a standard split, run multiple seeds, and make code/data available. If those hold, this is a useful contribution to the field.\n\nRecommendation: send it to review, with requests for stronger evaluation and reproducibility.","headline":"Plausible feed-forward alternative to optimization for HOI reconstruction, but the SOTA claim rests on a thin, curated real-data evaluation.","tokens_in":17789,"tokens_out":2215,"would_cite":true,"duration_ms":62554,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"From hand videos to 3D object models in one minute","keywords":["3D object reconstruction","hand-object interaction","feed-forward model","diffusion model","2D mask inpainting","3D shape completion","monocular video","synthetic dataset"],"falsifier":"Evaluate ForeHOI on a large, uncurated set of real hand-object videos with ground-truth scans, evenly spanning object categories, hand sizes, and viewpoints, and compare chamfer distance and F-scores against the figures reported for the curated benchmarks; a disproportionate drop on objects or viewpoints absent from the synthetic training set would show the sim-to-real claim is limited.","tokens_in":16589,"feed_emoji":"🖐️","tokens_out":6370,"duration_ms":63680,"temperature":0.7,"pith_summary":"The paper aims to establish that severely occluded hand-held objects can be reconstructed in 3D from short monocular video by a single feed-forward network, eliminating the hours-long per-object optimization used by prior methods. Its core claim is that predicting complete 2D object masks and 3D shape jointly, in a generative model that lets information flow between the two prediction tasks, resolves the occlusion problem. The authors train the model on a large synthetic dataset of hand-object interactions—400,000 clips—and report state-of-the-art accuracy on public benchmarks with roughly a hundredfold speedup, in about one minute per video. If correct, this makes 3D digitization of everyday objects from casual video practical for embodied AI, AR/VR, and 3D content creation.","feed_headline":"From hand videos to 3D object models in one minute","feed_subtitle":"A bidirectional 2D/3D completion network beats hour-long optimization at 100x speed.","key_machinery":"The key mechanism is a two-branch generative network with bidirectional cross-attention. One branch denoises per-view complete 2D object masks; the other denoises the 3D voxel latent. At each block, mask-branch features are fed into the geometry branch (replacing plain image features) and geometry features are fed back into the mask branch, so contour information guides volume completion and vice versa. This joint 2D/3D completion is what the paper argues overcomes severe hand-induced occlusion.","core_discovery":"ForeHOI is a feed-forward generative network—specifically, a diffusion model that learns to denoise random noise into meaningful structure—that takes a handful of RGB frames of a hand holding an object and directly outputs the object's complete 3D geometry. The central discovery is that two complementary completion tasks—inpainting the per-frame 2D object masks and filling the occluded parts of a 3D voxel grid—mutually reinforce each other when coupled through bidirectional cross-attention inside the denoising network. A hand-pose feature tells the model where occlusions come from, and the joint training on a purpose-built synthetic dataset lets it hallucinate unobserved object surfaces. At","pith_inferences":["A likely implication the paper does not develop: the bidirectional 2D/3D completion mechanism is a general template for any reconstruction task with structured occlusion—e.g., objects partially hidden by vegetation or medical tools—and could be tested by retraining on such data.","If the sim-to-real transfer is as strong as reported, one testable extension is scaling the synthetic data with more hand shapes, lighting, and object categories; the prediction is that real-world accuracy rises without any real-data fine-tuning.","The paper leaves pose estimation as a separate render-and-match post-process; integrating pose estimation into the feed-forward network itself is a natural next step and would likely remove the remaining overhead."],"forward_implications":["Reconstruction of hand-held objects from casual monocular video becomes an interactive-time operation (about one minute), making 3D asset creation from daily footage practical.","The joint 2D-mask/3D-shape completion recipe could be applied to other occlusion-heavy reconstruction tasks, such as objects behind tools or foreground clutter.","The synthetic dataset of 400,000 hand-object interaction clips supports training without real 3D annotations, which could lower the cost of future hand-object models.","Object pose estimation emerges from the reconstructed mesh, so a single model can provide both geometry and trajectory for downstream tasks.","If the speed and accuracy hold, applications like robot grasp planning, egocentric AR annotations, and interactive 3D modeling from phone videos become feasible."],"fun_headline_variants":["Hand videos to 3D objects in 1 minute","Reconstruct 3D objects from hand videos at 100x speed","ForeHOI: one-minute 3D object reconstruction from hand videos","Joint 2D/3D completion turns hand videos into 3D objects","Feed-forward 3D object reconstruction beats optimization by 100x"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The model is trained exclusively on synthetic renderings of hand grasps and is expected to transfer to real monocular video with no real-data fine-tuning; if that sim-to-real transfer is narrower than the chosen benchmark clips suggest, the claimed general performance would not hold.","fun_headline_variants_meta":{"raw":{"variants":["Hand videos to 3D objects in 1 minute","Reconstruct 3D objects from hand videos at 100x speed","ForeHOI: one-minute 3D object reconstruction from hand videos","Joint 2D/3D completion turns hand videos into 3D objects","Feed-forward 3D object reconstruction beats optimization by 100x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000437,"raw_usage":{"total_tokens":2079,"prompt_tokens":783,"completion_tokens":1296,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":1202}},"tokens_in":527,"tokens_out":1296,"duration_ms":9889,"temperature":1.0,"reasoning_tokens":1202,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T03:59:04.012432+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate ForeHOI on a large, uncurated set of real hand-object videos with ground-truth scans, evenly spanning object categories, hand sizes, and viewpoints, and compare chamfer distance and F-scores against the figures reported for the curated benchmarks; a disproportionate drop on objects or viewpoints absent from the synthetic training set would show the sim-to-real claim is limited.","supporting_citations":[],"review_version":1}