{"id":"760c7fe9-5f54-4857-bd23-a5554382a82f","arxiv_id":"2509.09555","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"InterAct is a unified 21.81-hour 3D human-object interaction benchmark with text annotations, quality-corrected data, and a multi-task model that achieves state-of-the-art results across six generation tasks.","lead":"The authors assembled over 21 hours of 3D human-object interaction motion from seven public datasets, cleaned it up with an optimization pipeline, and added text descriptions. They then built six benchmark tasks and a unified model that outperforms existing single-task methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hand-correction contact indicator is computed from the same inaccurate hand-object distances the correction aims to fix, and the reported quality gains are measured by the optimization objectives themselves; the benchmark's foundational data-quality claim is therefore not independently supported.","rationale":"The paper makes a plausible and useful contribution: it consolidates seven HOI datasets, standardizes representations and annotations, and proposes a unified generative formulation. The marker-based representation and the cross-dataset unification effort are genuine advances, and the interaction-imitation success-rate improvement is real independent evidence that the correction has some positive effect. However, the reader's weakest-assumption analysis pinpoints the most load-bearing vulnerability: the contact indicator that gates hand correction is derived from the same erroneous distances the correction is supposed to repair. The paper's own limitations section confirms that large-distance floating artifacts are identified as no-contact and left uncorrected. Because the quantitative correction metrics mirror the optimized losses, they cannot settle the question. This is not an internal inconsistency or a disagreement with consensus; it is an unvalidated empirical assumption at the core of the benchmark's quality claim. The proposed test—precision/recall of c_i against independent annotations—would settle it. Since the reader already returned CONDITIONAL and my concern is the same weakness rather than a new one, the verdict should remain unchanged; the paper should be accepted only if the data-quality validation is supplied or the claim is relaxed.","tokens_in":22777,"tokens_out":5053,"duration_ms":68626,"concrete_test":"Sample 200 frames from the BEHAVE and OMOMO subsets after correction; obtain independent contact labels by (a) manual annotation of the original RGB/D frames by two raters, or (b) a markerless hand-object contact estimator not trained on InterAct's corrected outputs. Compute the precision and recall of c_i from supplementary Eq. 4 against these labels at the pre-correction hand-object distance. If recall of true contacts with pre-correction distance > 10 cm is substantially below 90%, the optimizer is leaving true contacts unforced; if precision is substantially below 90%, it is manufacturing contacts. Either outcome would materially weaken the benchmark-quality claim; high precision and recall would validate the correction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"InterAct's value as a foundational benchmark depends on the correction pipeline producing physically faithful contacts. In Sec. 3.2, the hand-correction contact loss promotes contact only where the contact indicator c_i (supplementary Eq. 4) is positive, and c_i is a piecewise function of min_j d_j[i]—the hand-object distance computed from the very poses being corrected. For source sequences whose hand poses are inaccurate (the paper's own motivation for correction, citing BEHAVE and OMOMO), a hand that truly contacts the object but is reconstructed more than 10 cm away is assigned c_i=0 and is never pulled into contact; conversely, a noisy near-but-not-touching hand is assigned c_i>0 and pushed into the object. The paper's Sec. E concedes exactly this failure mode for floating objects: large distances may be identified as no contact and remain uncorrected. Table 2's quality metrics—penetration depth and contact ratio—are the same quantities minimized and maximized by the optimization objectives (E_pene and E_cont), so the reported improvement is partly circular. The user study (30 tuples, preference ranking) is small and does not isolate frame-level contact correctness. One independent signal, the physics-based imitation success gain (90.7% vs. 84.4% on corrected vs. raw), suggests the correction helps globally, but it does not validate per-contact localization. Thus the central claim that InterAct supplies reliably corrected, high-quality HOI data—on which both the benchmark and downstream SOTA claims rest—is weakest precisely where the correction decides what counts as contact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces InterAct, a large-scale 3D human-object interaction (HOI) benchmark built by consolidating seven existing datasets (21.81 h of base data, expanded to 30.70 h via optimization-based augmentation), with unified marker-based human representations and detailed text/action annotations. It also proposes a unified multi-task transformer/diffusion framework covering six HOI generation tasks (text-to-interaction, action-to-interaction, object-to-human, human-to-object, interaction prediction, and interaction imitation). A three-stage optimization pipeline is used for full-body correction, hand correction, and interaction augmentation. Experiments report improved contact/penetration statistics, state-of-the-art results on language- and action-conditioned tasks, scaling benefits for interaction prediction, and an improved physics-based imitation success rate.","tokens_in":23169,"tokens_out":3682,"duration_ms":45289,"significance":"If the data-quality and augmentation claims hold, InterAct would be a substantial community resource: it standardizes heterogeneous HOI data, supplies text annotations at scale, defines six benchmarking tasks, and releases data and models. The marker-based representation and the multi-task formulation with contact prediction are useful methodological contributions. The strongest independent evidence is the physics-based imitation success gain (90.7% vs. 84.4% on corrected vs. raw data), which suggests the correction helps global control. However, the central data-quality claim rests partly on metrics that coincide with the optimization objectives, so the benchmark's foundational value needs additional independent validation.","major_comments":[{"comment":"The two quantitative quality metrics reported in Table 2—penetration depth and contact ratio—are exactly the quantities minimized and maximized by the optimization losses E_pene and E_cont (Sec. 3.2, Supplementary Eqs. 1 and 4). Reporting improvements on these metrics after optimization is partly a check that the optimizer did what it was asked, not independent evidence of physical correctness. The user study uses only 30 tuples and only the BEHAVE and OMOMO subsets, so it is too small to establish per-contact localization across the seven datasets. I recommend independent validation: manual contact labels on a held-out subset, comparison against a separate capture modality, or a downstream metric not used in the correction objective.","section":"Sec. 5.1, Table 2"},{"comment":"The contact indicator c_i is a piecewise function of min_j d_j[i], the hand-object distance computed from the very poses being corrected. If the initial reconstruction places a truly touching hand more than epsilon_2 = 0.10 m away, c_i = 0 and no contact-promotion gradient is applied; conversely, a noisy near-but-not-touching hand can be pushed into the object. The paper's own limitation section (Sec. E) concedes that large distances may be identified as no contact and remain uncorrected. Since the motivation for correction is precisely that source hand poses are inaccurate (BEHAVE, OMOMO), this circularity weakens the central data-quality claim. Please validate the indicator against an independent source, e.g., manual contact annotation or a separate hand tracker.","section":"Sec. 3.2, Supplementary Eq. (4)"},{"comment":"The FID and MM-Dist feature encoder used for evaluation is trained on InterAct corrected data and then used to score models trained on the same corrected data. This is acceptable for ranking design variants within the paper, but it cannot fully support the absolute 'state-of-the-art' claim or the data-quality claim, because the encoder may have learned the specific correction statistics and artifacts of InterAct. Please report results with a fixed feature extractor not trained on the corrected data, or at least demonstrate that the relative FID ranking is stable across different feature extractors.","section":"Sec. 5.2, Tables 3 and 4"},{"comment":"The 'contact invariance' principle is an assumed axiom: displaced object trajectories with re-optimized human motion are asserted to preserve interaction semantics. The filtering step removes some low-quality augmentations, but no metric directly demonstrates semantic equivalence for the retained augmented data. The physical imitation success gain is useful but is reported on only four sequences and does not isolate augmentation quality. Please provide an ablation that trains on corrected data with and without augmentation, or a per-sample semantic-preservation evaluation (automatic or human).","section":"Sec. 3.2, Interaction Augmentation"}],"minor_comments":[{"comment":"The table formatting is very hard to read: the check marks and numeric columns are run together (e.g., '× ×0.017 0.048 22.3'). Please reformat with clear column separators so each row unambiguously shows the configuration and corresponding metric values.","section":"Table 2"},{"comment":"The sentence 'We only select sequences that contain rigid objects' appears twice verbatim in the Chairs processing paragraph. Please remove the duplicate.","section":"Supplementary Sec. B.2"},{"comment":"Typo: 'Addtional Implementation Details' should be 'Additional Implementation Details'.","section":"Supplementary Sec. C.6"},{"comment":"The claim of being 'the most extensive 3D HOI benchmark' is based on 21.81 h of base data, with 30.70 h including synthetic augmentation. Please state the base versus augmented split clearly in the abstract or contribution list to avoid overstating the captured data volume.","section":"Abstract and Sec. 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong fit for a computer-vision/graphics venue and the resource contribution is potentially valuable. The main risk is that the data-quality evaluation is partially circular; the authors need to provide at least one independent validation of correction quality. If they cannot, the paper should be treated as a method/benchmark paper whose data claims are weaker than advertised. The multi-task results and scaling analysis are solid but do not resolve this issue by themselves."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nInterAct is worth knowing about: it merges seven HOI datasets into one standardized benchmark with 21.8 hours of motion, 34k text descriptions, a unified marker representation, and six benchmark tasks. The consolidation and annotation effort is genuinely new and likely useful for the HOI generation subfield. The marker-based representation, with SMPL-X/SMPL-H correspondence error under 1 cm, is sensible. The multi-task diffusion model trained on this data shows real gains on some tasks, e.g., FID 2.161 vs 3.566 for action-conditioned, and the imitation success improvement (90.7% vs 84.4% on corrected vs raw data) is an external signal that the correction helps overall.\n\nThe main soft spot is the circularity in the data-quality evaluation. Table 2's metrics—penetration depth and contact ratio—are exactly the quantities optimized by the correction losses. Reporting that the optimized data scores better on these metrics is partly baked in. The stress-test point about the contact indicator in Eq. 4 is also valid: c_i is computed from min_j d_j[i], the hand-object distance in the very poses being corrected. If a hand is truly touching but reconstructed more than 10cm away, c_i=0 and it never gets pulled into contact; the paper's own Sec. E concedes this failure mode for floating objects. So the benchmark's foundational data-quality claim—that the correction reliably fixes artifacts—is not independently established. The user study is small (30 tuples) and only covers BEHAVE and OMOMO. The FID evaluation also relies on a feature extractor trained on the corrected data, which is another mild circularity.\n\nThat said, the paper's core value doesn't rest entirely on the correction being perfect. The standardized benchmark itself, with text annotations and six tasks, is a reproducible resource—once the data and code actually ship. Right now the abstract says 'publicly available', but the GitHub releases and dataset access are not yet in place, which is a real gap for verification.\n\nThis paper deserves a serious referee. The benchmark effort is significant, the writing is honest about limitations, and the methodological pieces, while drawing on known ideas, are combined in a new way. The referee should push for: independent validation of the correction (e.g., human annotation or external contact ground truth), a larger user study covering more datasets, more than one baseline per task, and actual data release.\n\nI'd bring it to a reading group focused on HOI or generative motion benchmarks, and I'd cite it if I worked on HOI generation. Recommend: send to peer review, expect revisions.","headline":"The consolidated benchmark is a real contribution, but the correction-quality gains are partly measured by its own objectives and the hand-contact indicator inherits the inaccuracies it is meant to fix.","tokens_in":23627,"tokens_out":2260,"would_cite":true,"duration_ms":24211,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InterAct claims to be the largest 3D human-object interaction benchmark to date, and shows a unified model reaching state-of-the-art on six generation tasks.","keywords":["3D human-object interaction","HOI benchmark","motion generation","text-to-interaction","contact correction","data augmentation","multi-task diffusion","marker-based representation"],"falsifier":"Re-capture a subset of corrected sequences with a high-fidelity hand tracking system (e.g., magnetic or glove-based) and measure whether the corrected hand poses are actually closer to this ground truth than the raw poses; if not, or if regions the indicator labeled non-contact are revealed to be in genuine contact, the correction claim is falsified. A simpler check: train the same generation model on raw versus corrected data; if raw-trained FID or contact metrics match or beat corrected-trained ones, the correction step is not delivering its claimed benefit.","tokens_in":22690,"feed_emoji":"🤝","tokens_out":4407,"duration_ms":50380,"temperature":0.7,"pith_summary":"The paper argues that the main bottleneck in 3D human-object interaction generation is data, not models, and presents InterAct as the largest such benchmark: 21.81 hours of standardized, text-annotated motion from seven existing datasets, expanded to 30.70 hours through a contact-preserving augmentation. It claims that a unified optimization pipeline removes common MoCap artifacts such as penetration, floating, and faulty hand poses, and that a single multi-task diffusion model trained on this data achieves state-of-the-art results across six tasks, including text-to-interaction and physics-based imitation. If true, the work gives the field a common data resource, a correction recipe, and a baseline, making interaction generation research more reproducible and more capable of handling new objects.","feed_headline":"One dataset trains six 3D human-object interaction tasks","feed_subtitle":"InterAct unifies 21.81 hours of MoCap data and beats prior methods on all six benchmark tasks.","key_machinery":"The method rests on four pieces: a marker-based human representation chosen because contact happens on the body surface, not at joints; a contact indicator ci that gates where the optimizer enforces hand-object contact; contact-invariance augmentation, which displaces the object and re-optimizes the body so that close vertex pairs stay close (the Ealign distance-matrix loss); and a multi-task output eta giving each marker's vector to the object surface, which lets one diffusion model handle text-, action- and object-conditioned generation jointly.","core_discovery":"On its own terms, the paper's central claim is that heterogeneous, artifact-laden human-object interaction recordings can be turned into a large, clean, richly annotated resource by (1) unifying human representation with surface markers, (2) correcting contacts and hand poses through gradient-based optimization, and (3) applying the principle of contact invariance to synthesize new training sequences at no extra MoCap cost. The paper then claims that a multi-task transformer-diffusion model, given this resource, outperforms existing approaches on all six benchmark tasks, and that corrected data improve downstream imitation success from 84.4% to 90.7%.","pith_inferences":["The contact-invariance augmentation principle is stated for single human-object pairs, but the same distance-matrix alignment could be applied to multi-object scenes or human-human-object interactions, as long as the contact graph is preserved.","The geometric contact indicator could be replaced by a learned contact prior trained on a small, high-accuracy hand tracking dataset; if that prior corrected mislabeled contacts, the pipeline might fix the very failures it currently leaves untouched.","Because the interaction-aware text encoder is trained with contrastive learning on 30 hours of paired text-motion data, it may transfer to video-based HOI retrieval or captioning tasks that currently lack paired 3D data."],"forward_implications":["A single transformer-diffusion model, trained once on InterAct-X, covers five kinematic generation tasks plus imitation, replacing task-specific pipelines.","Physics-based imitation policies trained on corrected data succeed 90.7% of the time versus 84.4% on raw data, so higher-quality inputs translate into better control.","Training the interaction prediction model on three copies of the dataset lowers global per-marker error from 0.106 m to 0.091 m, suggesting the benchmark supports scaling-law-style improvements.","Marker representations reduce penetration artifacts compared with joint- or SMPL-based representations, giving other HOI methods a simple drop-in representation.","The benchmark's 217 objects and six tasks establish a common evaluation ground for future HOI generation work."],"fun_headline_variants":["One dataset, six tasks, all state-of-the-art","Cleaned MoCap lifts imitation success to 90.7%","From 21.8h messy to 30.7h clean HOI benchmark","Contact invariance expands HOI data to 30.7 hours"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The correction stage decides where to enforce contact using a contact indicator computed from the hand-object distances in the very data it is correcting; the paper itself notes (Limitations, Sec. E) that large distances may be classified as no-contact and left uncorrected, so the pipeline's success depends on the indicator labeling contact correctly despite the source data being faulty.","fun_headline_variants_meta":{"raw":{"variants":["One dataset, six tasks, all state-of-the-art","Cleaned MoCap lifts imitation success to 90.7%","From 21.8h messy to 30.7h clean HOI benchmark","Contact invariance expands HOI data to 30.7 hours"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002128,"raw_usage":{"total_tokens":8090,"prompt_tokens":727,"completion_tokens":7363,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":7287}},"tokens_in":471,"tokens_out":7363,"duration_ms":56071,"temperature":1.0,"reasoning_tokens":7287,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:51:58.744557+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-capture a subset of corrected sequences with a high-fidelity hand tracking system (e.g., magnetic or glove-based) and measure whether the corrected hand poses are actually closer to this ground truth than the raw poses; if not, or if regions the indicator labeled non-contact are revealed to be in genuine contact, the correction claim is falsified. A simpler check: train the same generation model on raw versus corrected data; if raw-trained FID or contact metrics match or beat corrected-trained ones, the correction step is not delivering its claimed benefit.","supporting_citations":[],"review_version":1}