{"id":"f13e9960-9900-4c3e-a3a7-7911f25ef2d2","arxiv_id":"2511.20446","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A new dataset and score-based diffusion framework generate text-conditioned 3D interactions between multiple people and a shared object.","lead":"The authors introduce a new AI task called Human-Human-Object Interaction (HHOI) and a system that generates two or more people interacting with a shared object from a text description, like 'two friends sit on a bench.' They built a new 3D motion dataset and combine separate neural models for human-object and human-human relationships into one guided sampling process, and show it beats single-human baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"HOI model ignores object mesh M despite Eq. (1) defining p^M_c; object-instance realism is untested and may not generalize beyond canonical meshes.","rationale":"The reader's weakest_assumption accurately names the most load-bearing soft spot: the model is formally conditioned on object mesh M in Eq. (1), but the implementation explicitly discards M in Sec. 3.2. This is an internal inconsistency, not a mere disagreement with prior work. It bears directly on the abstract's claims of 'realistic HHOIs' and 'object-interaction realism': the reported contact and penetration metrics are measured on the training-time canonical meshes, so they provide no evidence for generalization across object instances of the same category. If the task is instead scoped to fixed/canonical meshes, that restriction is never stated in the abstract or contributions, so the headline claim is stronger than what is demonstrated. Other issues noted by the reader (weak baselines, missing error bars, no code/data release) are real but secondary; they affect confidence in the quantitative comparisons, whereas the mesh mismatch affects whether the central capability exists beyond the exact meshes used. The verdict remains CONDITIONAL: the central framework is plausible and has independent support (fresh data collection, ablations of the guided losses, and large margins over the custom baselines), but the object-mesh assumption should be either fixed (by conditioning on geometry) or explicitly scoped before the claim of realistic, text-driven HHOI generation is accepted.","tokens_in":19289,"tokens_out":9791,"duration_ms":110906,"concrete_test":"Hold out object mesh instances within a category (train on one 'chair' mesh, test on several unseen chair meshes with different seat heights/backrests). Generate HHOIs with identical text prompts using the described pipeline and measure contact distance, penetration ratio, and user-study realism separately per unseen mesh. If these metrics materially degrade relative to the training mesh, or if the generated HOI poses are invariant to the mesh swap, the fixed-M assumption is confirmed as load-bearing. A minimal sanity check is to run the pretrained HOI sampler with two different meshes of the same category and compare outputs; identical outputs would verify that M is unused.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 3.1 defines the HOI distribution as p^M_c, conditioned on the object mesh M (Eq. 1), but Sec. 3.2 immediately drops M: \"we do not use a mesh instance M as input when modeling HOI; rather, we assume a fixed M is provided for each scenario.\" The score network Ψ_HOI therefore receives only (ϕ_t, t, c), not geometry. The text c encodes category-level semantics, not the specific seat height, backrest, or footprint of an object instance. The physical-plausibility metrics in Table 3 (contact distance, penetration) are computed against the same canonical meshes used at training, so near-zero contact distances do not demonstrate that the model understands object geometry. For a held-out instance of a seen category (e.g., a bar stool vs. a low armchair, both described as \"chair\"), the model will emit the pose distribution learned from the single training mesh; contact and penetration can be arbitrarily bad. This is not a disagreement with external consensus; it is an internal mismatch between the formal model in Eq. (1) and the implemented model, and it directly affects the claimed 'realistic HHOIs' and 'object-interaction realism' in the abstract. The claim survives only under an unstated restriction to a fixed canonical mesh per category.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Human-Human-Object Interactions (HHOI) as a new generative modeling problem: given a textual description and an object, synthesize spatially and posturally plausible configurations of multiple humans interacting with that object. The authors contribute a new multi-view captured HHOI dataset, a synthetic data augmentation pipeline using image diffusion models, and a score-based framework that first trains separate HOI and HHI diffusion models and then combines them during ODE-based sampling through an inconsistency loss and a collision loss. They also extend the framework to multi-human settings and apply it to motion in-betweening. Quantitative and qualitative comparisons against two adapted baselines are reported on dyadic and multi-human generation.","tokens_in":19658,"tokens_out":5074,"duration_ms":55082,"significance":"If the central claims hold, this is a useful step toward text-driven multi-human object-interaction synthesis, a relatively underexplored area. The decomposition into HOI and HHI models with guided sampling is a practical and reasonably principled strategy, and the new dataset plus synthetic pipeline is a concrete contribution to the community. The paper also demonstrates a downstream application to object-aware multi-human motion generation. However, the evidence for the core claims is weakened by several load-bearing gaps: the implemented HOI model does not actually consume object geometry despite the formal definition in Eq. (1), the quantitative comparisons lack variance information and significance tests, and the main realism metric is computed with the authors' own pose encoder without independent validation. These issues do not invalidate the framework, but they need to be addressed before the claimed levels of realism and robustness can be accepted.","major_comments":[{"comment":"The formal model defines p^M_c as conditioned on the object mesh M (Eq. 1), but Sec. 3.2 states: \"For simplicity, we do not use a mesh instance M as input when modeling HOI; rather, we assume a fixed M is provided for each scenario.\" The score network Ψ_HOI therefore receives only (ϕ_t, t, c), not geometry. This is an internal mismatch between the stated objective and the implementation. Since Table 3 measures contact distance and penetration against the same canonical meshes used during training, the near-zero contact distances do not demonstrate generalization to other object instances of a category. Please either condition on a mesh representation or explicitly restrict the claim to fixed canonical meshes, and evaluate on held-out object instances to test object-instance generalization.","section":"Sec. 3.1 / Sec. 3.2, Eq. (1)"},{"comment":"All quantitative results are reported as point estimates without error bars, number of seeds, or significance tests. Table 2 in particular reports 100.0% success for 3-, 4-, and 5-human generation with no variance; the 100% values are especially hard to assess given the small number of trials suggested by Fig. 3 (\"failed in 10 trials\"). To support the claim of \"consistently outperforming\" the baselines, please report mean and standard deviation over multiple seeds and, where appropriate, confidence intervals or paired tests.","section":"Tables 1–3, Sec. 5.2"},{"comment":"The primary body-pose realism metric, Body Pose FD, is computed using the authors' own body pose encoder, which is also the representation used inside the generative model (Sec. 3.1). This creates a risk of favorable bias toward the proposed method because the metric is defined in a space learned jointly with the modeling objective. Please justify why this metric is not circular, or supplement it with an independent metric such as per-joint position error or an FD computed with a fixed, independently trained pose encoder. The same concern applies to the Distance FD, where details of the distribution and normalization are not given.","section":"Sec. 5.1, Metrics"},{"comment":"The two proposed sampling losses, L_inc and L_col, are central to the unified HHOI generation framework, but they are ablated only qualitatively in Fig. 12. Without quantitative ablation on the dyadic and multi-human metrics, it is unclear how much of the improvement over the baselines in Tables 1–3 comes from these losses rather than from the underlying diffusion models. Please provide a quantitative ablation isolating the contributions of the inconsistency loss and the collision loss.","section":"Sec. 3.3, Fig. 12"}],"minor_comments":[{"comment":"Sec. 4 states that CORE4D provides object annotations for \"six object categories,\" but Appendix A.3 lists five CORE4D categories (board, box, bucket, chair, desk). Please reconcile this discrepancy.","section":"Sec. 4 vs. Appendix A.3"},{"comment":"The definition of N'_i says \"the number of HHIs where human H_i appears as a target (i.e., in pairs of the form H_i → H_j)\", but in the notation H_i → H_j, H_i is the source. The formulas in Eqs. (12)–(13) also use mixed indices (R_{H_j_n}, H_j_n) that are not clearly defined. Please clarify the indexing and the source/target convention.","section":"Sec. 3.3, Eqs. (12)–(13)"},{"comment":"There is a typo: \"We acccount generation success\" should be \"We count generation success.\" Also, \"Penetration ration\" should be \"Penetration ratio.\"","section":"Sec. 5.1"},{"comment":"The abstract says \"we train an text-to-HOI and text-to-HHI model\" — \"an\" should be \"a.\" Please proofread for similar grammatical errors.","section":"Abstract / Sec. 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is in scope for the journal and addresses an interesting new task. My recommendation is major_revision because the central claim of realistic object-interaction generation is not fully supported while the model does not condition on object geometry, and because the quantitative evidence lacks statistical grounding. The framework itself is plausible, and the issues seem addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper defines a new task, HHOI, contributes a dataset, and shows a credible way to compose separate HOI and HHI diffusion models via guided sampling. The core idea is sensible and the ablations support it, but the evaluation is thinner than the claims, and the model has a real blind spot when it comes to object geometry.\n\nWhat's new: no prior work generates text-conditioned multi-human-object interactions. CORE4D is a dataset, not a generative model, and existing HOI/HHI methods don't combine both. The decomposition into two score-based models plus the joint sampling procedure is a reasonable integration. The multi-human extension via a DAG of relative HHI constraints is a nice touch, and the dataset, though modest, adds diversity beyond CORE4D.\n\nWhat the paper does well: the guided sampling is clearly derived, the inconsistency and collision losses are well-motivated, and the ablation in Fig. 12 confirms they matter. The motion in-betweening application with DNO is a sensible demonstration.\n\nWhere the soft spots are:\n\n1. The object mesh M is dropped from the HOI model. Eq. (1) conditions the distribution on M, but Sec. 3.2 says 'we do not use a mesh instance M as input.' The score network sees only text, not geometry. Contact and penetration metrics are measured against the same canonical meshes used in training, so they say nothing about a held-out chair with a different seat height or backrest. The abstract's 'object-interaction realism' overstates what the model can do. The claim survives only if you read it as category-level text-conditioned generation with a fixed per-category mesh.\n\n2. The quantitative evidence is thin. No error bars or significance tests on Tables 1-3. The 100% success rates for 3-5 humans have no seed variance. The FD uses the authors' own body-pose encoder, which is partly circular. The baselines are adapted from single-human methods and are clearly weaker; the margins are plausible but not independently verified.\n\n3. Minor reporting mismatch: main text says 5,078 frames across 11 categories; appendix says 13,669 HOI and 13,650 HHI samples across 19 categories. These numbers can be reconciled, but the paper should state what's what.\n\n4. No code or data released, so nothing is independently reproducible.\n\nThe central argument holds up in spirit. The paper is honest about the fixed-mesh simplification in the body, and the discussion notes the dataset limitation. But the strong claims in the abstract are ahead of the evidence. With error bars, held-out object instances, and code/data release, it would be much more convincing.\n\nWho it's for: researchers in human-scene interaction, embodied AI, and social motion generation. It deserves a serious referee—the task formulation and framework are worth discussing even if the evaluation needs revision.\n\nRecommendation: send to peer review. The reviewers should push on the object-geometry issue and ask for error bars and held-out object tests, but the core contribution is real.","headline":"New task and a plausible framework, but the object-geometry blind spot and thin evaluation keep the strong claims from landing.","tokens_in":20102,"tokens_out":4167,"would_cite":true,"duration_ms":40464,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Text-conditioned synthesis of 3D scenes in which multiple people coordinate around a shared object, trained on newly captured and synthetic data.","keywords":["human-human-object interaction","text-to-motion generation","score-based diffusion","multi-human synthesis","HOI","HHI","3D scene generation","diffusion models"],"falsifier":"Hold out a novel object instance of a category seen in training (e.g., a chair with a different seat height) and generate HHOIs with the same text prompts. If human-object penetration ratios or hand/hip contact distances degrade sharply relative to objects drawn from the training distribution, the fixed-M assumption is the cause.","tokens_in":19187,"feed_emoji":"🧑🤝🧑","tokens_out":4947,"duration_ms":52417,"temperature":0.7,"pith_summary":"This paper introduces Human–Human–Object Interaction (HHOI) as a generative modeling problem: given a text prompt, produce a 3D scene where two or more people interact with each other and a shared object. Because no large-scale HHOI dataset existed, the authors captured a multi-camera dataset and built a synthetic pipeline using pretrained image diffusion models. The method trains two separate score-based diffusion models—one for single-person–object interactions (HOI) and one for two-person interactions (HHI)—then combines them in a single reverse-ODE sampling pass with consistency and collision losses. The authors report that this produces more realistic dyadic and multi-human HHOIs than existing single-human HOI approaches, and it extends naturally to 3–5 people.","feed_headline":"Text prompts now generate multi-person object scenes","feed_subtitle":"A unified diffusion sampler merges two-person and person-object interactions into coherent 3D scenes.","key_machinery":"The core mechanism is a pair of score-based diffusion models trained with denoising score matching: an HOI model operating on (rotation, translation, scale, body-pose embedding) in an object-centric frame, and an HHI model operating on (body poses, relative rotation/translation) in a person-centric frame. At inference, both are solved as a probability-flow ODE, and the two streams are coupled by an inconsistency loss that drives variance of each person's pose, scale, translation, and rotation toward zero across the HOI/HHI views, plus a collision loss computed on a 24-capsule approximation of each body. A low-dimensional body-pose embedding (126D → 10D) learned by a small MLP keeps the diffu","core_discovery":"The central claim is that a text-driven HHOI generator can be assembled from two independently trained diffusion models: one predicting a person's pose, scale, and position relative to a canonical object frame (HOI), and one predicting a second person's relative pose with respect to the first (HHI). During a single reverse-diffusion sampling process, the two estimates are reconciled by an inconsistency loss that minimizes variance across the shared human representations and a collision loss that penalizes capsule-level interpenetration. The authors argue this decomposition lets the model generalize to an arbitrary number of people by adding HOI/HHI samples, while the guided sampling ensures","pith_inferences":["If the fixed-object-instance assumption is relaxed and the HOI model is conditioned on the actual object mesh, the framework could generalize to unseen object shapes; text-only conditioning is likely the bottleneck for fine-grained contact realism.","Because the HHI set must form a directed acyclic graph, the formulation could be recast as graph generation over humans plus object, connecting naturally to scene-graph and multi-agent generation work.","The capsule-based collision loss is a modular component that could transfer to other multi-agent generation tasks (e.g., hand-object manipulation or multi-robot coordination) where physical non-penetration matters.","A testable boundary condition is the 'implicit object' case: scenes where the object is present but not directly contacted (e.g., a desk between two people). The paper's data only partially covers these, so generation quality there is an open question."],"forward_implications":["Text-to-HHOI generation becomes feasible with a unified sampling process, producing coherent dyadic scenes that outperform single-human HOI baselines on body-pose and distance Fréchet distances.","The decomposition supports scaling to 3–5 people without retraining, with reported 100% generation success in the tested multi-human settings.","Generated HHOIs can be used as end-frame constraints for motion in-betweening, enabling multi-human motion that respects object interaction.","The synthetic data pipeline, combining text-to-image models and human mesh recovery, expands HHOI coverage to outdoor and large-object scenarios that are hard to capture in a studio.","The captured and synthetic dataset itself provides a new benchmark for future HHOI research."],"fun_headline_variants":["Single diffusion step yields multi-person object scenes from text","Two models, one sampling: text-driven HHOI generation","From text to human-human-object interactions in one pass","Unified diffuser synthesizes coherent multi-human object scenes","Text prompts now generate multi-person object interactions"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The HOI diffusion model never receives the object mesh as input; it assumes a fixed object instance per scenario, so the text prompt alone must carry the object's geometry—if a new instance differs from training shapes, generated interactions may become implausible.","fun_headline_variants_meta":{"raw":{"variants":["Single diffusion step yields multi-person object scenes from text","Two models, one sampling: text-driven HHOI generation","From text to human-human-object interactions in one pass","Unified diffuser synthesizes coherent multi-human object scenes","Text prompts now generate multi-person object interactions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000325,"raw_usage":{"total_tokens":1665,"prompt_tokens":760,"completion_tokens":905,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":828}},"tokens_in":504,"tokens_out":905,"duration_ms":8980,"temperature":1.0,"reasoning_tokens":828,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T20:15:06.205584+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out a novel object instance of a category seen in training (e.g., a chair with a different seat height) and generate HHOIs with the same text prompts. If human-object penetration ratios or hand/hip contact distances degrade sharply relative to objects drawn from the training distribution, the fixed-M assumption is the cause.","supporting_citations":[],"review_version":1}