Pith. sign in

REVIEW 4 major objections 4 minor 85 references

Text-conditioned synthesis of 3D scenes in which multiple people coordinate around a shared object, trained on newly captured and synthetic data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 20:15 UTC pith:N5A3XNH7

load-bearing objection New task and a plausible framework, but the object-geometry blind spot and thin evaluation keep the strong claims from landing. the 4 major comments →

arxiv 2511.20446 v3 pith:N5A3XNH7 submitted 2025-11-25 cs.CV

Learning to Generate Human-Human-Object Interactions from Textual Descriptions

Jeonghyeon Na , Sangwon Baik , Inhee Lee , Junyoung Lee , Hanbyul Joo This is my paper
classification cs.CV
keywords human-human-object interactiontext-to-motion generationscore-based diffusionmulti-human synthesisHOIHHI3D scene generationdiffusion models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces Human–Human–Object Interaction (HHOI) as a generative modeling problem: given a text prompt, produce a 3D scene where two or more people interact with each other and a shared object. Because no large-scale HHOI dataset existed, the authors captured a multi-camera dataset and built a synthetic pipeline using pretrained image diffusion models. The method trains two separate score-based diffusion models—one for single-person–object interactions (HOI) and one for two-person interactions (HHI)—then combines them in a single reverse-ODE sampling pass with consistency and collision losses. The authors report that this produces more realistic dyadic and multi-human HHOIs than existing single-human HOI approaches, and it extends naturally to 3–5 people.

Core claim

The central claim is that a text-driven HHOI generator can be assembled from two independently trained diffusion models: one predicting a person's pose, scale, and position relative to a canonical object frame (HOI), and one predicting a second person's relative pose with respect to the first (HHI). During a single reverse-diffusion sampling process, the two estimates are reconciled by an inconsistency loss that minimizes variance across the shared human representations and a collision loss that penalizes capsule-level interpenetration. The authors argue this decomposition lets the model generalize to an arbitrary number of people by adding HOI/HHI samples, while the guided sampling ensures

What carries the argument

The core mechanism is a pair of score-based diffusion models trained with denoising score matching: an HOI model operating on (rotation, translation, scale, body-pose embedding) in an object-centric frame, and an HHI model operating on (body poses, relative rotation/translation) in a person-centric frame. At inference, both are solved as a probability-flow ODE, and the two streams are coupled by an inconsistency loss that drives variance of each person's pose, scale, translation, and rotation toward zero across the HOI/HHI views, plus a collision loss computed on a 24-capsule approximation of each body. A low-dimensional body-pose embedding (126D → 10D) learned by a small MLP keeps the diffu

Load-bearing premise

The HOI diffusion model never receives the object mesh as input; it assumes a fixed object instance per scenario, so the text prompt alone must carry the object's geometry—if a new instance differs from training shapes, generated interactions may become implausible.

What would settle it

Hold out a novel object instance of a category seen in training (e.g., a chair with a different seat height) and generate HHOIs with the same text prompts. If human-object penetration ratios or hand/hip contact distances degrade sharply relative to objects drawn from the training distribution, the fixed-M assumption is the cause.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Text-to-HHOI generation becomes feasible with a unified sampling process, producing coherent dyadic scenes that outperform single-human HOI baselines on body-pose and distance Fréchet distances.
  • The decomposition supports scaling to 3–5 people without retraining, with reported 100% generation success in the tested multi-human settings.
  • Generated HHOIs can be used as end-frame constraints for motion in-betweening, enabling multi-human motion that respects object interaction.
  • The synthetic data pipeline, combining text-to-image models and human mesh recovery, expands HHOI coverage to outdoor and large-object scenarios that are hard to capture in a studio.
  • The captured and synthetic dataset itself provides a new benchmark for future HHOI research.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the fixed-object-instance assumption is relaxed and the HOI model is conditioned on the actual object mesh, the framework could generalize to unseen object shapes; text-only conditioning is likely the bottleneck for fine-grained contact realism.
  • Because the HHI set must form a directed acyclic graph, the formulation could be recast as graph generation over humans plus object, connecting naturally to scene-graph and multi-agent generation work.
  • The capsule-based collision loss is a modular component that could transfer to other multi-agent generation tasks (e.g., hand-object manipulation or multi-robot coordination) where physical non-penetration matters.
  • A testable boundary condition is the 'implicit object' case: scenes where the object is present but not directly contacted (e.g., a desk between two people). The paper's data only partially covers these, so generation quality there is an open question.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Human-Human-Object Interactions (HHOI) as a new generative modeling problem: given a textual description and an object, synthesize spatially and posturally plausible configurations of multiple humans interacting with that object. The authors contribute a new multi-view captured HHOI dataset, a synthetic data augmentation pipeline using image diffusion models, and a score-based framework that first trains separate HOI and HHI diffusion models and then combines them during ODE-based sampling through an inconsistency loss and a collision loss. They also extend the framework to multi-human settings and apply it to motion in-betweening. Quantitative and qualitative comparisons against two adapted baselines are reported on dyadic and multi-human generation.

Significance. If the central claims hold, this is a useful step toward text-driven multi-human object-interaction synthesis, a relatively underexplored area. The decomposition into HOI and HHI models with guided sampling is a practical and reasonably principled strategy, and the new dataset plus synthetic pipeline is a concrete contribution to the community. The paper also demonstrates a downstream application to object-aware multi-human motion generation. However, the evidence for the core claims is weakened by several load-bearing gaps: the implemented HOI model does not actually consume object geometry despite the formal definition in Eq. (1), the quantitative comparisons lack variance information and significance tests, and the main realism metric is computed with the authors' own pose encoder without independent validation. These issues do not invalidate the framework, but they need to be addressed before the claimed levels of realism and robustness can be accepted.

major comments (4)
  1. [Sec. 3.1 / Sec. 3.2, Eq. (1)] The formal model defines p^M_c as conditioned on the object mesh M (Eq. 1), but Sec. 3.2 states: "For simplicity, we do not use a mesh instance M as input when modeling HOI; rather, we assume a fixed M is provided for each scenario." The score network Ψ_HOI therefore receives only (ϕ_t, t, c), not geometry. This is an internal mismatch between the stated objective and the implementation. Since Table 3 measures contact distance and penetration against the same canonical meshes used during training, the near-zero contact distances do not demonstrate generalization to other object instances of a category. Please either condition on a mesh representation or explicitly restrict the claim to fixed canonical meshes, and evaluate on held-out object instances to test object-instance generalization.
  2. [Tables 1–3, Sec. 5.2] All quantitative results are reported as point estimates without error bars, number of seeds, or significance tests. Table 2 in particular reports 100.0% success for 3-, 4-, and 5-human generation with no variance; the 100% values are especially hard to assess given the small number of trials suggested by Fig. 3 ("failed in 10 trials"). To support the claim of "consistently outperforming" the baselines, please report mean and standard deviation over multiple seeds and, where appropriate, confidence intervals or paired tests.
  3. [Sec. 5.1, Metrics] The primary body-pose realism metric, Body Pose FD, is computed using the authors' own body pose encoder, which is also the representation used inside the generative model (Sec. 3.1). This creates a risk of favorable bias toward the proposed method because the metric is defined in a space learned jointly with the modeling objective. Please justify why this metric is not circular, or supplement it with an independent metric such as per-joint position error or an FD computed with a fixed, independently trained pose encoder. The same concern applies to the Distance FD, where details of the distribution and normalization are not given.
  4. [Sec. 3.3, Fig. 12] The two proposed sampling losses, L_inc and L_col, are central to the unified HHOI generation framework, but they are ablated only qualitatively in Fig. 12. Without quantitative ablation on the dyadic and multi-human metrics, it is unclear how much of the improvement over the baselines in Tables 1–3 comes from these losses rather than from the underlying diffusion models. Please provide a quantitative ablation isolating the contributions of the inconsistency loss and the collision loss.
minor comments (4)
  1. [Sec. 4 vs. Appendix A.3] Sec. 4 states that CORE4D provides object annotations for "six object categories," but Appendix A.3 lists five CORE4D categories (board, box, bucket, chair, desk). Please reconcile this discrepancy.
  2. [Sec. 3.3, Eqs. (12)–(13)] The definition of N'_i says "the number of HHIs where human H_i appears as a target (i.e., in pairs of the form H_i → H_j)", but in the notation H_i → H_j, H_i is the source. The formulas in Eqs. (12)–(13) also use mixed indices (R_{H_j_n}, H_j_n) that are not clearly defined. Please clarify the indexing and the source/target convention.
  3. [Sec. 5.1] There is a typo: "We acccount generation success" should be "We count generation success." Also, "Penetration ration" should be "Penetration ratio."
  4. [Abstract / Sec. 1] The abstract says "we train an text-to-HOI and text-to-HHI model" — "an" should be "a." Please proofread for similar grammatical errors.

Circularity Check

0 steps flagged

No significant circularity: the core HOI/HHI decomposition, guided-sampling losses, and evaluations are independently grounded in held-out data and external benchmarks.

full rationale

Walking the derivation chain, none of the load-bearing steps reduces to its own inputs. Eq. (1) defines the HOI distribution p^M_c, but Sec. 3.2 immediately states 'For simplicity, we do not use a mesh instance M as input when modeling HOI; rather, we assume a fixed M is provided for each scenario.' This is an implementation/generalization gap (the score network sees text, not mesh geometry), not a circular definition: the model is still trained by denoising score matching (Eq. 6) on held-out HOI/HHI samples, and the output is not used to define the training target. The guided HHOI samples (Eq. 15) combine two separately trained score networks with hand-designed inconsistency and collision losses (Eqs. 9, 14); these losses are not fitted parameters and are not hidden renamings of the evaluation metrics (mesh vertex penetration vs capsule overlap; contact distance is not optimized at all). Evaluation is against a held-out test split of CORE4D and the collected dataset (FD, CLIP, user study), so the central realism claim is not forced by construction. The paper does cite the authors' earlier works [1, 31] for text augmentation and PF-ODE augmentation, and [30] for synthetic HOI data, but these are methodological dependencies on external prior frameworks with independent validation; they are not invoked as a uniqueness theorem or as the sole support for the HHOI claim. The stated limitation (cannot directly learn HHOIs from HOI-only or HHI-only data) is a scope restriction, not a tautology. I find no equation where the predicted quantity equals a fitted input by definition.

Axiom & Free-Parameter Ledger

5 free parameters · 8 axioms · 0 invented entities

The method rests on a factorization of HHOI into HOI and HHI, plus a hand-tuned guidance procedure. No new physical entities are introduced. The free parameters are inference-time loss schedules and representation choices, not physically meaningful constants.

free parameters (5)
  • guided sampling weight λ1(t) = min(100000, 100/t^2)
    Hand-set schedule for inconsistency loss in Eq. (15); controls how strongly HOI/HHI samples are pulled to consensus.
  • guided sampling weight λ2(t) = min(1600000, 1600/t^2)
    Hand-set schedule for collision loss in Eq. (15).
  • guided loss start time = t=0.5
    Inconsistency and collision losses are applied only for t ∈ [ε, 0.5]; chosen in B.3.
  • body pose latent dimension H = 10
    Pose representation compressed from 126D to 10D; capacity choice affecting model flexibility.
  • per-capsule radii (24 values) = learned via Chamfer distance on 922K poses
    Capsule proxy for collision loss; fitted to SMPL-X meshes from pose datasets, auxiliary to the central generation result.
axioms (8)
  • domain assumption SMPL-X parametric body model adequately represents human pose and shape for HHOI.
    Used throughout; Eq. (2) and Sec 3.1 define human meshes via SMPL-X.
  • ad hoc to paper HHOI can be decomposed into independent HOI and HHI distributions that can be recombined via guided sampling.
    Sec 3.1 and 3.3; the entire method relies on this factorization and on consistency constraints at inference.
  • domain assumption Two humans engaging in an HHI share the same scale.
    Sec 3.1: 'We assume both humans share the same scale.'
  • ad hoc to paper Human pairs not explicitly connected by an HHI are assumed non-colliding (implicit HHI).
    Sec 3.3; the collision loss is built on this assumption.
  • domain assumption A 24-capsule approximation with learned radii is sufficient for physical plausibility.
    Sec B.2; used to compute collision loss efficiently.
  • domain assumption CLIP text embeddings plus LLM-based prompt augmentation sufficiently specify the interaction geometry.
    Sec 3.2 and B.4; the text prompt is the only conditioning signal for HOI/HHI, and the object mesh is not used.
  • domain assumption Pretrained image diffusion models (ComA, Flux) generate unbiased and sufficiently diverse HHOI priors.
    Sec 4; synthetic data generated by these models is used to cover scenarios that are hard to capture in studio.
  • standard math Denoising score matching yields correct conditional scores for the modeled distributions.
    Eq. (6) and standard score-based diffusion theory (Song et al.).

pith-pipeline@v1.3.0-alltime-deepseek · 19007 in / 13158 out tokens · 137345 ms · 2026-08-03T20:15:06.205584+00:00 · methodology

0 comments
read the original abstract

The way humans interact with each other, including interpersonal distances, spatial configuration, and motion, varies significantly across different situations. To enable machines to understand such complex, context-dependent behaviors, it is essential to model multiple people in relation to the surrounding scene context. In this paper, we present a novel research problem to model the correlations between two people engaged in a shared interaction involving an object. We refer to this formulation as Human-Human-Object Interactions (HHOIs). To overcome the lack of dedicated datasets for HHOIs, we present a newly captured HHOIs dataset and a method to synthesize HHOI data by leveraging image generative models. As an intermediary, we obtain individual human-object interaction (HOIs) and human-human interaction (HHIs) from the HHOIs, and with these data, we train an text-to-HOI and text-to-HHI model using score-based diffusion model. Finally, we present a unified generative framework that integrates the two individual model, capable of synthesizing complete HHOIs in a single advanced sampling process. Our method extends HHOI generation to multi-human settings, enabling interactions involving more than two individuals. Experimental results show that our method generates realistic HHOIs conditioned on textual descriptions, outperforming previous approaches that focus only on single-human HOIs. Furthermore, we introduce multi-human motion generation involving objects as an application of our framework.

Figures

Figures reproduced from arXiv: 2511.20446 by Hanbyul Joo, Inhee Lee, Jeonghyeon Na, Junyoung Lee, Sangwon Baik.

Figure 1
Figure 1. Figure 1: Results of our HHOI generation given object instances and text-prompt descriptions. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Method Overview. (a) The training and inference process of the HOI/HHI part. (b) The advanced HHOI sampling process by introducing inconsistency loss and collision loss. diversity [30, 20, 37, 31]. Various models [63, 10, 36, 71, 50, 64, 33, 66, 24] have been proposed to learn from the presented datasets, including models manipulating articulated objects [12, 32], and multiple objects at once [69, 32]. How… view at source ↗
Figure 3
Figure 3. Figure 3: HHOI generation result of dyadic, and multiple humans in action with our model and [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Motion in-betweening outputs from DNO and InterGen, given a naive standing pose as the [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: HHOIs Capture System Overview. We capture Human-Human-Object Interactions (HHOIs) with our multiple camera capture system. The object and human poses are tracked with AruCo markers [15] and DWPose [65] respectively. Splatting (3D-GS) [29] scene from the initial frames and manually align the template mesh to the physical object in the scene to determine its object pose. For dynamic objects, subsequent poses… view at source ↗
Figure 6
Figure 6. Figure 6: Statistics on Collected Data Samples. Our dataset is constructed by integrating data from CORE4D and our multiview capture system, alongside synthetic samples generated via our data generation pipeline. relative to the first. We repeat this process by switching the reference human, resulting in two HHI data samples per HHOI frame. After preprocessing, the data is split into training and test sets with a 9:… view at source ↗
Figure 7
Figure 7. Figure 7: HHOI Diffusion Architecture. HHOI diffusion consists of two disjoint diffusion models: HOI diffusion and HHI diffusion. Although the overall structure of each diffusion model is the same, they are implemented with separate networks due to their different target distributions. Each network learns the score function of the HOI or HHI distribution, respectively. SMPL-X Human Capsule-based Human [PITH_FULL_IM… view at source ↗
Figure 8
Figure 8. Figure 8 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Guided Prompt Provided to LLM for Text Augmentation. (a) Guided prompt for HOI text prompt augmentation. (b) Guided prompt for HHI text prompt augmentation. B.4 Text Augmentation for Training We adopt the LLM-based text augmentation method proposed in [1] to train our text-conditioned HHOI diffusion model [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Questionnaire for User Study. Participants select the multi-view image that best depicts the human-human and human-object relationship. D Additional Qualitative Results [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11 [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Ablation Study for Guided HHOI Sampling. HOI: “A motorcycle ride by someone.” HHI: “Two companions ride a motorcycle.” HOI: “A jockey rides a horse.” HHI: “Two riders journey on a horse together.” HOI: “An equestrian sits astride a horse.” HHI: “A horse is ridden by two riders.” HOI: “A child steers a horse.” HHI: “On a horse, two friends sit together.” HOI: “A man navigates a motorcycle.” HHI: “Two rider… view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative Results of HHOI Sampling for Unseen Text Prompts 23 [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

85 extracted references · 5 linked inside Pith

  1. [1]

    S. Baik, H. Kim, and H. Joo. Learning 3d object spatial relationships from pre-trained 2d diffusion models. ICCV, 2025. 3, 4, 5, 18

    2025

  2. [2]

    Baradel*, M

    F. Baradel*, M. Armando, S. Galaaoui, R. Brégier, P. Weinzaepfel, G. Rogez, and T. Lucas*. Multi-hmr: Multi-person whole-body human mesh recovery in a single shot.ECCV, 2024. 7, 18

    2024

  3. [3]

    B. L. Bhatnagar, X. Xie, I. A. Petrov, C. Sminchisescu, C. Theobalt, and G. Pons-Moll. Behave: Dataset and method for tracking human object interactions.CVPR, 2022. 2

    2022

  4. [4]

    M. J. Black, P. Patel, J. Tesch, and J. Yang. Bedlam: A synthetic dataset of bodies exhibiting detailed lifelike animated motion.CVPR, 2023. 2

    2023

  5. [5]

    Bochkovskii, A

    A. Bochkovskii, A. Delaunoy, H. Germain, M. Santos, Y . Zhou, S. R. Richter, and V . Koltun. Depth pro: Sharp monocular metric depth in less than a second.ICLR, 2025. 7, 18

    2025

  6. [6]

    Z. Cao, H. Gao, K. Mangalam, Q.-Z. Cai, M. V o, and J. Malik. Long-term human motion prediction with scene context.ECCV, 2020. 2

    2020

  7. [7]

    R. T. Q. Chen. torchdiffeq, 2018. URLhttps://github.com/rtqichen/torchdiffeq. 5

    2018

  8. [8]

    H. Ci, M. Wu, W. Zhu, X. Ma, H. Dong, F. Zhong, and Y . Wang. Gfpose: Learning 3d human pose prior with gradient fields.CVPR, 2023. 3

    2023

  9. [9]

    Connect cloud research

    CloudResearch. Connect cloud research. URL https://connect.cloudresearch.com/researcher/. 18

  10. [10]

    Diller and A

    C. Diller and A. Dai. Cg-hoi: Contact-guided 3d human-object interaction generation.CVPR, 2024. 1, 3

    2024

  11. [11]

    K. Fan, J. Tang, W. Cao, R. Yi, M. Li, J. Gong, J. Zhang, Y . Wang, C. Wang, and L. Ma. Freemotion: A unified framework for number-free text-to-motion synthesis.ECCV, 2024. 1, 3

    2024

  12. [12]

    Z. Fan, O. Taheri, D. Tzionas, M. Kocabas, M. Kaufmann, M. J. Black, and O. Hilliges. ARCTIC: A dataset for dexterous bimanual hand-object manipulation.CVPR, 2023. 2, 3

    2023

  13. [13]

    H.-S. Fang, J. Li, H. Tang, C. Xu, H. Zhu, Y . Xiu, Y .-L. Li, and C. Lu. Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time.TPAMI, 2022. 18

    2022

  14. [14]

    Fieraru, M

    M. Fieraru, M. Zanfir, E. Oneata, A.-I. Popa, V . Olaru, and C. Sminchisescu. Three-dimensional recon- struction of human interactions.CVPR, 2020. 2, 3

    2020

  15. [15]

    Garrido-Jurado, R

    S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez. Automatic generation and detection of highly reliable fiducial markers under occlusion.Pattern Recognition, 2014. 15

    2014

  16. [16]

    Ghosh, R

    A. Ghosh, R. Dabral, V . Golyanik, C. Theobalt, and P. Slusallek. Remos: 3d motion-conditioned reaction synthesis for two-person interactions.ECCV, 2024. 2, 3

    2024

  17. [17]

    W. Guo, X. Bie, X. Alameda-Pineda, and F. Moreno-Noguer. Multi-person extreme motion prediction. CVPR, 2022. 2

    2022

  18. [18]

    Guzov, A

    V . Guzov, A. Mir, T. Sattler, and G. Pons-Moll. Human poseitioning system (hps): 3d human pose estimation and self-localization in large scenes from body-mounted sensors.CVPR, 2021. 1, 2

    2021

  19. [19]

    Guzov, J

    V . Guzov, J. Chibane, R. Marin, Y . He, Y . Saracoglu, T. Sattler, and G. Pons-Moll. Interaction replica: Tracking human–object interaction and scene changes from human motion.3DV, 2024. 1, 2

    2024

  20. [20]

    Han and H

    S. Han and H. Joo. Learning canonicalized 3d human-object spatial relations from unbounded synthesized images.ICCV, 2023. 3

    2023

  21. [21]

    C.-H. P. Huang, H. Yi, M. Höschle, M. Safroshkin, T. Alexiadis, S. Polikovsky, D. Scharstein, and M. J. Black. Capturing and inferring dense full-body human-scene contact.CVPR, 2022. 2, 4 11

    2022

  22. [22]

    Huang, O

    Y . Huang, O. Taheri, M. J. Black, and D. Tzionas. Intercap: Joint markerless 3d tracking of humans and objects in interaction.GCPR, 2022. 2

    2022

  23. [23]

    Ionescu, D

    C. Ionescu, D. Papava, V . Olaru, and C. Sminchisescu. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments.TPAMI, 2008. 4

    2008

  24. [24]

    Jiang, Z

    N. Jiang, Z. He, Z. Wang, H. Li, Y . Chen, S. Huang, and Y . Zhu. Autonomous character-scene interaction synthesis from text instruction.SIGGRAPH Asia, 2024. 3

    2024

  25. [25]

    Jiang, Z

    N. Jiang, Z. Zhang, H. Li, X. Ma, Z. Wang, Y . Chen, T. Liu, Y . Zhu, and S. Huang. Scaling up dynamic human-scene interaction modeling.CVPR, 2024. 2

    2024

  26. [26]

    H. Joo, H. Liu, L. Tan, L. Gui, B. Nabbe, I. Matthews, T. Kanade, S. Nobuhara, and Y . Sheikh. Panoptic studio: A massively multiview system for social motion capture.CVPR, 2015. 1, 3, 15

    2015

  27. [27]

    H. Joo, T. Simon, M. Cikara, and Y . Sheikh. Towards social artificial intelligence: Nonverbal social signal prediction in a triadic interaction.CVPR, 2019. 3

    2019

  28. [28]

    Karunratanakul, K

    K. Karunratanakul, K. Preechakul, E. Aksan, T. Beeler, S. Suwajanakorn, and S. Tang. Optimizing diffusion noise can serve as universal motion priors.CVPR, 2024. 2, 9

    2024

  29. [29]

    Kerbl, G

    B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering.SIGGRAPH, 2023. 15

    2023

  30. [30]

    H. Kim, S. Han, P. Kwon, and H. Joo. Beyond the contact: Discovering comprehensive affordance for 3d objects from pre-trained 2d diffusion models.ECCV, 2024. 1, 3, 7

    2024

  31. [31]

    H. Kim, S. Baik, and H. Joo. David: Modeling dynamic affordance of 3d objects using pre-trained video diffusion models.ICCV, 2025. 3, 5

    2025

  32. [32]

    J. Kim, J. Kim, J. Na, and H. Joo. Parahome: Parameterizing everyday home activities towards 3d generative modeling of human-object interactions.CVPR, 2025. 2, 3

    2025

  33. [33]

    Kulkarni, D

    N. Kulkarni, D. Rempe, K. Genova, A. Kundu, J. Johnson, D. Fouhey, and L. Guibas. Nifty: Neural object interaction fields for guided human motion synthesis.CVPR, 2024. 3

    2024

  34. [34]

    J. N. Kundu, H. Buckchash, P. Mandikal, A. Jamkhandi, V . B. Radhakrishnan, et al. Cross-conditioned recurrent networks for long-term synthesis of inter-person human motion interactions.WACV, 2020. 2, 3

    2020

  35. [35]

    B. F. Labs. Flux.https://github.com/black-forest-labs/flux, 2024. 7

    2024

  36. [36]

    J. Li, A. Clegg, R. Mottaghi, J. Wu, X. Puig, and C. K. Liu. Controllable human-object interaction synthesis.ECCV, 2024. 1, 3

    2024

  37. [37]

    Li and A

    L. Li and A. Dai. GenZI: Zero-shot 3D human-scene interaction generation.CVPR, 2024. 3, 7, 18

    2024

  38. [38]

    R. Li, Y . Zhang, Y . Zhang, Y . Zhang, M. Su, J. Guo, Z. Liu, Y . Liu, and X. Li. Interdance: Reactive 3d dance generation with realistic duet interactions.arXiv preprint arXiv:2412.16982, 2024. 3

    Pith/arXiv arXiv 2024

  39. [39]

    Liang, W

    H. Liang, W. Zhang, W. Li, J. Yu, and L. Xu. Intergen: Diffusion-based multi-human motion generation under complex interactions.IJCV, 2024. 1, 2, 3, 10

    2024

  40. [40]

    D. Lim, C. Jeong, and Y . M. Kim. Mammos: Mapping multiple human motion with scene understanding and natural interactions.ICCVW, 2023. 1, 3

    2023

  41. [41]

    Y . Liu, C. Zhang, R. Xing, B. Tang, B. Yang, and L. Yi. Core4d: A 4d human-object-human interaction dataset for collaborative object rearrangement.arXiv preprint arXiv:2406.19353, 2024. 2, 3, 6, 15

    Pith/arXiv arXiv 2024

  42. [42]

    Müller, V

    L. Müller, V . Ye, G. Pavlakos, M. Black, and A. Kanazawa. Generative proxemics: A prior for 3d social interaction from images.CVPR, 2024. 2, 3

    2024

  43. [43]

    Chatgpt: Optimizing language models for dialogue, 2023

    OpenAI. Chatgpt: Optimizing language models for dialogue, 2023. URL https://openai.com/blog/ chatgpt. 18

    2023

  44. [44]

    Pavlakos, V

    G. Pavlakos, V . Choutas, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas, and M. J. Black. Expressive body capture: 3D hands, face, and body from a single image.CVPR, 2019. 4, 15

    2019

  45. [45]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision.ICML, 2021. 5, 7 12

    2021

  46. [46]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models.CVPR, 2022. 2, 7

    2022

  47. [47]

    Shafir, G

    Y . Shafir, G. Tevet, R. Kapon, and A. H. Bermano. Human motion diffusion as a generative prior.ICLR,

  48. [48]

    M. Shan, L. Dong, Y . Han, Y . Yao, T. Liu, I. Nwogu, G.-J. Qi, and M. Hill. Towards open domain text-driven synthesis of multi-person motions.ECCV, 2024. 1, 3

    2024

  49. [49]

    Siyao, T

    L. Siyao, T. Gu, Z. Yang, Z. Lin, Z. Liu, H. Ding, L. Yang, and C. C. Loy. Duolando: Follower gpt with off-policy reinforcement learning for dance accompaniment.ICLR, 2024. 3

    2024

  50. [50]

    W. Song, X. Zhang, S. Li, Y . Gao, A. Hao, X. Hou, C. Chen, N. Li, and H. Qin. Hoianimator: Generating text-prompt human-object animations using novel perceptive diffusion models.CVPR, 2024. 1, 3

    2024

  51. [51]

    Song and S

    Y . Song and S. Ermon. Generative modeling by estimating gradients of the data distribution.NeurIPS,

  52. [52]

    Song and S

    Y . Song and S. Ermon. Improved techniques for training score-based generative models.NeurIPS, 2020. 3

    2020

  53. [53]

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations.ICLR, 2021. 3, 5

    2021

  54. [54]

    Suhail, A

    M. Suhail, A. Mittal, B. Siddiquie, C. Broaddus, J. Eledath, G. Medioni, and L. Sigal. Energy-based learning for scene graph generation.CVPR, 2021. 3

    2021

  55. [55]

    Tanaka and K

    M. Tanaka and K. Fujiwara. Role-aware interaction generation from textual description.ICCV, 2023. 1, 3

    2023

  56. [56]

    P. Vincent. A connection between score matching and denoising autoencoders.Neural computation, 23(7): 1661–1674, 2011. 4

    2011

  57. [57]

    von Marcard, R

    T. von Marcard, R. Henschel, M. Black, B. Rosenhahn, and G. Pons-Moll. Recovering accurate 3d human pose in the wild using imus and a moving camera.ECCV, 2018. 1, 2, 4

    2018

  58. [58]

    M. Wu, F. Zhong, Y . Xia, and H. Dong. TarGF: Learning target gradient field for object rearrangement. NeurIPS, 2022. 3

    2022

  59. [59]

    Xiang, Y

    F. Xiang, Y . Qin, K. Mo, Y . Xia, H. Zhu, F. Liu, M. Liu, H. Jiang, Y . Yuan, H. Wang, et al. Sapien: A simulated part-based interactive environment.CVPR, 2020. 19

    2020

  60. [60]

    L. Xu, Z. Song, D. Wang, J. Su, Z. Fang, C. Ding, W. Gan, Y . Yan, X. Jin, X. Yang, et al. Actformer: A gan-based transformer towards general action-conditioned 3d human motion generation.ICCV, 2023. 1, 3

    2023

  61. [61]

    L. Xu, X. Lv, Y . Yan, X. Jin, S. Wu, C. Xu, Y . Liu, Y . Zhou, F. Rao, X. Sheng, et al. Inter-x: Towards versatile human-human interaction analysis.CVPR, 2024. 2, 3

    2024

  62. [62]

    L. Xu, Y . Zhou, Y . Yan, X. Jin, W. Zhu, F. Rao, X. Yang, and W. Zeng. Regennet: Towards human action-reaction synthesis.CVPR, 2024. 1, 3

    2024

  63. [63]

    S. Xu, Z. Li, Y .-X. Wang, and L.-Y . Gui. Interdiff: Generating 3d human-object interactions with physics-informed diffusion.ICCV, 2023. 1, 3

    2023

  64. [64]

    S. Xu, Z. Wang, Y .-X. Wang, and L.-Y . Gui. Interdreamer: Zero-shot text to 3d dynamic human-object interaction.NeurIPS, 2024. 3

    2024

  65. [65]

    Z. Yang, A. Zeng, C. Yuan, and Y . Li. Effective whole-body pose estimation with two-stages distillation. ICCV, 2023. 7, 15

    2023

  66. [66]

    H. Yi, J. Thies, M. J. Black, X. B. Peng, and D. Rempe. Generating human interaction motions in scenes with text control.arXiv preprint arXiv:2404.10685, 2024. 3

    Pith/arXiv arXiv 2024

  67. [67]

    Zhang, H

    J. Zhang, H. Luo, H. Yang, X. Xu, Q. Wu, Y . Shi, J. Yu, L. Xu, and J. Wang. Neuraldome: A neural modeling pipeline on multi-view human-object interactions.CVPR, 2023. 1, 2

    2023

  68. [68]

    Zhang, M

    J. Zhang, M. Wu, and H. Dong. Generative category-level object pose estimation via diffusion models. NeurIPS, 2024. 3, 4, 16

    2024

  69. [69]

    Zhang, J

    J. Zhang, J. Zhang, Z. Song, Z. Shi, C. Zhao, Y . Shi, J. Yu, L. Xu, and J. Wang. Hoi-mˆ 3: Capture multiple humans and objects interaction within contextual environment.CVPR, 2024. 1, 2, 3 13

    2024

  70. [70]

    Zhang, B

    X. Zhang, B. L. Bhatnagar, S. Starke, I. Petrov, V . Guzov, H. Dhamo, E. Pérez-Pellitero, and G. Pons- Moll. Force: Dataset and method for intuitive physics guided human-object interaction.arXiv preprint arXiv:2403.11237, 2024. 2

    Pith/arXiv arXiv 2024

  71. [71]

    Zhang, H

    Y . Zhang, H. Yang, C. Luo, J. Peng, Y . Wang, and Z. Zhang. Ood-hoi: Text-driven 3d whole-body human-object interactions generation beyond training domains.arXiv preprint arXiv:2411.18660, 2024. 1, 3

    Pith/arXiv arXiv 2024

  72. [72]

    K. Zhao, S. Wang, Y . Zhang, T. Beeler, and S. Tang. COINS: Compositional human-scene interaction synthesis with semantic control.ECCV, 2022. 1

    2022

  73. [73]

    Zheng, R

    Y . Zheng, R. Shao, Y . Zhang, T. Yu, Z. Zheng, Q. Dai, and Y . Liu. Deepmulticap: Performance capture of multiple characters using sparse multiview cameras.ICCV, 2021. 1, 3

    2021

  74. [74]

    Zheng, Y

    Y . Zheng, Y . Yang, K. Mo, J. Li, T. Yu, Y . Liu, C. K. Liu, and L. J. Guibas. Gimo: Gaze-informed human motion prediction in context.ECCV, 2022. 1, 2

    2022

  75. [75]

    Zheng, T

    Z. Zheng, T. Yu, Y . Liu, and Q. Dai. Pamir: Parametric model-conditioned implicit representation for image-based human reconstruction.TPAMI, 2021. 7, 15

    2021

  76. [76]

    {prompt}

    Y . Zhou, C. Barnes, J. Lu, J. Yang, and H. Li. On the continuity of rotation representations in neural networks.CVPR, 2019. 4 14 Appendix A Data Collection Details A.1 Multiview HHOIs Data Capture System To capture Human-Human-Object Interactions (HHOIs), we adopt a multi-camera setup inspired by Panoptic Studio [26]. As shown in Fig. 5, our system is co...

    2019

  77. [77]

    person",

    Each sentence must contain a word that clearly refers to a person. Feel free to use varied words such as "person", "someone", "man", "woman", "individual", or other natural alternatives

  78. [80]

    Keep the sentences simple and avoid unnecessary embellishments

    Do not include any objects other than the main two (human, {object}). Keep the sentences simple and avoid unnecessary embellishments

  79. [82]

    {prompt}

    Maintain diversity in the overall sentence pool. Avoid repeating similar expressions, but do not allow metaphors that are too distant. Step 2 After creating each sentence, carefully check whether it meets all the conditions in step 1. If any sentence does not satisfy the conditions, rewrite it. Step 3 Once all {N} sentences are created, output them with o...

  80. [83]

    person",

    Each sentence must contain a word that clearly refers to two people. Feel free to use varied words such as "person", "someone", "man", "woman", "individual", or other natural alternatives

Showing first 80 references.