Pith. sign in

REVIEW 4 major objections 6 minor 67 references

InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read InteractAnything claims that a text prompt plus any 3D object mesh is enough to synthesize a natural, contact-accurate human-object interaction without training on interaction datasets.

desk verdict A genuinely integrated zero-shot HOI pipeline worth serious review, but the load-bearing contact claim needs objective validation before acceptance. read the letter →

arxiv 2505.24315 v1 pith:72HB3B7Z submitted 2025-05-30 cs.CV

Jinlu Zhang , Yixin Chen , Zan Wang , Jie Yang , Yizhou Wang , Siyuan Huang This is my paper
classification cs.CV
keywords zero-shot3Dhuman-objectinteractionsynthesisobjectaffordanceparsingLLM-guidedinitialization2Ddiffusioninpaintingmulti-viewscoredistillationsamplingforceclosurecontact-awareposeoptimizationopen-setobjects
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

InteractAnything sets out to prove that a text prompt and a single 3D object mesh are enough to synthesize a natural, contact-accurate 3D human-object interaction, even for object categories the system has never seen, and without training on human-object interaction datasets. The method answers three sub-questions in sequence: which body part relates to the object and from which side, answered by a large language model selecting from fixed options; where on the object the body should touch, answered by a pre-trained 2D inpainting diffusion model whose outputs are converted into a 3D contact probability map; and what exact pose realizes the interaction, answered by score-distillation pose synthesis followed by force-closure-based contact optimization. The authors argue that prior text-to-3D and HOI methods fail on open-set objects because they either lack relation reasoning, cannot localize affordances on arbitrary geometry, or cannot enforce detailed contact such as hand grasping. If the claim holds, 3D interaction synthesis becomes a zero-shot operation usable for any object mesh, with consequences for AR/VR, simulation, and animation.

What carries the argument

The carrying mechanism is the open-set object affordance parser coupled with the force-closure refinement. For a given action, the parser renders the object from several views, creates inpainting masks from LLM-selected body-part labels and inferred human-object placement, and asks a pre-trained 2D diffusion inpainting model to draw the human interacting with the object; a 2D pose detector then supplies keypoints, and the distance from each keypoint to the object mask defines the per-view contact probability $f^{(i)}_{\mathrm{afford}}(p)=e^{-\|d_i(p)\|}$, which is averaged over views and re-projected onto the object surface to give a 3D affordance map $M$. That map weights the chamfer contact loss and selects the object vertices used in the finer force-closure loss $L_{fc}=\sum_{j\in[o]}(\sum_{i\in[h]} f_v(i,j)\cdot n(j))^2$, where $f_v(i,j)$ is the force applied by interacted human vertex $i$ at object vertex $j$ and $n(j)$ is the object normal. The same affordance map is what makes the method open-set: it comes from 2D diffusion knowledge rather than from 3D training assets.

What would settle it

A concrete test: take object-action pairs with known ground-truth interaction meshes, render each object alone from the views used by the parser, run the affordance parser, and compare the predicted contact-probability peak regions against the vertices actually touched in the ground-truth mesh; if the overlap is no better than chance on a held-out set, the central affordance claim fails. A complementary test is to measure contact error, such as vertex-to-surface distance and penetration depth, between the synthesized human mesh and object mesh: systematic floating hands or deep interpenetration would falsify the force-closure refinement claim.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that zero-shot 3D human-object interaction synthesis can be decomposed into three stages, each delegated to a pre-trained model, and that this decomposition yields fine-grained, text-aligned interactions for open-set objects without any HOI-specific training. The LLM stage turns a short instruction such as 'a person grasps the chair' into concrete initial values: object rotation, translation, scale, on-ground state, and the semantic body parts involved. The affordance stage renders the object into multiple views, writes LLM-guided full-body and body-part inpainting masks, lets a pre-trained 2D diffusion inpainter hallucinate the interacting human, detects 2D body keypoints, and converts per-view distance to the object mask into a contact probability that is aggregated and re-projected onto the 3D mesh. The pose stage uses multi-view score distillation sampling with a spatial constraint that keeps the human outside the object volume, then refines body and hand parameters with a force-closure loss, normal alignment, chamfer contact loss, penetration penalty, scale loss, and an optional ground loss. The final output is a posed SMPL-H human mesh, a parametric body-and-hand model, and an object mesh with corrected pose, scale, and placement, with contact concentrated where the affordance map says it should be.

Load-bearing premise

The load-bearing premise is that the images produced by the 2D inpainting model, after the language-model-guided masking, show a sufficiently correct human-object interaction that 2D body keypoints and a distance-based contact map give a trustworthy 3D contact probability distribution; if the inpainter draws a missing or misplaced body part, the contact map is wrong and all later optimization pushes the human toward the wrong region.

Editorial extensions

If this is right

  • The same object mesh can host different interactions: 'sits on the chair surface', 'pulls the chair back', and 'lifts the chair bottom' place contact on different regions of the same chair.
  • Open-set objects such as a motorcycle, a car, a baby doll, or a humanoid robot receive plausible interactions without any object-specific retraining.
  • Hand-level grasping detail emerges from the force-closure refinement, and the paper's ablation shows this detail is absent when the fine-grained optimization is removed.
  • LLM-guided initialization is load-bearing: ablating it drops the method below a strong baseline, because a randomly initialized human-object relation cannot be repaired by later optimization.
  • The generated interactions can be placed into existing 3D scenes with a ground loss, supporting applications such as sitting on a table or opening a door.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that a confidence gate on the affordance map could prevent the worst failure mode: when the inpainter draws a partial or misplaced body part, the contact map is wrong and the force-closure step optimizes toward that wrong region.
  • One extension is to replace SMPL-H with other articulated agents: the affordance parsing and force-closure losses are body-model-agnostic, while only the body prior and the pose estimator would need swapping.
  • A testable extension follows from the 2D origin of the contact map: as multi-view-consistent 2D diffusion models improve, the affordance map should improve automatically with no change to the optimization stages.
  • The LLM option-selection design reduces hallucination by constraining answers, but it also caps expressiveness: interactions whose relative position, scale, or body part is not among the predefined options cannot be initialized correctly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes InteractAnything, a zero-shot pipeline for synthesizing 3D human-object interactions (HOI) from a text prompt and an arbitrary object mesh. The method uses an LLM to infer initial human-object relations and choose options for object rotation, translation, scale, and state; a pre-trained 2D diffusion inpainting model together with OpenPose keypoints to produce a per-vertex object affordance map via a 2D distance-based probability function; multi-view SDS to initialize the human pose; and a final coarse-to-fine optimization that includes contact, normal, penetration, scale, ground, and a SMPL-H-adapted force-closure loss. The authors report CLIP similarity scores and GPT-4V selection percentages showing improvements over Magic3D, DreamFusion, DreamFusion*, and DreamHOI, along with ablations of the LLM initialization and fine-grained optimization. The central claim is that the method generates text-aligned, contact-aware, open-set 3D HOIs without training on HOI datasets.

Significance. If the central claims held, the paper would make a useful contribution: a zero-shot, training-free pipeline for open-set 3D HOI synthesis is a timely and practically relevant goal, and the idea of combining LLM commonsense with diffusionpriors is attractive. The paper also has concrete strengths: it explicitly avoids dataset-specific training, proposes a novel affordance-parsing mechanism, and includes ablations suggesting that the LLM initialization and the fine-grained optimization both contribute to the final quality. The release of a project page is a plus. However, the evaluation is thin and the load-bearing contact-validation claim is not tested: the affordance map is built from 2D projection proximity, which is not equivalent to 3D contact, and no ground-truth or simulator-based contact metric is used. The quantitative results also lack variance, trial counts, and independent validation of the GPT-4V preference scores. These gaps prevent the paper from being acceptable in its current form.

major comments (4)
  1. [Sec. 3.3, Eq. (4)-(5), and Sec. 3.5, Eqs. (7)-(9)] The entire contact claim relies on the affordance map P(p) computed from f_afford(p)=exp(-||d_i(p)||), where d_i(p) is a 2D distance in the image plane. Image-plane proximity is not the same as 3D contact: a body part rendered in front of the object but physically separated from it can still project onto the object mask and receive high affordance. Because the LLM-guided initialization places the human in a consistent relative position and the same placement is reused across the n_p views, this false-positive bias is not removed by averaging in Eq. (5). The map directly weights L_inter (Eq. 7) and selects vertices for L_fc (Eq. 9), so any systematic error in P(p) propagates into the optimized pose. The paper does not validate the map against ground-truth contact, does not include a random-map or oracle-map ablation, and its own limitation section (Sec. 5) only acknowledges a general dependence on 2D priors, not this specific projection-to-3D ambiguity. I request a quantitative validation using BEHAVE ground-truth contact vertices (e.g., precision/recall of predicted contact regions) or an equivalent simulator-based contact test, plus an ablation that replaces P(p) with a uniform or randomized affordance map to show that the proposed map itself is informative.
  2. [Sec. 4.2, Tables 1-3] The quantitative evaluation is not reported with sufficient rigor. Table 1 gives CLIP similarity scores as single means with no standard deviations, no per-prompt breakdown, and no statistical test; with only 5 prompts and 10 objects, these differences could easily be within noise. Tables 2 and 3 report values such as 45.6 and 52.1 that appear to be GPT-4V selection percentages, but the paper does not state whether these are percentages, how many trials were run, what the variance across trials was, or how the multiple tests mentioned in Sec. 4.2 were aggregated. GPT-4V is also the same model family that provides the LLM initialization and feedback used during generation, so an evaluation that uses GPT-4V to select the 'most reasonable' result may systematically favor outputs that align with the same prior. I ask for error bars and trial counts, a clearer definition of the metric, and either a human-study comparison or an independently validated metric (e.g., contact-distance against a ground-truth dataset) to rule out this shared-source bias.
  3. [Sec. 3.5, Eq. (9)] The force-closure-like loss L_fc is not sufficiently specified to be reproducible or to support the physical-plausibility claim. The symbol f_v(i,j) is described only as 'the force vector applied at vertex j from interacted human vertex i,' but no formula, source, or derivation is given. The sum inside the square and the dependence on normals n(j) are also unexplained. The paper states 'inspired by differentiable force closure [32]' but replaces the base model from MANO to SMPL-H, and it is unclear how the force vectors are computed in practice. Since the abstract and conclusion claim 'realistic 3D contact' and the loss is a central component of the final optimization, the authors should provide a precise definition of f_v, state how contacts are sampled, and ideally evaluate the resulting grasps with a physics-based quality metric or simulator; otherwise, the physical-plausibility claim remains unsubstantiated.
  4. [Sec. 3.2 and Sec. 4.4, Table 3] The ablation shows that removing the LLM-guided initialization drops the GPT-4V selection score, but it does not characterize the sensitivity of the method to the LLM's option choices or to the specific LLM model used. Since the initialization determines the inpainting masks (Eqs. 2-3), the affordance map, and the final pose, a different LLM or a different option set could materially change the results. I suggest adding an analysis of how the generated HOI varies when the LLM is replaced by another model or when the option choices are perturbed, so that the contribution of the LLM is more than a single point comparison.
minor comments (6)
  1. [Sec. 3.3, Eqs. (2)-(3)] The notation in these equations is under-specified: V^H, P^h, r_o^{-1}, s_f, s_p, c_i, t_o^{-1}, and the projection transform J are not all defined in the text or in the caption of Figure 2. Please define each symbol and its dimensionality.
  2. [Sec. 3.4] The 'spatial constraint' that prevents penetration is mentioned but not given an equation or a precise description. Since it is part of the pose-initialization optimization, a clear definition or citation would help reproducibility.
  3. [Sec. 4.1] The evaluation-set description says '5 prompts and 10 objects categories' but then lists 5 BEHAVE objects and 5 generative objects; please clarify the total number of prompts and the number of distinct object meshes used for each comparison.
  4. [Sec. 4.2, Tables 2-3] Please state explicitly whether the numbers in these tables are percentages, and if so, the total number of trials and the standard error; also indicate what 'multiple tests' means in terms of independent GPT-4V queries.
  5. [Sec. 3.5, Eq. (6) and Eq. (8)] The notation s'·L_g is unclear: s is defined as the object state in [0,1], but the prime is not defined. Please clarify the intended operation or remove the prime.
  6. [Sec. 5] The limitation paragraph acknowledges the need for simulator-based evaluation and multi-view consistent generation, but these caveats should be connected to the specific affordance-map and force-closure issues raised above; as written, the section reads as a general future-work list rather than a caveat on the current evidence.

Circularity Check

0 steps flagged · score 2.0 of 10

No formal circularity: the affordance-to-contact pipeline is feed-forward and the metrics are post hoc; minor non-load-bearing self-citations and a shared GPT evaluation bias keep the score at 2.

full rationale

The derivation chain is feed-forward: text and object mesh are mapped through LLM-based initialization, 2D inpainting, the Eq. (4)-(5) affordance computation, multi-view SDS pose synthesis, and the contact losses of Eq. (6)-(9). The final human pose is not defined as the affordance map; the map only selects object regions used by the chamfer and force-closure losses, while SDS, penetration, scale, and ground losses also shape the result, so no output reduces to an input by construction. The CLIP and GPT-4V metrics are applied post hoc and never feed back into the optimization, so no fitted parameter is renamed as a prediction. The paper does cite its own authors' prior work (refs 20, 21, and 63), but only as datasets or related 3D-reconstruction methods, not as a load-bearing uniqueness theorem or ansatz. The main limitations are the acknowledged reliance on 2D priors (Sec. 5) and the shared-source evaluation in which GPT-4 is used to guide initialization, mask generation, and prompt augmentation while GPT-4V judges the outputs; these weaken the independence of the evidence and the strength of the 'realistic 3D contact' claim, but they are validation concerns rather than equation-level circularity. Accordingly, no specific circular step meets the quoted-reduction bar; the score of 2 reflects only the minor, non-load-bearing self-citations.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

All core stages depend on pre-trained models whose outputs are not externally validated: the LLM must provide correct spatial commonsense, the inpainting model plus OpenPose must produce correct contact maps, and the SDS process must produce a plausible pose. Loss weights and option sets are hand-chosen and unreported. No new physical entity is introduced.

free parameters (3)
  • Loss weights phi_i and delta in Eqs. (6), (8), (9) = not reported
    These weights control the interplay of contact, normal, penetration, scale, and ground losses; values are chosen by hand and not specified, so the reported results depend on unstated tuning.
  • LLM option sets for relative position, scale, and object state (Sec. 3.2) = 4 discrete scale options; predefined position options with values not listed
    The hand-authored discrete choices determine object placement and scale; different option sets would change initialization and downstream results.
  • Number of views n_p and object downsampling D in affordance parsing (Sec. 3.3) = not reported
    The contact probability averaged over views and the object sampling density depend on these choices; their values are not given in the paper.
assumptions (5)
  • domain assumption Pre-trained 2D inpainting diffusion model (SDXL) can produce images of a person correctly interacting with an open-set object when given LLM-generated adaptive masks.
    Section 3.3 builds the entire affordance map on the inpainting output; no external validation against ground-truth contact is provided.
  • domain assumption OpenPose 2D keypoints from inpainted views are reliable proxies for 3D contact regions on the object surface.
    Eq. (4) converts keypoint-to-object distance into contact probabilities; the paper provides no ground-truth contact evaluation.
  • domain assumption LLM constrained multiple-choice reasoning yields correct relative human-object position, orientation, scale, and body part labels.
    Section 3.2 initializes object pose and all later spatial reasoning from these selections; the 'w/o LLM-init' ablation shows the pipeline depends on them.
  • domain assumption SDS gradients from a multi-view 2D diffusion prior can optimize a 3D human pose into a natural interaction with an embedded object mesh.
    Section 3.4 uses multi-view SDS as the main pose synthesis engine; no quantitative metric verifies that the resulting poses are natural.
  • ad hoc to paper Force-closure-inspired loss (Eq. 9) is a valid proxy for physically stable contact when applied to SMPL-H vertices.
    Eq. (9) is a hand-designed re-formulation of a MANO grasp force-closure estimator; no physical simulation or grasp-stability evaluation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing." pith.science (2026). https://pith.science/paper/72HB3B7Z

@misc{pith2026250524315,
  author       = {Pith},
  title        = {Pith review of: InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/72HB3B7Z}},
  note         = {Machine review of arXiv:2505.24315}
}
read the original abstract

Recent advances in 3D human-aware generation have made significant progress. However, existing methods still struggle with generating novel Human Object Interaction (HOI) from text, particularly for open-set objects. We identify three main challenges of this task: precise human-object relation reasoning, affordance parsing for any object, and detailed human interaction pose synthesis aligning description and object geometry. In this work, we propose a novel zero-shot 3D HOI generation framework without training on specific datasets, leveraging the knowledge from large-scale pre-trained models. Specifically, the human-object relations are inferred from large language models (LLMs) to initialize object properties and guide the optimization process. Then we utilize a pre-trained 2D image diffusion model to parse unseen objects and extract contact points, avoiding the limitations imposed by existing 3D asset knowledge. The initial human pose is generated by sampling multiple hypotheses through multi-view SDS based on the input text and object geometry. Finally, we introduce a detailed optimization to generate fine-grained, precise, and natural interaction, enforcing realistic 3D contact between the 3D object and the involved body parts, including hands in grasping. This is achieved by distilling human-level feedback from LLMs to capture detailed human-object relations from the text instruction. Extensive experiments validate the effectiveness of our approach compared to prior works, particularly in terms of the fine-grained nature of interactions and the ability to handle open-set 3D objects.

Figures

Figures reproduced from arXiv: 2505.24315 by the authors.

Figure 1
Figure 1. 3D human object interaction synthesis by InteractAnything. Given a simple text description with goal interaction and any object mesh as input, our method enables the generation of diverse, natural, detailed, and novel interactions for open-set 3D objects in a zero-shot manner. The orange and green boxes of (b) indicate detailed contact poses from different views. Abstract Recent advances in 3D human-aware generation… view at source ↗
Figure 2
Figure 2. Framework of InteractAnything. Given a text description and any object mesh as input, our approach begins by querying LLM to infer precise human-object relationships, which are used to initialize object properties. Next, we analyze the contact affordance of the object geometry. The human pose is synthesized using a pre-trained 2D diffusion model, guided by multi-view SDS loss and the designed spatial constraint. Fin… view at source ↗
Figure 3
Figure 3. Qualitative comparison results with baselines. ∗ indicates we re-implement this method by embedding object mesh into the diffusion process, which follows the pipeline of DreamHOI [67] and our method. More visualization results are presented in supplementary materials. where J is 2D projection transformation with camera pa￾rameter c i , and ϖ1 is the noise term. Similarly, we generate body-part-level inpainting masks… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Ablation study on the fine-grained optimization. Left figures are the results of removing fine-grained terms and right figures apple fine-grained terms to synthesize grasping details. Metrics Magic3D DreamFusion DreamFusion* DreamHOI Ours Overall 4.3 6.5 17.3 26.0 45.6…
Figure 6
Figure 6. Figure 6: Qualitative results of scene populating and scene in￾teraction applications from different views. Qualitative performance on Scene Populating. We demonstrate one possible way to populate scenes with Inter￾actAnything. We first utilize the HPS dataset [14] to obtain the…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 52 canonical work pages

  1. [32]

    Tengyu Liu, Zeyu Liu, Ziyuan Jiao, Yixin Zhu, and Song- Chun Zhu. Synthesizing diverse and physically stable grasps with arbitrary hand structures using differentiable force clo- sure estimator.IEEE Robotics and Automation Letters (RA- L), 7(1):470–477, 2021. 2, 6

    work page 2021

  2. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  3. [2]

    Behave: Dataset and method for tracking human object in- teractions

    Bharat Lal Bhatnagar, Xianghui Xie, Ilya A Petrov, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. Behave: Dataset and method for tracking human object in- teractions. InConference on Computer Vision and Pattern Recognition (CVPR), 2022. 2, 6

    work page 2022

  4. [3]

    Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J. Black. Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image. InComputer Vision – ECCV 2016, 2016. 5

    work page 2016

  5. [4]

    Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models

    Yukang Cao, Yan-Pei Cao, Kai Han, Ying Shan, and Kwan- Yee K Wong. Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 958–968, 2024. 2

    2024

  6. [5]

    Realtime multi-person 2d pose estimation using part affin- ity fields

    Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affin- ity fields. InConference on Computer Vision and Pattern Recognition (CVPR), 2017. 5

    work page 2017

  7. [6]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. InInternational Confer- ence on Computer Vision (ICCV), 2023. 2

    work page 2023

  8. [7]

    Comboverse: Compositional 3d as- sets creation using spatially-aware diffusion guidance

    Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, and Ziwei Liu. Comboverse: Compositional 3d as- sets creation using spatially-aware diffusion guidance. In European Conference on Computer Vision, pages 128–146. Springer, 2024. 2, 3

    work page 2024

Show all 67 references
  1. [8]

    Syn- thesis and optimization of force closure grasps via sequential semidefinite programming.Robotics Research: Volume 1, pages 285–305, 2018

    Hongkai Dai, Anirudha Majumdar, and Russ Tedrake. Syn- thesis and optimization of force closure grasps via sequential semidefinite programming.Robotics Research: Volume 1, pages 285–305, 2018. 2

    2018

  2. [9]

    In- terfusion: Text-driven generation of 3d human-object inter- action

    Sisi Dai, Wenhao Li, Haowen Sun, Haibin Huang, Chongyang Ma, Hui Huang, Kai Xu, and Ruizhen Hu. In- terfusion: Text-driven generation of 3d human-object inter- action. InEuropean Conference on Computer Vision, pages 18–35. Springer, 2024. 2, 3, 6

    2024

  3. [10]

    Gemini: A family of highly capable mul- timodal models.arXiv preprint arXiv:2312.11805, 2023

    Google DeepMind. Gemini: A family of highly capable mul- timodal models.arXiv preprint arXiv:2312.11805, 2023. 2

    2023 arXiv

  4. [11]

    Kaolin: A pytorch library for accelerating 3d deep learning re- search.https://github.com/NVIDIAGameWorks/ kaolin, 2022

    Clement Fuji Tsang, Maria Shugrina, Jean Francois Lafleche, Towaki Takikawa, Jiehan Wang, Charles Loop, Wenzheng Chen, Krishna Murthy Jatavallabhula, Edward Smith, Artem Rozantsev, Or Perel, Tianchang Shen, Jun Gao, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xi- ang, Jia...

    2022

  5. [12]

    Graphdreamer: Compositional 3d scene synthesis from scene graphs

    Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Sch ¨olkopf. Graphdreamer: Compositional 3d scene synthesis from scene graphs. InConference on Com- puter Vision and Pattern Recognition (CVPR), 2024. 3

    2024

  6. [13]

    Diffpose: Toward more reliable 3d pose estimation

    Jia Gong, Lin Geng Foo, Zhipeng Fan, Qiuhong Ke, Hos- sein Rahmani, and Jun Liu. Diffpose: Toward more reliable 3d pose estimation. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13041–13051, 2023. 5

    2023

  7. [14]

    Human poseitioning system (hps): 3d human pose estimation and self-localization in large scenes from body-mounted sensors

    Vladimir Guzov, Aymen Mir, Torsten Sattler, and Gerard Pons-Moll. Human poseitioning system (hps): 3d human pose estimation and self-localization in large scenes from body-mounted sensors. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021. 8

    2021

  8. [15]

    Chorus: Learning canon- icalized 3d human-object spatial relations from unbounded synthesized images

    Sookwan Han and Hanbyul Joo. Chorus: Learning canon- icalized 3d human-object spatial relations from unbounded synthesized images. InInternational Conference on Com- puter Vision (ICCV), 2023. 3, 5

    2023

  9. [16]

    Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J. Black. Resolving 3D human pose ambigu- ities with 3D scene constraints. InInternational Conference on Computer Vision (ICCV), 2019. 2, 3

    2019

  10. [17]

    Avatarclip: Zero-shot text- driven generation and animation of 3d avatars.arXiv preprint arXiv:2205.08535, 2022

    Fangzhou Hong, Mingyuan Zhang, Liang Pan, Zhongang Cai, Lei Yang, and Ziwei Liu. Avatarclip: Zero-shot text- driven generation and animation of 3d avatars.arXiv preprint arXiv:2205.08535, 2022. 2

    2022 arXiv

  11. [18]

    Gauhuman: Articu- lated gaussian splatting from monocular human videos

    Shoukang Hu, Tao Hu, and Ziwei Liu. Gauhuman: Articu- lated gaussian splatting from monocular human videos. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20418–20431, 2024. 2

    2024

  12. [19]

    Robust water- tight manifold surface generation method for shapenet mod- els.arXiv preprint arXiv:1802.01698, 2018

    Jingwei Huang, Hao Su, and Leonidas Guibas. Robust water- tight manifold surface generation method for shapenet mod- els.arXiv preprint arXiv:1802.01698, 2018. 6

    2018 arXiv

  13. [20]

    Full-body articulated human-object interaction

    Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu, and Siyuan Huang. Full-body articulated human-object interaction. InInterna- tional Conference on Computer Vision (ICCV), 2023. 2

    2023

  14. [21]

    Scaling up dynamic human-scene interaction model- ing

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction model- ing. InConference on Computer Vision and Pattern Recog- nition (CVPR), 2024. 2

    2024

  15. [22]

    End-to-end recovery of human shape and pose

    Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7122–7131, 2018. 5

    2018

  16. [23]

    Maximizing parallelism in the construc- tion of bvhs, octrees, and k-d trees

    Tero Karras. Maximizing parallelism in the construc- tion of bvhs, octrees, and k-d trees. InProceedings of ACM SIGGRAPH/Eurographics Conference on High- Performance Graphics, 2012. 6

    2012

  17. [24]

    3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics (TOG), 42(4):139–1, 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics (TOG), 42(4):139–1, 2023. 2

    2023

  18. [25]

    Beyond the contact: Discovering comprehensive affordance for 3d objects from pre-trained 2d diffusion models

    Hyeonwoo Kim, Sookwan Han, Patrick Kwon, et al. Beyond the contact: Discovering comprehensive affordance for 3d objects from pre-trained 2d diffusion models. InEuropean Conference on Computer Vision (ECCV), 2024. 4, 5

    2024

  19. [26]

    Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023. 2, 3

    2023

  20. [27]

    Controllable human-object interaction synthesis

    Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object interaction synthesis. InEuropean Conference on Computer Vision (ECCV), 2024. 2

    2024

  21. [28]

    Genzi: Zero-shot 3d human-scene in- teraction generation

    Lei Li and Angela Dai. Genzi: Zero-shot 3d human-scene in- teraction generation. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 20465–20474, 2024. 4

    2024

  22. [29]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. InConference on Computer Vi- sion and Pattern Recognition (CVPR), 2023. 2, 6, 7

    2023

  23. [30]

    One-stage 3d whole-body mesh recovery with component aware transformer

    Jing Lin, Ailing Zeng, Haoqian Wang, Lei Zhang, and Yu Li. One-stage 3d whole-body mesh recovery with component aware transformer. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 21159–21168, 2023. 5

    2023

  24. [31]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. InInternational Confer- ence on Computer Vision (ICCV), 2023. 2

    2023

  25. [33]

    Humangaus- sian: Text-driven 3d human generation with gaussian splat- ting

    Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng, Dahua Lin, Xihui Liu, and Ziwei Liu. Humangaus- sian: Text-driven 3d human generation with gaussian splat- ting. InConference on Computer Vision and Pattern Recog- nition (CVPR), 2024. 2

    2024

  26. [34]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. InInternational Conference on Learning Representa- tions (ICLR), 2023. 2

    2023

  27. [35]

    Wonder3d: Sin- gle image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. InConference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  28. [36]

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi- person linear model.ACM Transactions on Graphics (TOG), 34(6):248:1–248:16, 2015. 3

    2015

  29. [37]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. InEuropean Conference on Computer Vision (ECCV), 2020. 2

    2020

  30. [38]

    Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding

    Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding. InProceedings of the IEEE/CVF conference on computer vision and pattern recogn...

    2019

  31. [39]

    Gpt-3.5.https://platform.openai.com/ docs/models/gpt-3-5, 2023

    OpenAI. Gpt-3.5.https://platform.openai.com/ docs/models/gpt-3-5, 2023. Accessed: 2023-11-14. 2, 4

    2023

  32. [40]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from a single image. InProceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 6

    2019

  33. [41]

    Hoi-diff: Text-driven synthe- sis of 3d human-object interactions using diffusion models

    Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text-driven synthe- sis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:2312.06553, 2023. 2

    2023 arXiv

  34. [42]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 2, 4

    2023 arXiv

  35. [43]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. InIn- ternational Conference on Learning Representations (ICLR),

  36. [44]

    Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Askell Amanda, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever

    Alec Radford, JongWook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Askell Amanda, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision.Cornell University - arXiv...

    2021

  37. [45]

    Hierarchical text-conditional image gen- eration with clip latents.arXiv preprint arXiv:2204.06125,

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents.arXiv preprint arXiv:2204.06125,

    arXiv

  38. [46]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InConference on Com- puter Vision and Pattern Recognition (CVPR), 2022. 2

    2022

  39. [47]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. 2

    2022

  40. [48]

    Em- bodied hands: modeling and capturing hands and bodies to- gether.ACM Transactions on Graphics (TOG), 36(6):1–17,

    Javier Romero, Dimitrios Tzionas, and Michael J Black. Em- bodied hands: modeling and capturing hands and bodies to- gether.ACM Transactions on Graphics (TOG), 36(6):1–17,

  41. [49]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. InConference on Computer Vision and Pattern Recognition (CVPR), 2023. 2

    2023

  42. [50]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. InAdvances in Neural Informati...

    2022

  43. [51]

    Zero123++: a single image to consistent multi-view dif- fusion base model.arXiv preprint arXiv:2310.15110, 2023

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model.arXiv preprint arXiv:2310.15110, 2023. 2

    arXiv

  44. [52]

    Mvdream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023. 2, 6

    2023 arXiv

  45. [53]

    Grab: A dataset of whole-body human grasp- ing of objects

    Omid Taheri, Nima Ghorbani, Michael J Black, and Dim- itrios Tzionas. Grab: A dataset of whole-body human grasp- ing of objects. InEuropean Conference on Computer Vision (ECCV), 2020. 2

    2020

  46. [54]

    Deco: Dense estimation of 3d human-scene contact in the wild

    Shashank Tripathi, Agniv Chatterjee, Jean-Claude Passy, Hongwei Yi, Dimitrios Tzionas, and Michael J Black. Deco: Dense estimation of 3d human-scene contact in the wild. In International Conference on Computer Vision (ICCV), 2023. 2

    2023

  47. [55]

    Reconstructing action- conditioned human-object interactions using commonsense knowledge priors

    Xi Wang, Gen Li, Yen-Ling Kuo, Muhammed Kocabas, Emre Aksan, and Otmar Hilliges. Reconstructing action- conditioned human-object interactions using commonsense knowledge priors. InInternational Conference on 3D Vision (3DV), 2022. 3

    2022

  48. [56]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. InAdvances in Neural Information Processing Systems (NeurIPS), 2024. 2

    2024

  49. [57]

    Srinivasan, Jonathan T

    Chung-Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. Barron, and Ira Kemelmacher-Shlizerman. Hu- mannerf: Free-viewpoint rendering of moving people from monocular video. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 162...

    2022

  50. [58]

    InterDiff: Generating 3d human-object interactions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. InterDiff: Generating 3d human-object interactions with physics-informed diffusion. InInternational Confer- ence on Computer Vision (ICCV), 2023. 2, 3

    2023

  51. [59]

    Inter- dreamer: Zero-shot text to 3d dynamic human-object inter- action.Advances in Neural Information Processing Systems, 37:52858–52890, 2024

    Sirui Xu, Yu-Xiong Wang, Liangyan Gui, et al. Inter- dreamer: Zero-shot text to 3d dynamic human-object inter- action.Advances in Neural Information Processing Systems, 37:52858–52890, 2024

    2024

  52. [60]

    Generating human interaction motions in scenes with text control

    Hongwei Yi, Justus Thies, Michael J Black, Xue Bin Peng, and Davis Rempe. Generating human interaction motions in scenes with text control. InEuropean Conference on Com- puter Vision (ECCV), 2025. 2, 3

    2025

  53. [61]

    Whac: World-grounded hu- mans and cameras

    Wanqi Yin, Zhongang Cai, Ruisi Wang, Fanzhou Wang, Chen Wei, Haiyi Mei, Weiye Xiao, Zhitao Yang, Qingping Sun, Atsushi Yamashita, et al. Whac: World-grounded hu- mans and cameras. InEuropean Conference on Computer Vision, pages 20–37, 2024. 5

    2024

  54. [62]

    Pymaf-x: To- wards well-aligned full-body model regression from monoc- ular images.IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 45:12287–12303, 2023

    Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, and Yebin Liu. Pymaf-x: To- wards well-aligned full-body model regression from monoc- ular images.IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 45:12287–12303, 2023

    2023

  55. [63]

    Mixste: Seq2seq mixed spatio-temporal encoder for 3d human pose estimation in video

    Jinlu Zhang, Zhigang Tu, Jianyu Yang, Yujin Chen, and Jun- song Yuan. Mixste: Seq2seq mixed spatio-temporal encoder for 3d human pose estimation in video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13232–13242, 2022. 5

    2022

  56. [64]

    Zhang, Sam Pepose, Hanbyul Joo, Deva Ra- manan, Jitendra Malik, and Angjoo Kanazawa

    Jason Y . Zhang, Sam Pepose, Hanbyul Joo, Deva Ra- manan, Jitendra Malik, and Angjoo Kanazawa. Perceiving 3d human-object spatial arrangements from a single image in the wild. InEuropean Conference on Computer Vision (ECCV), 2020. 3, 6

    2020

  57. [65]

    Humannerf: Efficiently gen- erated human radiance field from sparse inputs

    Fuqiang Zhao, Wei Yang, Jiakai Zhang, Pei Lin, Yingliang Zhang, Jingyi Yu, and Lan Xu. Humannerf: Efficiently gen- erated human radiance field from sparse inputs. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7743–7753, 2022. 2

    2022

  58. [66]

    Zerong Zheng, Tao Yu, Yebin Liu, and Qionghai Dai. Pamir: Parametric model-conditioned implicit representa- tion for image-based human reconstruction.IEEE transac- tions on pattern analysis and machine intelligence, 44(6): 3170–3184, 2021. 6

    2021

  59. [67]

    Dreamhoi: Subject-driven generation of 3d human- object interactions with diffusion priors.arXiv preprint arXiv:2409.08278, 2024

    Thomas Hanwen Zhu, Ruining Li, and Tomas Jakab. Dreamhoi: Subject-driven generation of 3d human- object interactions with diffusion priors.arXiv preprint arXiv:2409.08278, 2024. 2, 3, 5, 6, 7, 8

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.