Pith. sign in

REVIEW 4 major objections 5 minor 53 references

Event-Driven Storytelling with Multiple Lifelike Humans in a 3D Scene

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An event-driven LLM pipeline can choreograph four or more characters in multi-room 3D scenes, with planning accuracy that holds across several LLM engines.

desk verdict Useful modular LLM-driven system for multi-character scene animation, but the benchmark evaluates single-character next-step plans and the scalability claims are overstated. read the letter →

arxiv 2507.19232 v1 pith:CNOJJ6ZF submitted 2025-07-25 cs.CV

Donggeun Lim , Jinseok Bae , Inwoo Hwang , Seungmin Lee , Hwanhee Lee , Young Min Kim This is my paper · ORCID
classification cs.CV
keywords event-drivenstorytellingmulti-humanmotionsynthesisLLM-basedplanning3Dsceneunderstandingscene-awarebehaviorarea-conditionedpositionsamplingmulti-characteranimationbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a large language model can direct the long-term, coordinated motion of many virtual humans in a 3D scene if the job is broken into small, sequentially planned events. The framework's narrator writes one event at a time, an event parser turns each event into concrete action labels and target areas, and a motion-matching module animates the characters. The authors claim this event-based decomposition keeps planning accurate as scenes grow to multi-room scale and as the cast grows past four or five characters, and that the results are stable across commercial and open LLMs. They support the claim with a 40-case benchmark covering object arrangement, regional context, scene state, and position inference, plus ablations and a 50-participant user study. If the claim holds, it gives game, VR, and film previsualization a practical way to populate spaces with characters whose behavior emerges from a story rather than from hand-scripted choreography.

What carries the argument

The load-bearing object is the event, an intermediate representation written by a narrator and parsed into $e=(C_e,\{p_i\},\{d_i\},\{a_i\})$: the characters involved, a target 2D position, a target orientation, and an action label for each. Events decompose the holistic spatio-temporal reasoning burden, letting the LLM think about one small interaction at a time while the system as a whole runs events concurrently. The supporting machinery is a scene describer that turns a 3D scene graph plus DBSCAN object clusters into a contextual textual description; an event parser that uses programming-structured prompts with spatial functions such as get distance between and semantic areas; and a low-level motion module that plans paths with windowed cooperative A* and synthesizes motion with motion matching. The key move is that the LLM never has to reason at coordinate level: it selects semantic areas, and exact coordinates are sampled from those areas afterward.

What would settle it

Have independent annotators, who did not design the system, write the expected plan (action label and target area) for each of the 40 benchmark test cases and measure how often they agree; if agreement is low, or if a simple baseline that always picks the most common action and a central area matches the reported success rates, the benchmark is not measuring planning quality.

Watch

Extended reading notes

Core claim

The central claim is that contextual multi-human motion in 3D scenes stops being one monolithic synthesis problem once you let an LLM plan it as a sequence of events. Each event names a small set of characters, an action, and a rough location; the event parser then grounds that location through spatial-reasoning tools written as Python functions, and area-conditioned position sampling picks exact coordinates. The paper reports success rates of 0.90 total with GPT-4o, 0.74 with GPT-4o mini, and 0.72 with Llama-3.1-70B on its benchmark, with the event pipeline beating an ablation that plans without event decomposition, beating a scene representation that is a raw object list, and beating raw scene-graph input. It also reports that asking the LLM to output coordinates directly fails even on simple localization cases, while the area-conditioned sampling succeeds. The authors present this as the first system to address multi-character contextual motion at this scale.

Load-bearing premise

The load-bearing premise is that the benchmark's 40 hand-built test cases, with their expected action labels and target areas, actually measure what scene-aware multi-agent planning should do; if those expected plans do not reflect real user needs, the reported success rates only show internal consistency.

Editorial extensions

If this is right

  • A single free-text instruction can drive full-scene storytelling: characters will find objects, use spaces, and interact with each other without per-character scripting.
  • Because new events are generated only for idle characters, the runtime loop supports long-horizon operation and mid-run user instructions, so interactive rehearsal or gameplay direction becomes possible.
  • The ablation results imply scene description quality matters more than raw information: an object list loses spatial reasoning, while an exhaustive scene graph hurts performance and token efficiency; a distilled regional description is the better input.
  • Robustness across LLM engines means the planning module can be swapped for cheaper or local models, with GPT-4o-class models giving the best success rates but smaller open models still producing usable plans.
  • Area-conditioned position sampling, rather than direct coordinate output, is the component that makes LLM plans executable at the level of precise 3D placement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is using the narrator as a runtime game director: the same event loop could adapt a storyline to player actions in an interactive 3D environment, not just follow a pre-set instruction.
  • The benchmark's expected plans were authored by the system designers; an independent check would be to have multiple annotators write expected action labels and areas for the same 40 cases and measure inter-annotator agreement before trusting the pass or fail signal.
  • The appendix result that vision-based planning performs far worse suggests that current VLMs do not yet replace text scene descriptions for this type of reasoning; a hybrid that feeds visual grounding into the textual description may combine the strengths of both.
  • If the event decomposition is the real source of the scaling gain, the same pattern could transfer to other multi-agent LLM planning tasks such as embodied robotics or crowd simulation, where monolithic planning currently fails beyond a few agents.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an event-driven framework for generating contextual motions of multiple lifelike humans in a 3D scene. The system decomposes the problem into a sequence of events, uses an LLM-based narrator and event parser to plan high-level behavior, and couples this with area-conditioned position sampling and a low-level motion synthesis module. The authors introduce a 40-case benchmark with criteria for object arrangement, regional context, scene state, and position inference, and evaluate their full pipeline against ablations (w/o Event, Object List, Scene Graph, Direct Inference) across six LLM backbones, adding a user study with 50 participants. The central claims are that the framework generates long-term motions with more than 4-5 characters in multi-room scenes and is robust to the choice of LLM engine.

Significance. If the claims are borne out, the work is a useful system demonstration: it packages LLM planning, scene graph description, area-conditioned location sampling, and motion matching into a single pipeline that produces coordinated multi-character animations in scenes larger than those typically handled by single-character or pairwise motion generation. The paper's strengths are its internally consistent ablation evidence (the full pipeline outperforms all ablations across all tested LLM backbones), the large and consistent gain of the area-conditioned position sampling module on the PI test cases (Table 2), the broad LLM coverage including open-source and commercial models, and the author-provided code, benchmark, and result videos. The internal consistency is a genuine positive. However, the significance of the headline scalability and long-horizon storytelling claims is constrained by the benchmark design, which tests a single next-step plan for a fixed target character rather than a temporally extended multi-character storyline.

major comments (4)
  1. [Section 4.2, benchmark setup and pass/fail conditions] The benchmark does not directly support the abstract's claim of 'long-term motions with more than 4-5 characters in various multi-room scale scenes.' Each test case provides a mock planning history H and demands only the next plan for a fixed target character c1; pass/fail requires (1) a nonempty plan, (2) ag in AS ∩ Ae, and (3) position within an expected area. This measures coarse label-and-area retrieval for a single character, not multi-agent coordination over a sequence of events. The scalability discussion tied to Figure 7 varies character counts but does not change the single-plan evaluation. I recommend either adding evaluation over full event sequences with multiple target characters and temporal consistency checks, or visibly narrowing the claims to what the benchmark supports.
  2. [Table 1 and Section 4.3, robustness to LLM choice] The claim of robustness to LLM engines is weakened by the large variation in success rates across backbones: GPT-4o achieves 0.90 total, Qwen2.5-72B 0.71, Llama-3.1-70B 0.72, but Llama-3.1-8B and Qwen2.5-7B drop to 0.35 and 0.39, respectively. If 'robust' means that the pipeline completes without crashing and improves over ablations on each backbone, the claim is defensible; if it means consistent performance, the data contradict it. I ask the authors to state explicitly which notion of robustness they intend and to add a sentence acknowledging the strong model-size dependence.
  3. [Section 4.4 and Appendix C, user study] The user study is a preference vote among ablation outputs, not a head-to-head comparison with prior systems, and it covers only 4-5 events per scenario (and 2 events for MPH11). Allowing participants to select multiple options weakens the preference signal because ties are pooled with genuine preferences; no statistical significance test or inter-rater agreement is reported. This is acceptable as a qualitative check but is not enough to support the 'high scalability' claim, especially since the authors themselves note that the MPH11 result differs from the other scenes. Please report per-scenario vote counts unpooled, add a significance test, and avoid claims beyond the evidence.
  4. [Section 4.2, benchmark authorship and expected plans] All quantitative conclusions rest on a benchmark whose expected plans tau_e = (A_e, p~_e) were authored by the same group that proposes the method. The paper does not report inter-annotator agreement on the expected areas and action sets, nor does it show that the expected plans correlate with a downstream application metric. Because the benchmark is the sole anchor for the framework's superiority claims, I recommend adding (a) a second annotator or agreement measure, (b) an external or at least a more objective grounding test for a sample of cases, or (c) an explicit admission that the reported success rates are relative to the authors' definition and may not generalize to other definitions of plan quality.
minor comments (5)
  1. [Section 3.1.3, code example] The code snippet in Figure 4 contains a typo ('chiars' instead of 'chairs'); please correct it and align the variable names with the narrative text.
  2. [Section A.5.2, evaluation settings] The benchmark uses temperature 0.1 while the user study uses temperature 0.0; the paper should justify this discrepancy or state whether it affects the interpretability of the user study results.
  3. [Section B.2, VLM experiments] The sentence introducing Table 4 reads 'In Table 4 Table 4 presents...'; please remove the duplicated phrase.
  4. [Figure 12, semantic area representation] The figure caption does not explain how the colored semantic areas are computed from the object bounding boxes; adding one sentence on the geometric definition of the areas would make the sampling step reproducible.
  5. [Section 4.2, benchmark definition] The notation uses both p~_e and p_g with a tilde to denote area-level positions, but the relationship between area-level and coordinate-level outputs is not formally defined; a short formal definition would remove ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the event-planning pipeline and its ablations are evaluated against fixed gold plans that the system can fail, and no load-bearing self-citation is used.

full rationale

The paper's derivation chain is modular and self-contained: a 3D scene graph is converted into a contextual description, the narrator produces an event, the event parser grounds it via Python-style spatial tools and area-conditioned sampling, and the motion synthesis module executes it. The benchmark's expected plans tau_e=(A_e, p~e) are fixed gold data defined by the authors before runs; generated plans are compared against them with explicit pass/fail conditions, and Table 1 shows large variation across backbones (GPT-4o 0.90 vs Llama-3.1-8B 0.35) and across ablations, so success is not forced by construction. The ablations (w/o Event, Object List, Scene Graph, Direct Inference) test the method's components against the same benchmark and produce different scores, which would not occur if the metric were definitionally equivalent to the pipeline. No load-bearing self-citation appears: the technical building blocks (motion matching [7], windowed cooperative A* [37], scene graph extraction [19], code-as-policies prompting [26,38,50]) are external prior work, and the paper does not invoke any uniqueness theorem or prior result from its own authors to justify its design. The fact that the benchmark and user study are self-constructed and the user study compares only internal ablations is a legitimate limitation on external validity, but it is not circularity: the expected plans are not fitted to the model outputs, and the pipeline can and does fail them. Any concerns about the benchmark measuring only one-step label-and-area retrieval, or about the robustness-to-LLM-choice claim being undercut by the Llama-3.1-8B result, are correctness or evidence-weight issues, not reductions of a derived result to its inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

All quantitative claims rest on the author-built benchmark and user study; no external benchmark is used. The hand-set hyperparameters and hand-authored action and expected-plan labels are inputs to that evaluation rather than fitted constants in a derivation, but they still constrain what the results mean. No physical entities are postulated; the event is an internal data structure.

free parameters (2)
  • Area-conditioned position sampling hyperparameters (area extent for close_to, adjacent_to, etc.) = user-controlled, numeric values not reported
    Appendix A.3.3 states that the specific area size represented by each semantic expression such as close_to is controlled by user hyperparameters. These areas directly determine whether sampled positions fall inside expected areas in the PI test cases.
  • DBSCAN clustering parameters (eps, min_samples) = eps=1.0, min_samples=2
    Appendix A.3.1 sets these values by hand for functional-region detection. They shape the scene description that the narrator and event parser receive.
assumptions (4)
  • domain assumption The author-defined expected plans tau_e=(A_e, p~e) and per-scene action label sets A_S are valid ground truth for scene-aware planning.
    Section 4.2: pass/fail is decided by matching the generated action and position to these author-written expected values. No inter-annotator agreement, external norm, or third-party benchmark anchors this ground truth.
  • domain assumption Textual scene description D generated from the scene graph and DBSCAN clusters preserves the spatial and contextual information needed for planning.
    Sections 3.1.1 and 3.1.2 feed D into the narrator and event parser; if the description omits or distorts spatial relations, the high-level plans cannot be grounded correctly.
  • domain assumption The predefined per-scene action label sets and motion databases cover the behaviors that open-ended user instructions can request.
    Section 4.2 defines A_S per scene; Appendix A.4.1 builds motion databases for those labels. The claim of open-ended storytelling depends on this coverage, which is not measured or bounded.
  • domain assumption Relative preference among ablation videos by 50 general participants is a valid measure of contextual plausibility and lifelikeness.
    Section 4.4 uses these selections to claim the framework represents the given scenario more effectively; no expert raters, realism metric, or statistical test validates this assumption.
invented entities (1)
  • Event representation e = (C_e, {p_i}, {d_i}, {a_i})
    purpose: Serve as the intermediate semantic unit that lets the narrator delegate spatial grounding to the event parser and the motion synthesizer.
    This is an internal data structure rather than an empirical entity. It exists only in the software, and its usefulness is shown only through the author-built benchmark and user study.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Event-Driven Storytelling with Multiple Lifelike Humans in a 3D Scene." pith.science (2026). https://pith.science/paper/CNOJJ6ZF

@misc{pith2026250719232,
  author       = {Pith},
  title        = {Pith review of: Event-Driven Storytelling with Multiple Lifelike Humans in a 3D Scene},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CNOJJ6ZF}},
  note         = {Machine review of arXiv:2507.19232}
}
read the original abstract

In this work, we propose a framework that creates a lively virtual dynamic scene with contextual motions of multiple humans. Generating multi-human contextual motion requires holistic reasoning over dynamic relationships among human-human and human-scene interactions. We adapt the power of a large language model (LLM) to digest the contextual complexity within textual input and convert the task into tangible subproblems such that we can generate multi-agent behavior beyond the scale that was not considered before. Specifically, our event generator formulates the temporal progression of a dynamic scene into a sequence of small events. Each event calls for a well-defined motion involving relevant characters and objects. Next, we synthesize the motions of characters at positions sampled based on spatial guidance. We employ a high-level module to deliver scalable yet comprehensive context, translating events into relative descriptions that enable the retrieval of precise coordinates. As the first to address this problem at scale and with diversity, we offer a benchmark to assess diverse aspects of contextual reasoning. Benchmark results and user studies show that our framework effectively captures scene context with high scalability. The code and benchmark, along with result videos, are available at our project page: https://rms0329.github.io/Event-Driven-Storytelling/.

Figures

Figures reproduced from arXiv: 2507.19232 by the authors.

Figure 1
Figure 1. Our framework populates 3D scenes with multiple characters. The generated characters interact with their surroundings and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our framework. The framework consists [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the high-level action planning module. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: An example of a programming-structured prompt used [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results of our framework across diverse 3D scenes. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: An example of a test case. In this test case, a system is [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Benchmark results by number of characters. Our ap [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Comparison of average token usage. Object List uses the [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: User study results. Our system correctly generates the [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Spatial relationships used in the scene graph construc [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 12
Figure 12. Figure 12: Semantic area representation examples. not generate new events and waits until a character com￾pletes their event. A.3.3. Event Parser The event parser utilizes programming-structured prompts and the area-conditioned position sampling method to parse events into low-l…
Figure 11
Figure 11. Figure 11: Key area extraction on MPH8 from PROX dataset [ [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 13
Figure 13. Figure 13: Matching features example. Our motion synthesis module utilizes the following matching features: • Keyjoint Positions: Positions of key joints J expressed in the character’s local frame (R 3J ). • Keyjoint Velocities: Velocities of key joints J in the local frame (R 3…
Figure 14
Figure 14. Figure 14: House scene. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_14.png]
Figure 17
Figure 17. Figure 17: An example of a top-view image used in our VLM [PITH_FULL_IMAGE:figures/full_fig_p012_17.png]
Figure 16
Figure 16. Figure 16: Restaurant scene. A.5.2. Test Settings Here, we provide additional details of our benchmark test settings. To address the randomness inherent in LLMs, we repeat each test case five times and average the results. We also set the temperature parameter, which affects out…
Figure 19
Figure 19. Figure 19: Test scenario employed in the user study for the House [PITH_FULL_IMAGE:figures/full_fig_p013_19.png]
Figure 18
Figure 18. Figure 18: Test scenario employed in the user study for the [PITH_FULL_IMAGE:figures/full_fig_p013_18.png]
Figure 21
Figure 21. Figure 21: Test scenario employed in the user study for the Restau [PITH_FULL_IMAGE:figures/full_fig_p014_21.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 40 canonical work pages

  1. [2]

    Circle: Capture in rich contextual environ- ments

    Joao Pedro Ara ´ujo, Jiaman Li, Karthik Vetrivel, Rishi Agar- wal, Jiajun Wu, Deepak Gopinath, Alexander William Clegg, and Karen Liu. Circle: Capture in rich contextual environ- ments. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 21211–21221,

    work page

  2. [3]

    Lan- Figure 19

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- Figure 19. Test scenario employed in the user study for the House scene. Figure 20. Test scenario employed in the user study for the Office scene. guage models are few-shot learners. Adva...

    work page 1901

  3. [4]

    Test scenario employed in the user study for the Restau- rant scene

    Zhongang Cai, Jianping Jiang, Zhongfei Qing, Xinying Guo, Mingyuan Zhang, Zhengyu Lin, Haiyi Mei, Chen Wei, Ruisi 13 Figure 21. Test scenario employed in the user study for the Restau- rant scene. Wang, Wanqi Yin, et al. Digital life project: Autonomous 3d characters with social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision an...

    work page 2024

  4. [5]

    Gener- ating human motion in 3d scenes from text descriptions

    Zhi Cen, Huaijin Pi, Sida Peng, Zehong Shen, Minghui Yang, Shuai Zhu, Hujun Bao, and Xiaowei Zhou. Gener- ating human motion in 3d scenes from text descriptions. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 1855–1866, 2024. 1, 2

    work page 2024

  5. [6]

    Sitcom-crafter: A plot-driven human motion generation sys- tem in 3d scenes

    Jianqi Chen, Panwen Hu, Xiaojun Chang, Zhenwei Shi, Michael Christian Kampffmeyer, and Xiaodan Liang. Sitcom-crafter: A plot-driven human motion generation sys- tem in 3d scenes. arXiv preprint arXiv:2410.10790, 2024. 2, 7

    arXiv 2024

  6. [7]

    Motion matching and the road to next- gen animation

    Simon Clavet et al. Motion matching and the road to next- gen animation. In Proc. of GDC, page 4, 2016. 5, 9, 11

    work page 2016

  7. [8]

    Llms are not intelligent thinkers: Introducing mathematical topic tree benchmark for comprehensive evaluation of llms

    Arash Gholami Davoodi, Seyed Pouyan Mousavi Davoudi, and Pouya Pezeshkpour. Llms are not intelligent thinkers: Introducing mathematical topic tree benchmark for comprehensive evaluation of llms. arXiv preprint arXiv:2406.05194, 2024. 5

    arXiv 2024

  8. [9]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

Show all 53 references
  1. [10]

    A density-based algorithm for discovering clusters in large spatial databases with noise

    Martin Ester, Hans-Peter Kriegel, J ¨org Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, pages 226–231,

  2. [11]

    Remos: 3d motion- conditioned reaction synthesis for two-person interactions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Chris- tian Theobalt, and Philipp Slusallek. Remos: 3d motion- conditioned reaction synthesis for two-person interactions. In European Conference on Computer Vision , pages 418–

  3. [12]

    Generating exe- cutable action plans with environmentally-aware language models

    Maitrey Gramopadhye and Daniel Szafir. Generating exe- cutable action plans with environmentally-aware language models. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 3568–3575. IEEE, 2023. 1, 2

    2023

  4. [13]

    Resolving 3d human pose ambiguities with 3d scene constraints

    Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J Black. Resolving 3d human pose ambiguities with 3d scene constraints. In Proceedings of the IEEE/CVF international conference on computer vision , pages 2282– 2292, 2019. 8, 10

    2019

  5. [14]

    Stochas- tic scene-aware motion prediction

    Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael J Black. Stochas- tic scene-aware motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11374–11384, 2021. 1, 2, 11

    2021

  6. [15]

    Learned motion matching

    Daniel Holden, Oussama Kanoun, Maksym Perepichka, and Tiberiu Popa. Learned motion matching. ACM Transactions on Graphics (ToG), 39(4):53–1, 2020. 11

    2020

  7. [16]

    An embodied general- ist agent in 3d world

    Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied general- ist agent in 3d world. In Proceedings of the 41st Inter- national Conference on Machine Learning , pages 20413– 20451, 2024. 3

    2024

  8. [17]

    Language models as zero-shot planners: Extract- ing actionable knowledge for embodied agents

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extract- ing actionable knowledge for embodied agents. In Interna- tional conference on machine learning , pages 9118–9147. PMLR, 2022. 1, 2

    2022

  9. [18]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 2, 7

    2024 arXiv

  10. [19]

    Sceneverse: Scaling 3d vision-language learning for grounded scene understanding

    Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. Sceneverse: Scaling 3d vision-language learning for grounded scene understanding. In European Conference on Computer Vision, pages 289–310. Springer, 2024. 3, 9

  11. [20]

    A peek into token bias: Large language models are not yet genuine reasoners

    Bowen Jiang, Yangxinyu Xie, Zhuoqun Hao, Xiaomeng Wang, Tanwi Mallick, Weijie Su, Camillo Taylor, and Dan Roth. A peek into token bias: Large language models are not yet genuine reasoners. In Proceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Process...

    2024

  12. [21]

    Autonomous character-scene interaction synthesis from text instruction

    Nan Jiang, Zimo He, Zi Wang, Hongjie Li, Yixin Chen, Siyuan Huang, and Yixin Zhu. Autonomous character-scene interaction synthesis from text instruction. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 1, 2

    2024

  13. [22]

    Smart-llm: Smart multi-agent robot task planning using large language models

    Shyam Sundar Kannan, Vishnunandan LN Venkatesh, and Byung-Cheol Min. Smart-llm: Smart multi-agent robot task planning using large language models. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 12140–12147. IEEE, 2024. 2, 3, 7

    2024

  14. [23]

    Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal naviga- tion

    Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal naviga-...

  15. [24]

    In- teractive task planning with language models.arXiv preprint arXiv:2310.10645, 2023

    Boyi Li, Philipp Wu, Pieter Abbeel, and Jitendra Malik. In- teractive task planning with language models.arXiv preprint arXiv:2310.10645, 2023. 2

    2023 arXiv

  16. [25]

    Intergen: Diffusion-based multi-human motion genera- tion under complex interactions

    Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human motion genera- tion under complex interactions. International Journal of Computer Vision, 132(9):3463–3483, 2024. 1, 2

    2024

  17. [26]

    Code as policies: Language model programs for embodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 9493–9500. IEEE, 2023. 1, 2, 4

    2023

  18. [27]

    Phys- reaction: Physically plausible real-time humanoid reaction synthesis via forward dynamics guided 4d imitation

    Yunze Liu, Changxi Chen, Chenjing Ding, and Li Yi. Phys- reaction: Physically plausible real-time humanoid reaction synthesis via forward dynamics guided 4d imitation. In Pro- ceedings of the 32nd ACM International Conference on Mul- timedia, pages 3771–3780, 2024. 1, 2

    2024

  19. [28]

    Amass: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442–5451, 2019. 11

    2019

  20. [29]

    Generating continual human motion in diverse 3d scenes

    Aymen Mir, Xavier Puig, Angjoo Kanazawa, and Gerard Pons-Moll. Generating continual human motion in diverse 3d scenes. In 2024 International Conference on 3D Vision (3DV), pages 903–913. IEEE, 2024. 1, 2

    2024

  21. [30]

    Gsm- symbolic: Understanding the limitations of mathemati- cal reasoning in large language models

    Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, On- cel Tuzel, Samy Bengio, and Mehrdad Farajtabar. Gsm- symbolic: Understanding the limitations of mathemati- cal reasoning in large language models. arXiv preprint arXiv:2410.05229, 2024. 2, 5

    arXiv

  22. [31]

    Grid: Scene-graph-based instruction-driven robotic task planning

    Zhe Ni, Xiaoxin Deng, Cong Tai, Xinyue Zhu, Qinghong- bing Xie, Weihang Huang, Xiang Wu, and Long Zeng. Grid: Scene-graph-based instruction-driven robotic task planning. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 13765–13772. IEEE,

    2024

  23. [32]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Mered- ith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023. 2

    2023

  24. [33]

    Expressive body capture: 3d hands, face, and body from a single image

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    2019

  25. [34]

    Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning

    Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou- Chakra, Ian D Reid, and Niko S ¨underhauf. Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning. In CoRL, 2023. 1, 2

    2023

  26. [35]

    Towards open domain text-driven synthesis of multi-person motions

    Mengyi Shan, Lu Dong, Yutao Han, Yuan Yao, Tao Liu, Ifeoma Nwogu, Guo-Jun Qi, and Mitch Hill. Towards open domain text-driven synthesis of multi-person motions. In European Conference on Computer Vision , pages 67–86. Springer, 2024. 1, 2

    2024

  27. [36]

    Large language models can be easily distracted by irrelevant context

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Sch ¨arli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR, 2023. 2

    2023

  28. [37]

    Cooperative pathfinding

    David Silver. Cooperative pathfinding. In Proceedings of the aaai conference on artificial intelligence and interactive digital entertainment, pages 117–122, 2005. 5, 9

    2005

  29. [38]

    Progprompt: Generating situated robot task plans using large language models

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE In- ternational Conference on Robotics and Automat...

    2023

  30. [39]

    Retargetable ar: Context-aware augmented real- ity in indoor scenes based on 3d scene graph

    Tomu Tahara, Takashi Seno, Gaku Narita, and Tomoya Ishikawa. Retargetable ar: Context-aware augmented real- ity in indoor scenes based on 3d scene graph. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 249–255, 2020. 9

    2020

  31. [40]

    Role-aware interac- tion generation from textual description

    Mikihiro Tanaka and Kent Fujiwara. Role-aware interac- tion generation from textual description. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15999–16009, 2023. 1, 2

    2023

  32. [41]

    V oyager: An open-ended embodied agent with large language models

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandku- mar. V oyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Re- search, 2024. 2

    2024

  33. [42]

    Synthesizing long-term 3d human motion and in- teraction in 3d scenes

    Jiashun Wang, Huazhe Xu, Jingwei Xu, Sifei Liu, and Xiao- long Wang. Synthesizing long-term 3d human motion and in- teraction in 3d scenes. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 9401–9411, 2021. 1, 2

    2021

  34. [43]

    Towards diverse and natural scene-aware 3d human motion synthesis

    Jingbo Wang, Yu Rong, Jingyuan Liu, Sijie Yan, Dahua Lin, and Bo Dai. Towards diverse and natural scene-aware 3d human motion synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20460–20469, 2022. 5

    2022

  35. [44]

    Humanise: Language-conditioned hu- man motion generation in 3d scenes

    Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Humanise: Language-conditioned hu- man motion generation in 3d scenes. Advances in Neural Information Processing Systems, 35:14959–14971, 2022

    2022

  36. [45]

    Move as you say interact as you can: Language-guided human motion generation with scene af- fordance

    Zan Wang, Yixin Chen, Baoxiong Jia, Puhao Li, Jinlu Zhang, Jingze Zhang, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Move as you say interact as you can: Language-guided human motion generation with scene af- fordance. In Proceedings of the IEEE/CVF Conference on Compu...

  37. [46]

    Chain-of-thought prompting elicits reasoning in large lan- guage models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models. Advances in neural information processing systems, 35:24824–24837, 2022. 4

    2022

  38. [47]

    Inter-x: Towards versatile human- human interaction analysis

    Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, et al. Inter-x: Towards versatile human- human interaction analysis. In Proceedings of the IEEE/CVF 15 Conference on Computer Vision and Pattern Recognition ...

    2024

  39. [48]

    Halluci- nation is inevitable: An innate limitation of large language models

    Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Halluci- nation is inevitable: An innate limitation of large language models. arXiv preprint arXiv:2401.11817, 2024. 2

    2024 arXiv

  40. [49]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024. 2, 7

    2024 arXiv

  41. [50]

    Llm- grounder: Open-vocabulary 3d visual grounding with large language model as an agent

    Jianing Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David F Fouhey, and Joyce Chai. Llm- grounder: Open-vocabulary 3d visual grounding with large language model as an agent. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages ...

    2024

  42. [51]

    Generating human interaction motions in scenes with text control

    Hongwei Yi, Justus Thies, Michael J Black, Xue Bin Peng, and Davis Rempe. Generating human interaction motions in scenes with text control. In European Conference on Com- puter Vision, pages 246–263. Springer, 2024. 1, 2

    2024

  43. [52]

    Building cooperative embodied agents modularly with large language models

    Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. In The Twelfth International Confer- ence on Learning Representations, 2024. 2

    2024

  44. [53]

    Synthesizing diverse human motions in 3d in- door scenes

    Kaifeng Zhao, Yan Zhang, Shaofei Wang, Thabo Beeler, and Siyu Tang. Synthesizing diverse human motions in 3d in- door scenes. In Proceedings of the IEEE/CVF international conference on computer vision , pages 14738–14749, 2023. 1, 2 16

    2023

  45. [437]

    Springer, 2024. 1, 2

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.