Pith. sign in

REVIEW 4 major objections 4 minor 133 references

Canvas3D argues that direct 3D object manipulation gives users precise spatial control over generated images, and reports it beating slider-based controls on every measured metric.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Canvas3D lets users arrange objects in a 3D canvas generated from a text prompt, then feeds depth, skeleton, and lighting constraints to diffusion models to produce images that match the layout.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A well-built HCI system with a genuine integration, but the evaluation does not isolate the direct-manipulation claim. the 4 major comments →

arxiv 2508.07135 v1 pith:HIN47JUO submitted 2025-08-10 cs.HC

Canvas3D: Empowering Precise Spatial Control for Image Generation with Constraints from a 3D Virtual Canvas

Runlin Duan , Yuzhao Chen , Rahul Jain , Yichen Hu , Jingyu Shi , Karthik Ramani This is my paper
classification cs.HC
keywords Controllable Image GenerationSpatial ControlConditional Generative Models3D InteractionInteractive CanvasDepth ConditioningText-to-Image GenerationUser Study
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the bottleneck in controllable image generation is not the generative model but the input: text, sketches, and sliders are clumsy ways to express where objects should sit in 3D. Canvas3D replaces those with a virtual 3D canvas that a user prompt builds automatically, letting people drag, rotate, pose, and re-light objects as if arranging a real scene. The system then encodes the arrangement into depth images, skeleton maps, and lighting metadata that a conditional generative model can consume. In a within-subject comparison against a slider-based system using the same generation backbone, the paper reports Canvas3D winning on every objective spatial-alignment metric and on user-rated interactivity, control, effort, and frustration. A second open-ended study reports that the system works with everyday prompts and earns a strong usability score.

Core claim

Canvas3D's central claim is that direct 3D manipulation gives users genuinely precise spatial control over generated images, and that this precision survives the trip from user intent to final image. The authors argue that a 3D engine captures spatial intent intuitively because users manipulate actual objects rather than sliders or bounding boxes. The closed-ended study gave participants a target image and asked them to reproduce its spatial composition with Canvas3D or with a slider-based baseline; both used the same conditional generative backbone. Canvas3D outperformed the baseline on all five metrics, with significant advantages on GPT-CLIP (p=0.0024), Uni-Det (p=0.0034), and Recall (p=0

What carries the argument

The load-bearing mechanism is the spatial-condition encoding pipeline: an automatically constructed 3D canvas plus a function library that exports the user's arrangement as depth images, scene screenshots, OpenPose-format skeletons, and lighting JSON, along with native mesh data. This encoding sits between the interaction layer and the generative model; it is what makes the user's mouse movements into constraints a conditional diffusion model can actually obey. The automatic object registration and scene synthesis from the prompt matter too, because they remove the setup burden and keep the comparison about spatial control rather than 3D modeling skill.

Load-bearing premise

The pipeline assumes the encoded spatial conditions, especially the depth image, preserve the user's 3D arrangement intact; the paper's own study shows that objects close in depth can be conflated, and that chaining separate models for pose and lighting adds style inconsistency.

What would settle it

Place two objects very close together in depth on the Canvas3D canvas under the same prompt, and check whether the generated image keeps them as distinct objects; the paper reports this failing for a car near a house. More broadly, rerunning the closed-ended comparison with a larger sample and more scenes would settle whether the Uni-Det and Recall advantages persist beyond the 12 participants.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Users can specify object placement, orientation, human posture, camera viewpoint, and lighting by arranging a 3D scene, then regenerate while keeping the same spatial constraints.
  • Spatial-alignment metrics and user ratings both improve relative to slider-based control when the generation backbone is held fixed.
  • The system lowers the skill barrier: no sketching ability or slider calibration is needed, since the canvas is created automatically from a text prompt.
  • The same encoded conditions (depth, skeleton, lighting) can be retargeted to other conditional generative models through the extensible encoder library.
  • Because the canvas enforces physical constraints, common scene violations such as floating or intersecting objects are reduced before generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The documented near-depth failure suggests depth-only encoding is the weak link; combining depth with instance segmentation or an explicit relation graph would likely resolve cases where two objects merge, and this is a cheap test the paper does not run.
  • The comparison's outcome is tied to the slider baseline's interaction design; a head-to-head against sketch-based or drag-based controllers would clarify whether the advantage comes from 3D direct manipulation or simply from not using sliders.
  • The same prompt-to-canvas workflow could plausibly steer non-image generative tasks, such as 3D model generation or embodied-agent instructions, since the system already exports native 3D meshes and scene metadata; the paper only sketches those uses.
  • The reported effect sizes come from 12 participants; a larger replication varying scenes and user backgrounds would show how far the advantage generalizes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents Canvas3D, an interactive system for image generation with explicit spatial control. A user enters a text prompt; the system registers 3D objects from ShapeNet/Objaverse, synthesizes an initial scene with an LLM, and maps mouse/keyboard input to object affordances in a Unity-based canvas. The user can rearrange objects, adjust camera and lighting, and configure human posture. The system encodes the resulting arrangement into spatial conditions (depth, skeleton, lighting JSON, etc.) that are passed to conditional generative models (Uni-Con, IC-Light). The paper reports a within-subject study (n=12) comparing Canvas3D to a slider-based baseline with the same generative backbone, finding significant advantages on GPT-CLIP, Uni-Det, Recall, and several NASA-TLX/Likert measures, plus an open-ended usability session with SUS score 82.22. The central claim is that direct 3D manipulation gives users significantly better spatial control than slider-based control.

Significance. If the result holds, Canvas3D is a valuable contribution to controllable generation: it provides an end-to-end pipeline from prompt to manipulable 3D scene to spatial conditions, and the interaction design aligns with natural manipulation. The system is implemented and documented in enough detail to be reproduced, and the use of a within-subject design with counterbalancing and qualitative interviews is appropriate. However, the current evaluation cannot uniquely attribute the observed gains to the proposed manipulation mechanism because the baseline also differs in visual representation and condition fidelity. The paper would need a controlled comparison or additional experiment to support its central claim.

major comments (4)
  1. [§5.1.1, Figures 10/15] The comparison in §5.1.1/Figures 10,15 does not isolate the proposed 3D-manipulation interaction. The baseline differs from Canvas3D in at least three ways: input modality (sliders vs direct manipulation), visual feedback (bounding boxes vs textured meshes), and the fidelity of the spatial condition (box-derived depth vs mesh-derived depth) fed to the same Uni-Con backbone. §6.1.3 itself attributes baseline failures to bounding boxes being 'frequently misinterpreted as a single object.' Thus the significant gains in GPT-CLIP, Uni-Det, and Recall (Table 1) could arise from the condition-encoding difference alone. To support the central HCI claim, the study needs a control that holds the condition representation fixed (e.g., sliders with mesh-based depth) or adds a third condition isolating each factor.
  2. [§5.1.2, Figure 18] The closed-ended study uses a single target image (Fig. 18) with one object set and one spatial layout; Table 3 aggregates counts/times for that stimulus. With n=12 and one stimulus, the claim that Canvas3D 'consistently outperforms' the baseline does not generalize across object categories, scene complexity, or spatial arrangements. Additional target scenes (or at least a per-stimulus analysis and a clear acknowledgment of this scope limit) are needed before drawing general conclusions about spatial controllability.
  3. [§5.1.3, Table 1] Table 1 reports five objective metrics without correction for multiple comparisons, and Fig. 13 adds many subjective tests; the smallest p-values would survive Bonferroni, but the authors should report adjusted p-values or FDR and include effect sizes/confidence intervals so readers can judge magnitudes. In addition, GPT-CLIP and GPT Spatial rely on GPT-generated captions/judgments with no reported reliability (e.g., agreement with human raters or repeatability). Because both conditions are evaluated with the same LLM judge, this is not circular, but it is a source of measurement uncertainty that should be quantified.
  4. [§7.3] Section 7.3 documents that the depth condition can lose spatial distinctions when objects are close in depth (P11 quote). This is an acknowledged limitation, but the discussion does not connect it to the closed-ended comparison. Since the two conditions use different depth encoders (mesh vs boxes), this failure mode may affect the condition-fidelity confound differently across conditions, and it also bounds the central 'precise spatial control' claim. Please discuss how this limitation interacts with the objective metrics and whether the system-level advantages persist when depth resolution is the bottleneck.
minor comments (4)
  1. [Figures 2/4/9] Figure 2 contains an untranslated editing note ('放citation'), and Figures 4 and 9 contain Chinese annotation text ('字加大', '字体加粗加大'). These are leftover author annotations and must be removed.
  2. [§6.2.1/6.2.2] Sections 6.2.1 and 6.2.2 are both titled 'System Usability Questionnaire'; the second appears to be the System Feature Questionnaire. There are also typos: 'Metrice' (§5.1.3), 'Geneartive' (§2.2 heading), 'faciliate', 'perprndicular', and 'wildly'.
  3. [Table 3] Table 3 reports time-to-first-liked and liked-ratio rows without p-values or confidence intervals. If these are exploratory, say so explicitly; otherwise provide the corresponding tests.
  4. [Appendix A.6] The Uni-Det score is defined by listing five spatial relationships, but the exact formula for comparing positions/depths of detected boxes is not given (thresholds, normalization, per-relationship scoring). As written, the metric is not fully reproducible.

Circularity Check

0 steps flagged

No significant circularity: the comparison is an empirical user study with external generative backbones and metrics; no fitted parameter is renamed as a prediction and no load-bearing self-citation chain appears.

full rationale

This paper contains no formal derivation whose conclusion equals its premises. The central claim is an empirical comparison: Canvas3D outperforms a slider/bounding-box baseline on spatial metrics and subjective ratings. The generation backbone (Uni-Con), lighting model (IC-Light), and the quantitative metrics (CLIP, UniDet, Recall) are external to the system, and the perceived-control ratings come from 12 participants. The only passages that could resemble circularity are (1) the taxonomy of spatial composition, which is a design frame rather than a derived prediction; (2) several self-citations in related work, none of which supplies the evaluation's force; and (3) the use of GPT-family models both for scene synthesis and for some caption-based metrics, which is a possible shared-model bias but not a definitional reduction — the metrics are computed on images produced by an external conditional diffusion model, not on the system's own outputs in a way that would force the reported differences. The baseline confound (3D meshes vs. bounding boxes, direct manipulation vs. sliders) is a threat to attribution, not a circularity: the comparison is underdetermined but empirically meaningful. No fitted parameter is renamed as a prediction, and no load-bearing uniqueness/self-citation chain forces the conclusion.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claims rest on several domain assumptions rather than on fitted parameters: that 3D direct manipulation better captures intent, that conditional models render encoded constraints faithfully, that LLM-initialized layouts do not skew the comparison, and that LLM-based metrics are valid. No new physical or conceptual entities are introduced; the system is an assembly of existing models and interaction components. The two listed free parameters are evaluation design choices, not model parameters.

free parameters (2)
  • Uni-Det relationship set (5 relations) = house-front-left-trees; house-back-left-lamp; house-back-car; car-back-right-bench; bench-back-flowerpot
    Hand-curated for the single target image in the closed-ended study. The Uni-Det metric score depends directly on this list, so the reported spatial accuracy is tied to this choice.
  • GPT Spatial rubric thresholds = 20, 40, 60, 80, 100
    Hand-written scoring levels in the GPT-4V meta-prompt in Appendix A.2.5. No sensitivity analysis or human calibration is provided, so the spatial score scale is arbitrary.
axioms (4)
  • domain assumption Direct manipulation in a 3D canvas captures spatial intent more accurately and intuitively than slider-based control.
    This is the central design premise. The user study supports it relative to a self-built slider baseline, but it does not isolate 3D manipulation from the use of visible mesh objects, an LLM-initialized layout, or other confounds.
  • domain assumption The conditional generative backbones (Uni-Con, IC-Light) faithfully render depth, skeleton, and lighting conditions produced by the system.
    The claim that generated images 'accurately reflect user intentions' depends on external models. Section 7.3 documents a depth-confusion failure, showing the assumption does not always hold.
  • domain assumption LLM-based scene synthesis produces reasonable initial layouts that do not bias the comparison.
    The baseline lacks an equivalent LLM-initialized layout. A good starting scene may reduce user effort and improve final images independent of the interaction design, so the comparison is not fully controlled.
  • domain assumption GPT-4V and CLIP-based scores are valid proxies for spatial accuracy and image quality.
    Quantitative results in Table 1 rely on LLM-generated captions, GPT-4V spatial judgments, and CLIP embeddings. These are used as ground truth without human annotation or calibration against perceptual judgments.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Canvas3D: Empowering Precise Spatial Control for Image Generation with Constraints from a 3D Virtual Canvas." pith.science (2026). https://pith.science/paper/HIN47JUO

@misc{pith2026250807135,
  author       = {Pith},
  title        = {Pith review of: Canvas3D: Empowering Precise Spatial Control for Image Generation with Constraints from a 3D Virtual Canvas},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HIN47JUO}},
  note         = {Machine review of arXiv:2508.07135}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Generative AI (GenAI) has significantly advanced the ease and flexibility of image creation. However, it remains a challenge to precisely control spatial compositions, including object arrangement and scene conditions. To bridge this gap, we propose Canvas3D, an interactive system leveraging a 3D engine to enable precise spatial manipulation for image generation. Upon user prompt, Canvas3D automatically converts textual descriptions into interactive objects within a 3D engine-driven virtual canvas, empowering direct and precise spatial configuration. These user-defined arrangements generate explicit spatial constraints that guide generative models in accurately reflecting user intentions in the resulting images. We conducted a closed-end comparative study between Canvas3D and a baseline system. And an open-ended study to evaluate our system "in the wild". The result indicates that Canvas3D outperforms the baseline on spatial control, interactivity, and overall user experience.

Figures

Figures reproduced from arXiv: 2508.07135 by Jingyu Shi, Karthik Ramani, Rahul Jain, Runlin Duan, Yichen Hu, Yuzhao Chen.

Figure 1
Figure 1. Figure 1: The workflow of controlling spatial composition using Canvas3D. A user starts by prompting the desired contents to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The taxonomy of spatial composition. Including [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The mouse-and-keyboard interaction design for the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: The architecture of the scene synthesis module. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The decision tree algorithm (Algorithm 2) creates interactive objects by three stages: (1) classifying the input objects, [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example of spatial condition encoding result. The [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The user interface of Canvas3D. The user interface consists of three components: (a) 3D interactive Canvas, (b) prompt ⼤ [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The architecture of the user interface integration. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The example of comparing user-generated images using Canvas3D (left three columns) and a baseline (right three [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: The example of user-generated images using Canvas3D for scene depiction in the open-ended experiment. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: The example of user-generated images using Canvas3D for pose depiction in the open-ended experiment. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: NASA-TLX and system comparison. Our system demonstrated significantly improved spatial control (Canvas3D: [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: The result for the Open-Ended Experiment: The [PITH_FULL_IMAGE:figures/full_fig_p015_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Baseline Interface Strongly Disagree Disagree Somewhat Disagree Neutral Somewhat Agree Agree Strongly Agree [PITH_FULL_IMAGE:figures/full_fig_p019_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: System Usability Scale A.2.2 Meta Prompt for Object Registration. System Content: You are a professional scene designer. You have 20 [indoor/outdoor] furniture categories (listed as [categories list]). Based on the user’s requirements, select the categories needed to create the requested scene. For each chosen category, specify how many items from that category should be included. You should also consider… view at source ↗
Figure 17
Figure 17. Figure 17: Target Image for Controllabily Evaluation [PITH_FULL_IMAGE:figures/full_fig_p020_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Masked Target Image for Controllabily Evaluation [PITH_FULL_IMAGE:figures/full_fig_p020_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

133 extracted references · 58 canonical work pages

  1. [1]

    Remi Alkemade, Fons J Verbeek, and Stephan G Lukosch. 2017. On the efficiency of a VR hand gesture-based interface for 3D object manipulations in conceptual design. International Journal of Human–Computer Interaction 33, 11 (2017), 882–901

    2017

  2. [2]

    Autodesk. 2025. Autodesk | 3D Design, Engineering & Construction Software. https://www.autodesk.com/ Accessed: 2025-04-09

    2025

  3. [3]

    Omri Avrahami, Rinon Gal, Gal Chechik, Ohad Fried, Dani Lischinski, Arash Vahdat, and Weili Nie. 2024. Diffuhaul: A training-free method for object dragging in images. In SIGGRAPH Asia 2024 Conference Papers . 1–12

    2024

  4. [4]

    Lonni Besançon, Paul Issartel, Mehdi Ammi, and Tobias Isenberg. 2017. Mouse, tactile, and tangible input for 3D manipulation. In Proceedings of the 2017 CHI conference on human factors in computing systems . 4727–4740

    2017

  5. [5]

    Shariq Farooq Bhat, Niloy Mitra, and Peter Wonka. 2024. Loosecontrol: Lift- ing controlnet for generalized depth conditioning. In ACM SIGGRAPH 2024 Conference Papers. 1–11

    2024

  6. [6]

    Stephen Brade, Bryan Wang, Mauricio Sousa, Sageev Oore, and Tovi Gross- man. 2023. Promptify: Text-to-image generation through interactive prompt exploration with large language models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology . 1–14

    2023

  7. [7]

    Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. 2024. Spatialbot: Precise spatial understanding with vision language models. arXiv preprint arXiv:2406.13642 (2024)

    Pith/arXiv arXiv 2024

  8. [8]

    Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2019. Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE transactions on pattern analysis and machine intelligence 43, 1 (2019), 172– 186

    2019

  9. [9]

    Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime multi- person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition . 7291–7299

    2017

  10. [10]

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al

  11. [11]

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. 2023. Attend-and-excite: Attention-based semantic guidance for text-to-image diffu- sion models. ACM transactions on Graphics (TOG) 42, 4 (2023), 1–10

    2023

  12. [12]

    DaEun Choi, Sumin Hong, Jeongeon Park, John Joon Young Chung, and Juho Kim. 2024. CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AI. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems . 1–25

    2024

  13. [13]

    John Joon Young Chung and Eytan Adar. 2023. Promptpaint: Steering text-to- image generation through paint medium-like interactions. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology . 1–17

    2023

  14. [14]

    Charles E Connor and James J Knierim. 2017. Integration of objects and space in perception and memory. Nature neuroscience 20, 11 (2017), 1493–1503

    2017

  15. [15]

    Hai Dang, Lukas Mecke, and Daniel Buschek. 2022. Ganslider: How users control generative models for images using multiple sliders with and without feedforward information. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems . 1–15

    2022

  16. [16]

    Jonas Degrave, Michiel Hermans, Joni Dambre, and Francis Wyffels. 2019. A dif- ferentiable physics engine for deep learning in robotics.Frontiers in neurorobotics 13 (2019), 6

    2019

  17. [17]

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli Van- derBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi

  18. [18]

    Francine L Dolins and Robert W Mitchell. 2010. Spatial cognition, spatial percep- tion: mapping the self and space . Cambridge University Press

    2010

  19. [19]

    Wenqi Dong, Bangbang Yang, Lin Ma, Xiao Liu, Liyuan Cui, Hujun Bao, Yuewen Ma, and Zhaopeng Cui. 2024. Coin3d: Controllable and interactive 3d assets generation with proxy-guided conditioning. InACM SIGGRAPH 2024 Conference Papers. 1–10

    2024

  20. [20]

    Runlin Duan, Xiyun Hu, Min Liu, Jingyu Shi, and Karthik Ramani. 2025. pARa- metric: Empowering In Situ Parametric Modeling in Augment Reality for Per- sonal Fabrication. Journal of Computing and Information Science in Engineering 25, 4 (2025), 041001

    2025

  21. [21]

    Runlin Duan, Nachiketh Karthik, Jingyu Shi, Rahul Jain, Maria C Yang, and Karthik Ramani. 2024. ConceptVis: Generating and Exploring Design Con- cepts for Early-Stage Ideation Using Large Language Model. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Vol. 88377. American Society of Mechan...

    2024

  22. [22]

    David Eberly. 2006. 3D game engine design: a practical approach to real-time computer graphics. CRC Press

    2006

  23. [23]

    Abdelrahman Eldesokey and Peter Wonka. 2024. Build-a-scene: Interac- tive 3d layout control for diffusion-based image generation. arXiv preprint arXiv:2408.14819 (2024)

    Pith/arXiv arXiv 2024

  24. [24]

    Exocortex Technologies

    Inc. Exocortex Technologies. 2013. Clara.io: Online 3D Modeling, 3D Rendering, Free 3D Models. https://clara.io/ Accessed: 2025-04-07

    2013

  25. [25]

    Nan Fairbrother. 1974. The nature of landscape design . Architectural Press London

    1974

  26. [26]

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. 2023. Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems 36 (2023), 18225–18250

    2023

  27. [27]

    Donato Fiorella, Andrea Sanna, and Fabrizio Lamberti. 2010. Multi-touch user interface evaluation for 3D object manipulation on mobile devices. Journal on Multimodal User Interfaces 4 (2010), 3–10

    2010

  28. [28]

    Gaspare Galati, Gina Pelle, Alain Berthoz, and Giorgia Committeri. 2010. Multi- ple reference frames used by the human brain for spatial perception and memory. Experimental brain research 206 (2010), 109–120

    2010

  29. [29]

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. 2024. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405.10314 (2024)

    Pith/arXiv arXiv 2024

  30. [30]

    James J Gibson. 2014. The ecological approach to visual perception: classic edition . Psychology press. Canvas3D: Empowering Precise Spatial Control for Image Generation with Constraints from a 3D Virtual Canvas Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

    2014

  31. [31]

    Eg Su Goh, Mohd Shahrizal Sunar, and Ajune Wanis Ismail. 2019. 3D object manipulation techniques in handheld mobile augmented reality interface: A review. IEEE Access 7 (2019), 40581–40601

    2019

  32. [32]

    Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, and Yezhou Yang. 2022. Benchmarking spatial relationships in text-to-image generation. arXiv preprint arXiv:2212.10015 (2022)

    Pith/arXiv arXiv 2022

  33. [33]

    Bruce Gooch, Erik Reinhard, Chris Moulding, and Peter Shirley. 2001. Artistic composition for image creation. In Rendering Techniques 2001: Proceedings of the Eurographics Workshop in London, United Kingdom, June 25–27, 2001 12. Springer, 83–88

    2001

  34. [34]

    Daniel Gopher, Lilach Armony, and Yaakov Greenshpan. 2000. Switching tasks and attention policies. Journal of Experimental Psychology: General 129, 3 (2000), 308

    2000

  35. [35]

    Daniel J Graham and Christoph Redies. 2010. Statistical regularities in art: Relations with visual coding and perception. Vision research 50, 16 (2010), 1503–1509

    2010

  36. [36]

    SG Hart. 1988. Development of NASA-TLX (Task Load Index): Results of empir- ical and theoretical research. Human mental workload/Elsevier (1988)

    1988

  37. [37]

    Fengming He, Xiyun Hu, Jingyu Shi, Xun Qian, Tianyi Wang, and Karthik Ramani. 2023. UBI edge: authoring edge-based opportunistic tangible user interfaces in augmented reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1–14

    2023

  38. [38]

    Yihan Hou, Manling Yang, Hao Cui, Lei Wang, Jie Xu, and Wei Zeng. 2024. C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Lan- guage Model. In Proceedings of the CHI Conference on Human Factors in Comput- ing Systems. 1–18

    2024

  39. [39]

    Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu

  40. [40]

    Zheng Huang, Qiang Liu, Stephen Westland, Michael R Pointer, M Ronnier Luo, and Kaida Xiao. 2018. Light dominates colour preference when correlated colour temperature differs. Lighting Research & Technology 50, 7 (2018), 995–1012

    2018

  41. [41]

    Adobe Inc. 2025. Adobe Substance 3D Stager. https://www.adobe.com/products/ substance3d/apps/stager.html Accessed: 2025-04-07

    2025

  42. [42]

    Jacek Jankowski and Martin Hachet. 2013. A survey of interaction techniques for interactive 3D environments. In Eurographics 2013-STAR

    2013

  43. [43]

    Youngseung Jeon, Seungwan Jin, Patrick C Shih, and Kyungsik Han. 2021. Fash- ionQ: an ai-driven creativity support tool for facilitating ideation in fashion design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18

    2021

  44. [44]

    Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu

  45. [45]

    Shehroz S Khan and Daniel Vogel. 2012. Evaluating visual aesthetics in photo- graphic portraiture. In Proceedings of the Eighth Annual Symposium on Compu- tational Aesthetics in Graphics, visualization, and imaging . 55–62

    2012

  46. [46]

    Minseok Kim and Jae Yeol Lee. 2016. Touch and hand gesture-based interac- tions for directly manipulating 3D virtual objects in mobile augmented reality. Multimedia Tools and Applications 75 (2016), 16529–16550

    2016

  47. [47]

    Janin Koch, Andrés Lucero, Lena Hegemann, and Antti Oulasvirta. 2019. May AI? Design ideation with cooperative contextual bandits. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems . 1–12

    2019

  48. [48]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Humansd: A native skeleton-guided diffusion model for human image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 15988–15998

  49. [49]

    Mieke HR Leyssen, Sarah Linsen, Jonathan Sammartino, and Stephen E Palmer

  50. [50]

    Boyu Li, Linping Yuan, Zhe Yan, Qianxi Liu, Yulin Shen, and Zeyu Wang. 2024. AniCraft: Crafting Everyday Objects as Physical Proxies for Prototyping 3D Character Animation in Mixed Reality. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology . 1–14

    2024

  51. [51]

    Xirui Li, Charles Herrmann, Kelvin CK Chan, Yinxiao Li, Deqing Sun, Chao Ma, and Ming-Hsuan Yang. 2024. A simple approach to unifying diffusion-based conditional generation. arXiv preprint arXiv:2410.11439 (2024)

    Pith/arXiv arXiv 2024

  52. [52]

    James M Leonhardt, Jesse R Catlin, and Dante M Pirouz. 2015. Is your prod- uct facing the ad’s center? Facing direction affects processing fluency and ad evaluation. Journal of Advertising 44, 4 (2015), 315–325

    2015

  53. [53]

    David Chuan-En Lin, Hyeonsu B Kang, Nikolas Martelaro, Aniket Kittur, Yan- Ying Chen, and Matthew K Hong. 2024. Inkspire: Sketching Product Designs with AI. In Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. 1–6

    2024

  54. [54]

    David Chuan-En Lin, Hyeonsu B Kang, Nikolas Martelaro, Aniket Kittur, Yan- Ying Chen, and Matthew K Hong. 2025. Inkspire: Supporting Design Ex- ploration with Generative AI through Analogical Sketching. arXiv preprint arXiv:2501.18588 (2025)

    Pith/arXiv arXiv 2025

  55. [55]

    Daochang Liu, Junyu Zhang, Anh-Dung Dinh, Eunbyung Park, Shichao Zhang, and Chang Xu. 2025. Generative Physical AI in Vision: A Survey. arXiv preprint arXiv:2501.10928 (2025)

    Pith/arXiv arXiv 2025

  56. [56]

    Vivian Liu, Jo Vermeulen, George Fitzmaurice, and Justin Matejka. 2023. 3DALL- E: Integrating text-to-image AI in 3D design workflows. In Proceedings of the 2023 ACM designing interactive systems conference . 1955–1977

    2023

  57. [57]

    Wentong Liao, Kai Hu, Michael Ying Yang, and Bodo Rosenhahn. 2022. Text to image generation with semantic-spatial aware gan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 18187–18196

    2022

  58. [58]

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin Johnson. 2023. Scalable 3d captioning with pretrained models. Advances in Neural Information Processing Systems 36 (2023), 75307–75337

    work page 2023

  59. [59]

    Hira Maqbool. 2023. Aesthetic choices: Defining the range of aesthetic views in interactive digital media including games and 3D virtual environments (3D VEs). (2023)

    work page 2023

  60. [60]

    Daniel Mendes, Fabio Marco Caputo, Andrea Giachetti, Alfredo Ferreira, and Joaquim Jorge. 2019. A survey on 3d virtual object manipulation: From the desktop to immersive virtual environments. InComputer graphics forum, Vol. 38. Wiley Online Library, 21–45

    work page 2019

  61. [61]

    Siwei Meng, Yawei Luo, and Ping Liu. 2025. Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC. arXiv preprint arXiv:2502.07007 (2025)

    Pith/arXiv arXiv 2025

  62. [62]

    Ziyi Liu, Zhengzhe Zhu, Enze Jiang, Feichi Huang, Ana M Villanueva, Xun Qian, Tianyi Wang, and Karthik Ramani. 2023. Instrumentar: Auto-generation of augmented reality tutorials for operating digital instruments through recording embodied demonstration. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1–17

    work page 2023

  63. [63]

    Sicheng Mo, Fangzhou Mu, Kuan Heng Lin, Yanli Liu, Bochen Guan, Yin Li, and Bolei Zhou. 2024. Freecontrol: Training-free spatial control of any text-to-image diffusion model with any condition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 7465–7475

    work page 2024

  64. [64]

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. 2024. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, Vol. 38. 4296–4304

    work page 2024

  65. [65]

    Maria Adriana Neroni, Alfred Oti, and Nathan Crilly. 2021. Virtual Reality design-build-test games with physics simulation: opportunities for researching design cognition. International Journal of Design Creativity and Innovation 9, 3 (2021), 139–173

    work page 2021

  66. [66]

    Jeongseok Oh, Seungju Kim, and Seungjun Kim. 2024. LumiMood: A Creativity Support Tool for Designing the Mood of a 3D Scene. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems . 1–21

    2024

  67. [67]

    Zhenxing Mi, Kuan-Chieh Wang, Guocheng Qian, Hanrong Ye, Runtao Liu, Sergey Tulyakov, Kfir Aberman, and Dan Xu. 2025. I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models. arXiv preprint arXiv:2502.10458 (2025)

    Pith/arXiv arXiv 2025

  68. [68]

    OpenAI. 2025. OpenAI. https://openai.com/ Accessed: 2025-04-09

    work page 2025

  69. [69]

    Jonas Oppenlaender, Rhema Linder, and Johanna Silvennoinen. 2024. Prompting AI art: An investigation into the creative skill of prompt engineering. Interna- tional journal of human–computer interaction (2024), 1–23

    work page 2024

  70. [70]

    Stephen E Palmer, Jonathan S Gardner, and Thomas D Wickens. 2008. Aesthetic issues in spatial composition: Effects of position and direction on framing single objects. Spatial vision 21, 3 (2008), 421–450

    work page 2008

  71. [71]

    Stephen E Palmer, Karen B Schloss, and Jonathan Sammartino. 2013. Visual aesthetics and human preference. Annual review of psychology 64, 1 (2013), 77–107

    work page 2013

  72. [72]

    Open Robotics. 2025. Robot Operating System (ROS). https://www.ros.org/ Accessed: 2025-04-09

    work page 2025

  73. [73]

    Henry Rankin Poore. 1976. Composition in art. Courier Corporation

    work page 1976

  74. [74]

    Xun Qian, Fengming He, Xiyun Hu, Tianyi Wang, and Karthik Ramani. 2022. Arnnotate: An augmented reality interface for collecting custom dataset of 3d hand-object interaction pose estimation. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology . 1–14

    work page 2022

  75. [75]

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al . 2023. Unicontrol: A unified diffusion model for controllable visual generation in the wild. arXiv preprint arXiv:2305.11147 (2023)

    Pith/arXiv arXiv 2023

  76. [76]

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen

  77. [77]

    Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. 2023. Drag your gan: Interactive point-based manipu- lation on the generative image manifold. In ACM SIGGRAPH 2023 conference proceedings. 1–11

    work page 2023

  78. [78]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)

    Pith/arXiv arXiv 2019

  79. [79]

    Jason L Reisman, Philip L Davidson, and Jefferson Y Han. 2009. A screen-space formulation for 2D and 3D direct manipulation. InProceedings of the 22nd annual ACM symposium on User interface software and technology . 69–78. Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Runlin Duan, Yuzhao Chen, Rahul Jain, Yichen Hu, Jingyu Shi, and Karthik Ramani

    work page 2009

  80. [80]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695

    2022

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.