REVIEW 6 cited by
BlenderAlchemy: Editing 3D Graphics with Vision-Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Graphics design is important for various applications, including movie production and game design. To create a high-quality scene, designers usually need to spend hours in software like Blender, in which they might need to interleave and repeat operations, such as connecting material nodes, hundreds of times. Moreover, slightly different design goals may require completely different sequences, making automation difficult. In this paper, we propose a system that leverages Vision-Language Models (VLMs), like GPT-4V, to intelligently search the design action space to arrive at an answer that can satisfy a user's intent. Specifically, we design a vision-based edit generator and state evaluator to work together to find the correct sequence of actions to achieve the goal. Inspired by the role of visual imagination in the human design process, we supplement the visual reasoning capabilities of VLMs with "imagined" reference images from image-generation models, providing visual grounding of abstract language descriptions. In this paper, we provide empirical evidence suggesting our system can produce simple but tedious Blender editing sequences for tasks such as editing procedural materials and geometry from text and/or reference images, as well as adjusting lighting configurations for product renderings in complex scenes.
Forward citations
Cited by 6 Pith papers
-
ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients"
ToolGrad inverts tool-use dataset generation: build valid tool-call chains first, synthesize queries second, yielding lower cost and near-100% pass rates.
-
DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation
DI-PCG learns to invert procedural 3D generators by diffusing over their parameters, conditioned on DINOv2 image features, producing editable assets in seconds from a photo.
-
Floating Radiance Networks
FlaRe combines per-primitive latent radiance descriptors on planar Gaussians with a shared decoder and hardware ray tracing, making rendering, secondary rays, editing, and mesh extraction work in one scene model.
-
LL3M: Large Language 3D Modelers
A multi-agent LLM system generates editable 3D assets as Blender Python code, using documentation retrieval and visual self-critique to refine results.
-
VLMaterial: Procedural Material Generation with Large Vision-Language Models
A fine-tuned vision-language model can generate editable Blender procedural material programs from a single input image, matching appearance better than prior generative and retrieval baselines.
-
Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era
A survey that organizes over 100 instruction-guided image and multimedia editing papers into a process-based taxonomy, with an emphasis on LLM and MLLM empowered methods.
Discussion (0). Continue with ORCID to comment.