Pith. sign in

REVIEW 3 major objections 4 minor 67 references

Handle-based Mesh Deformation Guided By Vision Language Model

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A vision-language model can replace manual handle placement for text-driven mesh deformation, with a handle-based interface keeping output quality high.

desk verdict Useful VLM-driven handle deformation pipeline, but the 'arbitrary mesh' claim rests on a favorable subset of shapes and the headline metric shares a model family with the method. read the letter →

arxiv 2506.04562 v2 pith:ZQFSTMZK submitted 2025-06-05 cs.GR cs.CV

Xingpeng Sun , Shiyang Jia , Zherong Pan , Kui Wu , Aniket Bera This is my paper
classification cs.GRcs.CV
keywords handle-baseddeformationmeshvision-languagemodeltext-driven3Deditingconesingularitymulti-viewvotingboundedbiharmonicweightstraining-free
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a training-free pipeline that lets a user deform any 3D mesh by typing an instruction such as "raise its head" or "stretch all chair legs forward." A vision-language model (VLM) reads multi-view renders of the mesh, breaks the instruction into sub-steps, identifies the part to move, and picks drag handles from a candidate set found by cone singularity detection. The VLM's 2D drag predictions are converted into 3D handle motions through a membrane-energy-regularized optimization, and results from several camera views are averaged to suppress prediction noise. The central claim is that restricting deformation to a handle-based interface preserves fine detail and yields high-quality results without per-category training, outperforming text-driven baselines in alignment and plausibility scores.

What carries the argument

The load-bearing mechanism is the handle-based deformation interface, specifically bounded biharmonic weights (BBW), combined with cone singularity detection to generate a sparse superset of candidate handles. Cone singularities—vertices where Gauss curvature concentrates, such as the tips of horns or limbs—provide geometrically meaningful points for the VLM to choose; once the VLM predicts a drag vector in 2D for a chosen handle, the method solves a membrane-energy-regularized least-squares problem for the handle positions and propagates motion to the whole mesh using BBW. Multi-view voting averages the per-camera solutions so that 2D-to-3D projection bias and VLM uncertainty cancel out.

What would settle it

Render a perfectly smooth, featureless mesh such as a sphere or ellipsoid, prompt it with a clear deformation like "stretch the top upward," and run the pipeline: if cone singularity detection returns no candidate handles inside the segmented region, the method cannot produce any deformation, confirming the limiting case stated in Sec. VI.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that handle-based deformation can serve as a geometric regularizer for text-driven editing: instead of optimizing thousands of vertices against a CLIP loss, the method lets a VLM choose a few existing feature points, predict where to drag them in screen space, and lets a standard deformation model fill in the rest. The VLM is used in four roles—decomposing the prompt, segmenting the sub-part via 2D open-vocabulary masks fused by graph cut, selecting handles, and predicting drag directions—so the whole system is open-vocabulary and requires no training on object categories. The paper argues this is why its outputs score higher on GPTEval3D and show lower membrane energy distortion than TextDeformer, MeshUp, and MagicClay despite a slightly lower CLIP ViT-B/32 score, which TextDeformer optimizes directly as its loss.

Load-bearing premise

The method assumes that cone singularities—points where the surface curvature concentrates—will exist on the part to be deformed, so a smooth or featureless object like a sphere offers no useful handles for the VLM to drag, a failure the paper itself concedes.

Editorial extensions

If this is right

  • Any watertight mesh and any text instruction can be edited without per-category training or fine-tuning of a deformation model; only the VLM is optionally fine-tuned on a small dataset of artist-annotated handle drags.
  • Because the deformed space is low-frequency, fine-grained surface detail and sharp features of the input mesh are preserved, at the cost of being unable to synthesize new high-frequency detail.
  • Averaging over six axis-aligned views makes results robust to prompt paraphrase and to VLM coordinate uncertainty, so semantically similar prompts yield consistent deformations.
  • The pipeline's intermediate outputs (segmentation masks, handles, and drags) are explainable and user-editable, allowing artists to correct a result by adjusting a mask or a drag rather than restarting.
  • The same VLM-guided handle selection can drive other handle-based deformation models, since experiments with ARAP produce results similar to those with BBW.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If VLM spatial reasoning continues to improve, the multi-view voting and fine-tuning components may become less necessary, but the cone-singularity handle superset would remain the bottleneck for smooth shapes.
  • The framework suggests a general design principle: use a geometric low-degree-of-freedom interface to regularize any generative or language model that outputs imprecise spatial instructions, not just for meshes but also for cages, skeletons, or splines.
  • A testable extension would be replacing cone singularities with learned or semantic keypoint proposals that work on smooth shapes, which would lift the stated all-smooth failure mode while keeping the handle-based quality guarantee.
  • Because the method is restricted to low-frequency modifications, it could be combined with a detail-generation pass to cover the full spectrum from coarse pose changes to fine surface carving, a division of labor the paper notes but does not implement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a pipeline for text-driven deformation of a single watertight mesh. The user text is decomposed into sub-instructions; for each sub-instruction, a VLM (GPT-4o) selects camera views, and Grounding-DINO/SAM masks are fused by a graph-cut optimization to segment the relevant mesh sub-part. Cone singularities on the sub-part form a candidate handle superset, the VLM selects handles and predicts target positions in image space per view, and an optimization with membrane-energy regularization produces handle displacements that are averaged over views and then applied through biharmonic deformation. The paper reports CLIP and GPTEval3D scores against TextDeformer, MagicClay, and MeshUp, together with ablations, a user study, and a supplement containing the full prompts and fine-tuning data. The central claims are open-vocabulary generalization without per-category retraining and consistently high-quality output guaranteed by the handle-based interface.

Significance. The contribution is practically appealing: it combines classical geometry tools (cone singularity detection, graph cut, biharmonic deformation) with a VLM in an explainable, modular pipeline, and it is computationally light (about 8 minutes and 10 VLM API calls per example). The user study provides independent human evidence for the qualitative claims, and the supplementary material is transparent about prompts, fine-tuning data, and failure cases. The strengths are real: the authors share full prompts, provide an ablation of handle selection and multi-view voting, and report run-time breakdowns. However, the significance of the central claim is tempered by the fact that the evaluation set and the handle-superset construction favor high-curvature, semantically rich models, and by the reliance on a same-family VLM as the alignment evaluator.

major comments (3)
  1. [§IV-C and §VI] The central claim of open-vocabulary deformation of arbitrary meshes is not supported for meshes whose deformable parts lack cone singularities. The handle superset \bar{H} is generated by cone singularity detection, and the paper concedes in §VI that the method "may be flawed on all-smooth shapes, e.g. a sphere." The evaluation in §V is restricted to 150 "organic"/"semantic-rich" shapes, which selects for high-curvature parts and excludes the failure regime. Moreover, the "Without GPT Handle Selection" ablation in Table I randomly draws from \bar{H}, so it shows that VLM selection improves on random selection within \bar{H}, but it does not establish that \bar{H} contains a handle sufficient to realize each prompt. The quality ceiling imposed by \bar{H} is therefore unknown. Please add experiments on smooth or low-curvature meshes, or compare against an oracle handle set or manual handle placement on a subset, or explicitly narrow the claimed scope.
  2. [§IV-D and Table II] The abstract and introduction repeatedly describe the method as "training-free," but the pipeline in §IV-D includes supervised fine-tuning of GPT-4o on an artist-created dataset, and the numbers reported for "Ours" in Tables I and III match the fine-tuned row of Table II (CLIP 0.2524, GPTEval3D Overall 1147.5173). This is an internal inconsistency: if the headline results rely on fine-tuning, the "training-free" claim should be qualified to mean "no per-category training," or the main quantitative results should also be reported with the unfine-tuned VLM.
  3. [§V and Table III] The headline alignment metric may be confounded by model-family affinity. GPTEval3D uses GPT-4V as the judge, while the proposed method uses GPT-4o; both are vision-language models from the same family. Because the same kind of VLM both produces the deformation and evaluates it, the reported GPTEval3D advantage (1205 vs. 981–1037 for Prompt Asset Alignment) could partly reflect the evaluator's preference for outputs generated by a model with similar behavior rather than genuine semantic alignment. The user study (93.53% alignment) provides independent evidence, but it covers only 16 prompt-mesh sets and is reported without full statistical detail. Please discuss this risk and either report human ratings on a larger subset or compare with an independent evaluator (e.g., a different VLM family or a non-VLM text-image retrieval metric).
minor comments (4)
  1. [§V] There are typos: "utilze" should be "utilize," and "we only requires a network-independent optimization" should be "we only require."
  2. [Fig. 4 and Fig. 20] The handle visualization color is inconsistent: Figure 4 says the handle superset is drawn in yellow, while the prompt in Figure 20 of the supplement describes handles visualized in green spheres.
  3. [§V-d] The user study reports aggregate percentages (93.53% alignment, 85.71% visual salience) but does not state the total number of pairwise comparisons, how ties were handled, or inter-rater agreement; please add these details.
  4. [§VI] "Unsymmetric deformation" should be "asymmetric deformation" or "non-symmetric deformation."

Circularity Check

1 steps flagged · score 4.0 of 10

Membrane-energy quality metric is the optimization objective itself; central VLM handle-selection claim retains independent CLIP and user-study support.

  1. fitted input called prediction [Section IV-D (Eq. 2) and Section V-c (Table IV and surrounding text)]
    "We optimize for the target positions for all the handle points in the super-set \bar{H}(Cm), regularized by the finite-element-discretized membrane energy [38]. ... Finally, we compare the mesh distortion induced by both methods as measured by Emembrane. As summarized in Table IV, we see that our method generates much smaller distortion on average."

    The membrane energy minimized in Eq. (2) is the same quantity reported as Emembrane in Table IV. The optimization chooses handle positions by minimizing projection error plus lambda*Emembrane, and the remaining vertices are then fixed by biharmonic deformation, so the reported low Emembrane is a direct consequence of the objective being optimized rather than an independent test of output quality. The abstract's claim 'low distortion - quantified via membrane energy' is therefore self-confirming for this metric. The comparison with TextDeformer and MeshUp is still informative as a trade-off statement, but it does not independently validate the quality of the deformations.

full rationale

The paper is an empirical system paper and does not present a formal derivation chain that could reduce to its inputs. The central claim - that a VLM can translate text prompts into handle selections and drags - is tested by CLIP scores, GPTEval3D, and a 21-person user study; the user study and CLIP are independent of the deformation optimization. I find one genuine by-construction element: the reported 'low distortion' metric, Emembrane, is exactly the membrane-energy regularizer minimized in Eq. (2). Reporting this optimized loss as an evaluation result is self-confirming for that distortion claim, though the comparison against baselines that optimize different objectives is still meaningful as a trade-off statement. The GPTEval3D evaluator is a GPT-4V model while the method uses GPT-4o; this shared model family is a validity concern, but it is not a definitional or equation-level reduction, so I do not count it as pith circularity under the hard rules. The paper's own limitations - low-frequency-only modification, failure on all-smooth shapes such as a sphere, and possible self-intersections - bound the 'arbitrary mesh' claim but are explicit scope restrictions rather than circular reasoning. There are no load-bearing self-citations: the cone-singularity references [15,51] and deformation references [4,38] are from other research groups. Overall the central contribution survives with independent support; the circularity score is moderate only because one advertised quantitative result (membrane-energy distortion) is a restatement of the objective being optimized.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The central claim relies on two domain assumptions about VLM behavior and geometry: that high-curvature cone singularities correspond to useful handles, and that six renderings plus a 2D segmenter produce an accurate 3D part label. Three hand-set numerical knobs (lambda, distortion bound, view count) affect output quality and are not derived from first principles.

free parameters (3)
  • Membrane energy weight lambda = 0.01
    Chosen by hand in Eq. (2) to balance handle position fidelity against distortion; no sensitivity analysis is reported.
  • Distortion bound for cone singularity detection = 0.22 average, adaptive
    Controls the number of candidate handles; the paper adaptively lowers it when no singularities appear on a segmented region, making it a per-mesh tuning choice.
  • Number of camera views = 6
    Six axis-aligned views are used; the paper states more views improve results at higher VLM cost, so this is a quality-versus-cost knob.
assumptions (3)
  • domain assumption Cone singularities are reliable semantic keypoint candidates for the VLM handle selection
    Sec. IV-C assumes meaningful handles sit at high Gauss curvature points and uses cone singularity detection to build the handle superset; Sec. VI concedes smooth shapes like spheres defeat this.
  • domain assumption 2D masks from Grounding-DINO/SAM can be lifted to a correct 3D mesh segmentation via graph cut with six axis-aligned views
    Sec. IV-B uses projected masks as unary costs in Eq. (1) and relies on the optimizer to label invisible faces; incorrect masks or missed views produce a wrong deformable region.
  • standard math Bounded biharmonic weights produce valid linear deformation spaces from a sparse handle set
    Sec. IV-D inherits the BBW framework from [4]; all vertex positions are treated as linear functions of handles, which is a standard result in geometry processing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Handle-based Mesh Deformation Guided By Vision Language Model." pith.science (2026). https://pith.science/paper/ZQFSTMZK

@misc{pith2026250604562,
  author       = {Pith},
  title        = {Pith review of: Handle-based Mesh Deformation Guided By Vision Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZQFSTMZK}},
  note         = {Machine review of arXiv:2506.04562}
}
read the original abstract

Mesh deformation is a fundamental tool in 3D content manipulation. Despite extensive prior research, existing approaches often suffer from low output quality, require significant manual tuning, or depend on data-intensive training. To address these limitations, we introduce a training-free, handle-based mesh deformation method. % Our core idea is to leverage a Vision-Language Model (VLM) to interpret and manipulate a handle-based interface through prompt engineering. We begin by applying cone singularity detection to identify a sparse set of potential handles. The VLM is then prompted to select both the deformable sub-parts of the mesh and the handles that best align with user instructions. Subsequently, we query the desired deformed positions of the selected handles in screen space. To reduce uncertainty inherent in VLM predictions, we aggregate the results from multiple camera views using a novel multi-view voting scheme. % Across a suite of benchmarks, our method produces deformations that align more closely with user intent, as measured by CLIP and GPTEval3D scores, while introducing low distortion -- quantified via membrane energy. In summary, our approach is training-free, highly automated, and consistently delivers high-quality mesh deformations.

Figures

Figures reproduced from arXiv: 2506.04562 by the authors.

Figure 1
Figure 1. A gallery of results generated using our text-driven handle-based mesh deformation, where the deformed meshes (right) are guided by the text prompt. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An illustration of our pipeline working for the example in Figure 1 (a). From left to right: We first extract a series of sub-instructions [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. In the example of Figure 1 (a), we use [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (21 more)
Figure 4
Figure 4. Figure 4: For the two camera views selected during the sub-part identification [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: When guided by two views, the noise is straightly elongated (left). [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: From left to right, we show the original mesh, the deformed mesh using our method, TextDeformer, MagicClay, and MeshUp. Compared with our [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Results of using ARAP deformation and biharmonic deformation. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Our method produces consistent mesh deformations for semantically [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: We show user-guided sub-part identification. While certain regions of [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: We show user-guided deformation magnitude control. Artists may [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Detected cone singularity points are drawn in red dots. Distortion [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Left: Origin; Middle: Camel with a slightly taller hump; Right: [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Example visualization [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: Example visualization. Stretching all chair legs to the front of the chair. A deer leaning forward. A tall barchair. Pull the airplane wings backwards [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]
Figure 17
Figure 17. Figure 17: The horse legs is deformed with artifacts. [PITH_FULL_IMAGE:figures/full_fig_p014_17.png]
Figure 16
Figure 16. Figure 16: Example visualization. D. Baseline Hyperparameters We compare our approach against TextDeformer, Mag￾icClay, and MeshUp to evaluate its effectiveness. For all methods, we use the authors’ released source code with de￾fault hyperparameters, applying minor adjustments o…
Figure 19
Figure 19. Figure 19: The instruction provided to GPT-4o for view selection. [PITH_FULL_IMAGE:figures/full_fig_p015_19.png]
Figure 20
Figure 20. Figure 20: The instruction provided to GPT-4o for selecting handles and directions from camera view. [PITH_FULL_IMAGE:figures/full_fig_p016_20.png]
Figure 21
Figure 21. Figure 21: The instruction provided to GPT-4o for selecting handles and directions from camera view (continued). [PITH_FULL_IMAGE:figures/full_fig_p017_21.png]
Figure 25
Figure 25. Figure 25: Example 4 image [PITH_FULL_IMAGE:figures/full_fig_p017_25.png]
Figure 26
Figure 26. Figure 26: The finetune data example provided to GPT-4o for selecting handles and directions from camera view. [PITH_FULL_IMAGE:figures/full_fig_p018_26.png]
Figure 27
Figure 27. Figure 27: Example 1 image [PITH_FULL_IMAGE:figures/full_fig_p018_27.png]
Figure 29
Figure 29. Figure 29: The instruction for using GPT Evaluation based on GPTEval3D. [PITH_FULL_IMAGE:figures/full_fig_p019_29.png]
Figure 32
Figure 32. Figure 32: Image feed into GPT [PITH_FULL_IMAGE:figures/full_fig_p019_32.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 54 canonical work pages

  1. [1]

    Scalable locally injective mappings,

    M. Rabinovich, R. Poranne, D. Panozzo, and O. Sorkine-Hornung, “Scalable locally injective mappings,” ACM Transactions on Graphics , vol. 36, no. 2, pp. 16:1–16:16, Apr. 2017

    work page 2017

  2. [2]

    Mean value coordinates,

    M. S. Floater, “Mean value coordinates,” Computer aided geometric design, vol. 20, no. 1, pp. 19–27, 2003

    work page 2003

  3. [3]

    Green coordinates,

    Y . Lipman, D. Levin, and D. Cohen-Or, “Green coordinates,” ACM transactions on graphics (TOG) , vol. 27, no. 3, pp. 1–10, 2008

    work page 2008

  4. [4]

    Bounded biharmonic weights for real-time deformation

    A. Jacobson, I. Baran, J. Popovic, and O. Sorkine, “Bounded biharmonic weights for real-time deformation.” ACM Trans. Graph., vol. 30, no. 4, p. 78, 2011

    work page 2011

  5. [5]

    Joint-aware manipulation of deformable models,

    W. Xu, J. Wang, K. Yin, K. Zhou, M. Van De Panne, F. Chen, and B. Guo, “Joint-aware manipulation of deformable models,” ACM Transactions on Graphics (TOG) , vol. 28, no. 3, pp. 1–9, 2009

    work page 2009

  6. [6]

    Efficient learning of mesh-based physical simulation with bi-stride multi-scale graph neural network,

    Y . Cao, M. Chai, M. Li, and C. Jiang, “Efficient learning of mesh-based physical simulation with bi-stride multi-scale graph neural network,” in International conference on machine learning . PMLR, 2023, pp. 3541–3558

    work page 2023

  7. [7]

    Neural cages for detail-preserving 3d deformations,

    W. Yifan, N. Aigerman, V . G. Kim, S. Chaudhuri, and O. Sorkine- Hornung, “Neural cages for detail-preserving 3d deformations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 75–83

    work page 2020

  8. [8]

    Learning skeletal articulations with neural blend shapes,

    P. Li, K. Aberman, R. Hanocka, L. Liu, O. Sorkine-Hornung, and B. Chen, “Learning skeletal articulations with neural blend shapes,” ACM Transactions on Graphics (TOG) , vol. 40, no. 4, pp. 1–15, 2021

    work page 2021

Show all 67 references
  1. [9]

    Rignet: neural rigging for articulated characters,

    Z. Xu, Y . Zhou, E. Kalogerakis, C. Landreth, and K. Singh, “Rignet: neural rigging for articulated characters,” ACM Transactions on Graph- ics (TOG), vol. 39, no. 4, pp. 58–1, 2020

    2020

  2. [10]

    Keypointdeformer: Unsupervised 3d keypoint discovery for shape con- trol,

    T. Jakab, R. Tucker, A. Makadia, J. Wu, N. Snavely, and A. Kanazawa, “Keypointdeformer: Unsupervised 3d keypoint discovery for shape con- trol,” in Computer Vision and Pattern Recognition (CVPR) , 2021

    2021

  3. [11]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  4. [12]

    Clay: A controllable large-scale generative model for creating high-quality 3d assets,

    L. Zhang, Z. Wang, Q. Zhang, Q. Qiu, A. Pang, H. Jiang, W. Yang, L. Xu, and J. Yu, “Clay: A controllable large-scale generative model for creating high-quality 3d assets,” ACM Transactions on Graphics (TOG), vol. 43, no. 4, pp. 1–20, 2024

    2024

  5. [13]

    Textde- former: Geometry manipulation using text guidance,

    W. Gao, N. Aigerman, T. Groueix, V . Kim, and R. Hanocka, “Textde- former: Geometry manipulation using text guidance,” in ACM SIG- GRAPH 2023 Conference Proceedings , 2023, pp. 1–11

    2023

  6. [14]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763. JOURNAL OF...

    2021

  7. [15]

    Efficient cone singu- larity construction for conformal parameterizations,

    M. Li, Q. Fang, Z. Zhang, L. Liu, and X.-M. Fu, “Efficient cone singu- larity construction for conformal parameterizations,” ACM Transactions on Graphics (TOG) , vol. 42, no. 6, pp. 1–13, 2023

    2023

  8. [16]

    Pixel2mesh++: Multi-view 3d mesh generation via deformation,

    C. Wen, Y . Zhang, Z. Li, and Y . Fu, “Pixel2mesh++: Multi-view 3d mesh generation via deformation,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 1042–1051

    2019

  9. [17]

    Phys- ically based deformable models in computer graphics,

    A. Nealen, M. M ¨uller, R. Keiser, E. Boxerman, and M. Carlson, “Phys- ically based deformable models in computer graphics,” in Computer graphics forum, vol. 25, no. 4. Wiley Online Library, 2006, pp. 809– 836

    2006

  10. [18]

    Deformable medical image registration: A survey,

    A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” IEEE transactions on medical imaging , vol. 32, no. 7, pp. 1153–1190, 2013

    2013

  11. [19]

    A survey on cage-based deformation of 3d models,

    D. Str ¨oter, J.-M. Thiery, K. Hormann, J. Chen, Q. Chang, S. Besler, J. S. Mueller-Roemer, T. Boubekeur, A. Stork, and D. W. Fellner, “A survey on cage-based deformation of 3d models,” in Computer Graphics Forum. Wiley Online Library, 2024, p. e15060

    2024

  12. [20]

    Foldingnet: Point cloud auto-encoder via deep grid deformation,

    Y . Yang, C. Feng, Y . Shen, and D. Tian, “Foldingnet: Point cloud auto-encoder via deep grid deformation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 206– 215

    2018

  13. [21]

    Nerfies: Deformable neural radiance fields,

    K. Park, U. Sinha, J. T. Barron, S. Bouaziz, D. B. Goldman, S. M. Seitz, and R. Martin-Brualla, “Nerfies: Deformable neural radiance fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5865–5874

    2021

  14. [22]

    3d geometry-aware deformable gaussian splatting for dynamic view synthesis,

    Z. Lu, X. Guo, L. Hui, T. Chen, M. Yang, X. Tang, F. Zhu, and Y . Dai, “3d geometry-aware deformable gaussian splatting for dynamic view synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 8900–8910

    2024

  15. [23]

    Deforming radiance fields with cages,

    T. Xu and T. Harada, “Deforming radiance fields with cages,” in European Conference on Computer Vision . Springer, 2022, pp. 159– 175

    2022

  16. [24]

    Tuttenet: Injective 3d deformations by composition of 2d mesh deformations,

    B. Sun, T. Groueix, C. Song, Q. Huang, and N. Aigerman, “Tuttenet: Injective 3d deformations by composition of 2d mesh deformations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21 378–21 389

    2024

  17. [25]

    Dreamfusion: Text- to-3d using 2d diffusion,

    B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, “Dreamfusion: Text- to-3d using 2d diffusion,” arXiv preprint arXiv:2209.14988 , 2022

    2022 arXiv

  18. [26]

    Shapewalk: Compositional shape editing through language-guided chains,

    H. Slim and M. Elhoseiny, “Shapewalk: Compositional shape editing through language-guided chains,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 22 574–22 583

    2024

  19. [27]

    ShapeTalk: A language dataset and framework for 3d shape edits and deformations,

    P. Achlioptas, I. Huang, M. Sung, S. Tulyakov, and L. Guibas, “ShapeTalk: A language dataset and framework for 3d shape edits and deformations,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  20. [28]

    Text2mesh: Text-driven neural stylization for meshes,

    O. Michel, R. Bar-On, R. Liu, S. Benaim, and R. Hanocka, “Text2mesh: Text-driven neural stylization for meshes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 13 492–13 502

    2022

  21. [29]

    Self-correcting llm-controlled diffusion models,

    T.-H. Wu, L. Lian, J. E. Gonzalez, B. Li, and T. Darrell, “Self-correcting llm-controlled diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 6327–6336

    2024

  22. [30]

    Blenderalchemy: Editing 3d graphics with vision-language models,

    I. Huang, G. Yang, and L. Guibas, “Blenderalchemy: Editing 3d graphics with vision-language models,” in European Conference on Computer Vision. Springer, 2024, pp. 297–314

    2024

  23. [31]

    Blendergym: Bench- marking foundational model systems for graphics editing,

    Y . Gu, I. Huang, J. Je, G. Yang, and L. Guibas, “Blendergym: Bench- marking foundational model systems for graphics editing,” in Proceed- ings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 18 574–18 583

    2025

  24. [32]

    Chat2layout: Interactive 3d furniture layout with a multimodal llm,

    C. Wang, H. Zhong, M. Chai, M. He, D. Chen, and J. Liao, “Chat2layout: Interactive 3d furniture layout with a multimodal llm,” arXiv preprint arXiv:2407.21333 , 2024

    2024 arXiv

  25. [33]

    Holodeck: Language guided generation of 3d embodied ai environments,

    Y . Yang, F.-Y . Sun, L. Weihs, E. VanderBilt, A. Herrasti, W. Han, J. Wu, N. Haber, R. Krishna, L. Liu et al., “Holodeck: Language guided generation of 3d embodied ai environments,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, p...

    2024

  26. [34]

    Robust inside-outside segmentation using generalized winding numbers,

    A. Jacobson, L. Kavan, and O. Sorkine-Hornung, “Robust inside-outside segmentation using generalized winding numbers,” ACM Transactions on Graphics (TOG) , vol. 32, no. 4, pp. 1–12, 2013

    2013

  27. [35]

    3d-llm: Injecting the 3d world into large language models,

    Y . Hong, H. Zhen, P. Chen, S. Zheng, Y . Du, Z. Chen, and C. Gan, “3d-llm: Injecting the 3d world into large language models,” Advances in Neural Information Processing Systems , vol. 36, pp. 20 482–20 494, 2023

    2023

  28. [36]

    Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent,

    J. Yang, X. Chen, S. Qian, N. Madaan, M. Iyengar, D. F. Fouhey, and J. Chai, “Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 7694–7701

    2024

  29. [37]

    3dmit: 3d multi-modal instruction tuning for scene understanding,

    Z. Li, C. Zhang, X. Wang, R. Ren, Y . Xu, R. Ma, X. Liu, and R. Wei, “3dmit: 3d multi-modal instruction tuning for scene understanding,” in 2024 IEEE International Conference on Multimedia and Expo Work- shops (ICMEW). IEEE, 2024, pp. 1–5

    2024

  30. [38]

    Fem simulation of 3d deformable solids: a practitioner’s guide to theory, discretization and model reduction,

    E. Sifakis and J. Barbic, “Fem simulation of 3d deformable solids: a practitioner’s guide to theory, discretization and model reduction,” in ACM SIGGRAPH 2012 Courses . Association for Computing Machinery, 2012

    2012

  31. [39]

    Quasi-newton methods for real-time simulation of hyperelastic materials,

    T. Liu, S. Bouaziz, and L. Kavan, “Quasi-newton methods for real-time simulation of hyperelastic materials,” ACM Transactions on Graphics (TOG), vol. 36, no. 3, p. 23, 2017

    2017

  32. [40]

    Shapellm: Universal 3d object understanding for embodied interaction,

    Z. Qi, R. Dong, S. Zhang, H. Geng, C. Han, Z. Ge, L. Yi, and K. Ma, “Shapellm: Universal 3d object understanding for embodied interaction,” in European Conference on Computer Vision. Springer, 2025, pp. 214– 238

    2025

  33. [41]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su et al., “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” arXiv preprint arXiv:2303.05499 , 2023

    2023 arXiv

  34. [42]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    2023

  35. [43]

    Randomized cuts for 3D mesh analysis,

    A. Golovinskiy and T. Funkhouser, “Randomized cuts for 3D mesh analysis,” ACM TOG, vol. 27, no. 5, Dec. 2008

    2008

  36. [44]

    Cgal: the computational geometry algorithms library,

    P. Alliez and A. Fabri, “Cgal: the computational geometry algorithms library,” in ACM SIGGRAPH 2016 Courses , 2016, pp. 1–8

    2016

  37. [45]

    Sculpt3d: Multi-view consistent text-to-3d generation with sparse 3d prior,

    C. Chen, X. Yang, F. Yang, C. Feng, Z. Fu, C.-S. Foo, G. Lin, and F. Liu, “Sculpt3d: Multi-view consistent text-to-3d generation with sparse 3d prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 10 228–10 237

    2024

  38. [46]

    Text2tex: Text-driven texture synthesis via diffusion models,

    D. Z. Chen, Y . Siddiqui, H.-Y . Lee, S. Tulyakov, and M. Nießner, “Text2tex: Text-driven texture synthesis via diffusion models,” in Pro- ceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 18 558–18 568

    2023

  39. [47]

    Computing sparse cones with bounded distortion for conformal parameterizations,

    Q. Fang, W. Ouyang, M. Li, L. Liu, and X.-M. Fu, “Computing sparse cones with bounded distortion for conformal parameterizations,” ACM Transactions on Graphics (TOG) , vol. 40, no. 6, pp. 1–9, 2021

    2021

  40. [48]

    Meshup: Multi-target mesh deformation via blended score distillation,

    H. Kim, I. Lang, N. Aigerman, T. Groueix, V . G. Kim, and R. Hanocka, “Meshup: Multi-target mesh deformation via blended score distillation,” arXiv preprint arXiv:2408.14899 , 2024

    2024 arXiv

  41. [49]

    Magicclay: Sculpting meshes with generative neural fields,

    A. Barda, V . G. Kim, N. Aigerman, A. H. Bermano, and T. Groueix, “Magicclay: Sculpting meshes with generative neural fields,” SIG- GRAPH Asia (Conference track) , 2024

    2024

  42. [50]

    Layoutvlm: Differentiable optimization of 3d layout via vision-language models,

    F.-Y . Sun, W. Liu, S. Gu, D. Lim, G. Bhat, F. Tombari, M. Li, N. Haber, and J. Wu, “Layoutvlm: Differentiable optimization of 3d layout via vision-language models,” arXiv preprint arXiv:2412.02193 , 2024

    2024 arXiv

  43. [51]

    Computing sparse integer-constrained cones for conformal parameterizations,

    M. Li, Q. Fang, W. Ouyang, L. Liu, and X.-M. Fu, “Computing sparse integer-constrained cones for conformal parameterizations,” ACM Trans. Graph., vol. 41, no. 4, Jul. 2022

    2022

  44. [52]

    Thingi10k: A dataset of 10,000 3d-printing models,

    Q. Zhou and A. Jacobson, “Thingi10k: A dataset of 10,000 3d-printing models,” arXiv preprint arXiv:1605.04797 , 2016

    2016 arXiv

  45. [53]

    Gpt-4v(ision) is a human-aligned evaluator for text-to-3d generation,

    T. Wu, G. Yang, Z. Li, K. Zhang, Z. Liu, L. Guibas, D. Lin, and G. Wetzstein, “Gpt-4v(ision) is a human-aligned evaluator for text-to-3d generation,” in CVPR, 2024

    2024

  46. [54]

    Fusiondeformer: text-guided mesh deformation using diffusion models,

    H. Xu, Y . Wu, X. Tang, J. Zhang, Y . Zhang, Z. Zhang, C. Li, and X. Jin, “Fusiondeformer: text-guided mesh deformation using diffusion models,” The Visual Computer , vol. 40, no. 7, pp. 4701–4712, 2024

    2024

  47. [55]

    Decor-gan: 3d shape detailization by conditional refinement,

    Z. Chen, V . G. Kim, M. Fisher, N. Aigerman, H. Zhang, and S. Chaud- huri, “Decor-gan: 3d shape detailization by conditional refinement,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 15 740–15 749

    2021

  48. [56]

    Symmetry-aware mesh processing,

    A. Golovinskiy, J. Podolak, and T. Funkhouser, “Symmetry-aware mesh processing,” in Mathematics of Surfaces XIII. Springer, 2009, pp. 170– 188. JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 12 SUPPLEMENTARY MATERIAL In this supplementary material, we provide the...

    2009

  49. [57]

    Focus on how well they correspond to the given text description

    Text prompt and Asset Alignment. Focus on how well they correspond to the given text description. An ideal model should accurately reflect all objects and surroundings mentioned in the text prompt, capturing the corresponding attributes as described. Please first describe each...

  50. [58]

    Look at both the RGB and normal images and imagine a 3D model from the multi-view images

    3D Plausibility. Look at both the RGB and normal images and imagine a 3D model from the multi-view images. Determine which model appears more natural, solid, and plausible. Pay attention to any irregularities, such as abnormal body proportions, duplicated parts, or the presenc...

  51. [59]

    Focus on the local parts of the normal maps

    Low-Level Geometry Details. Focus on the local parts of the normal maps. The geometry should accurately represent the intended shape. Note that meaningless noise is not considered as high-frequency details. Determine which one has a more well-organized and efficient structure,...

  52. [60]

    When evaluating these aspects, focus on one of them at a time

    Considering all the degrees above, which one is better overall? Take a really close look at each of the multi-view images for these two 3D objects before providing your answer. When evaluating these aspects, focus on one of them at a time. Try to make independent decisions bet...

  53. [61]

    Left (object 1) is better

  54. [62]

    Right (object 2) is better

  55. [63]

    IMPORTANT: PLEASE USE THE THIRD OPTION SPARSELY

    Cannot decide. IMPORTANT: PLEASE USE THE THIRD OPTION SPARSELY . Then, in the last row, summarize your final decision by <option for criterion 1> <option for criterion 2> <option for criterion 3> <option for criterion 4> . An example output looks like follows:

  56. [64]

    Text prompt & Asset Alignment: The left one xxxx; The right one xxxx; The left/right one is better or cannot decide

  57. [65]

    The left one xxxx; The right one xxxx; The left/right one is better or cannot decide

    3D Plausibility. The left one xxxx; The right one xxxx; The left/right one is better or cannot decide

  58. [66]

    The left one xxxx; The right one xxxx; The left/right one is better or cannot decide

    Low-Level Geometry Details. The left one xxxx; The right one xxxx; The left/right one is better or cannot decide

  59. [67]

    Final answer: x x x x (e.g., 1 2 3 1 / 3 3 2 1 / 3 2 1 1) Fig

    Overall, xxxx The left/right one is better or cannot decide. Final answer: x x x x (e.g., 1 2 3 1 / 3 3 2 1 / 3 2 1 1) Fig. 30. Render Fig. 31. Surface normal Fig. 32. Image feed into GPT

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.