Pith. sign in

REVIEW 3 major objections 5 minor 81 references

Sharp-It: A Multi-view to Multi-view Diffusion Model for 3D Synthesis and Manipulation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sharp-It is a multi-view diffusion model that sharpens low-quality 3D objects while preserving their coarse shape.

desk verdict Sharp-It is a clean, well-executed multi-view refiner for Shap-E outputs, but its headline numbers are measured on a filtered distribution that skips the very failures it claims to fix. read the letter →

arxiv 2412.02631 v1 pith:AL5DZNP5 submitted 2024-12-03 cs.CV cs.LG

Yiftach Edelstein , Or Patashnik , Dana Cohen-Bar , Lihi Zelnik-Manor This is my paper · ORCID
classification cs.CVcs.LG
keywords 3Dgenerationmulti-viewdiffusionShap-Eeditingtextureenhancementgeometryrefinementconsistencysparse-viewreconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the quality gap between native 3D generative models and multi-view reconstruction pipelines can be closed by a dedicated multi-view-to-multi-view diffusion model. Sharp-It takes six rendered views of a coarse 3D object—for example, one produced by Shap-E—and, guided by a text prompt, enriches the set with fine geometry and texture while keeping the views consistent and the original shape intact. The enhanced multi-view set can then be handed to any existing sparse-view reconstruction method, yielding a high-quality 3D asset in seconds. If this works as described, it would give text-to-3D pipelines both the controllability of native 3D generators and the visual quality of image-diffusion-based approaches, while also enabling editing and controlled generation.

What carries the argument

The central object is the Sharp-It diffusion model itself: a Stable Diffusion/Zero123++-based UNet whose input is expanded to eight channels (four noisy latent channels plus four VAE-encoded channels from the six input views), with text conditioning in cross-attention and cross-view correspondence enforced through the shared self-attention of the multi-view grid. The mechanism that carries the argument is the paired training setup, where Shap-E's encoder deliberately degrades high-quality Objaverse objects to create realistic low-to-high multi-view training pairs; the model learns to fix exactly this kind of degradation while inheriting a plausibly correct coarse geometry from the input renderings.

What would settle it

Take a holdout set of objects generated directly from text by Shap-E (not encoder-degraded renders), run Sharp-It on them, and measure whether the reconstructed assets show the same repaired detail and geometry as the test-set examples; a clear gap between enhancement quality on encoder-degraded renders and on true text-conditioned generations would confirm the distribution mismatch in the training data.

Watch

Extended reading notes

Core claim

The paper introduces Sharp-It, a diffusion model that operates in parallel on a fixed $3 \times 2$ multi-view grid and is conditioned on both the rendered views of a low-quality 3D object and a text prompt. Trained on pairs built by encoding Objaverse objects with Shap-E's encoder and rendering the decoded outputs as degraded inputs, it learns to add geometric detail and appearance features while preserving the coarse structure and cross-view correspondence. Once the enhanced views are produced, a feed-forward sparse-view reconstruction method such as InstantMesh lifts them to a high-quality 3D object. The paper demonstrates that this recipe yields lower FID and higher CLIP/DINO agreement than SDS-based refinement and SDEdit-based multi-view enhancement, and that it supports text-to-3D generation, latent-space editing via Shap-Editor and DDPM inversion, prompt-based appearance editing, and controlled generation from Spice-E structural priors.

Load-bearing premise

The training data assumes that the degradation produced by Shap-E's encoder on clean Objaverse objects looks like the artifacts Shap-E actually generates from text, so the enhancer learns to fix the right kind of damage.

Editorial extensions

If this is right

  • Native 3D generators that output only coarse shapes can be upgraded to production-quality assets in about ten seconds per object, without per-shape optimization.
  • Because the coarse geometry is inherited from the input, the pipeline sidesteps classic multi-view artifacts such as the Janus problem and flat geometries.
  • Latent-space edits made by tools like Shap-Editor and DDPM inversion in Shap-E become high-quality 3D edits once the multi-view output is passed through Sharp-It.
  • Prompt swapping at inference changes appearance while preserving shape, giving users a fast, direct route to texture and material editing.
  • The enhanced multi-view set is reconstruction-agnostic, so improvements in sparse-view reconstruction models can be plugged in without retraining Sharp-It.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension beyond the paper's claims: because Sharp-It's conditioning accepts any six-view render at fixed camera angles, the same enhancement step should transfer to other native 3D generators whose outputs can be rendered this way, provided analogous degraded-to-high pairs can be built.
  • The training distribution hinges on the encoder being representative of text-conditioned generation; if a holdout of true Shap-E text outputs shows Janus faces or broken parts that encoder-degraded renders do not, the enhancer may underperform exactly where it matters.
  • The text prompt's ablative contribution suggests Sharp-It uses the prompt not only to preserve identity but to supply plausible materials and micro-texture; varied-prompt experiments could quantify how much of the gain is prompt-driven versus input-driven.
  • Pairing Sharp-It with a reconstruction method that models lighting (rather than baking it into texture) could enable relighting and material control, a capability the current InstantMesh pipeline does not provide.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Sharp-It, a multi-view-to-multi-view diffusion model that takes a set of renderings of a low-quality 3D object, such as one generated by Shap-E, and produces an enriched, higher-detail multi-view set that can then be lifted to a 3D asset with an existing sparse-view reconstruction method. The model is built on Zero123++/Stable Diffusion, with an 8-channel UNet input that concatenates latent noise with a VAE-encoded multi-view input, plus text conditioning and cross-view self-attention. Training pairs are created by encoding Objaverse objects with Shap-E's encoder, rendering the decoded shapes as degraded inputs, and pairing them with the original high-quality renderings. The paper reports large FID/CLIP/DINO improvements over baselines, ablations for text conditioning and diverse lighting, and applications to text-to-3D generation, editing, appearance editing, and controlled generation.

Significance. If the reported results hold, Sharp-It would be a practically useful bridge between native 3D generative models and high-quality multi-view diffusion models: it preserves the controllability of Shap-E's latent space while adding fine geometric and textural detail, and it runs in about ten seconds per object. The architecture and loss are described clearly, the training-data construction is explicit, and the quantitative gains in Table 1 are large. The method is also evaluated against several plausible baselines, and the applications section demonstrates a coherent pipeline. The main weakness is that the quantitative evaluation is performed almost entirely on the same filtered encoder-degraded distribution used for training, and only on 2D multi-view images rather than on reconstructed 3D geometry; this leaves the central claim about improving actual Shap-E text-to-3D outputs under-supported.

major comments (3)
  1. [§3.2, §4.1, Table 1] The training and test distributions do not establish the claimed behavior on actual Shap-E text-conditioned generations. Training pairs are built by encoding Objaverse meshes with Shap-E's encoder and rendering the decoded shapes, and the dataset construction removes objects whose degraded rendering is 'significantly different' from the original, plus thin and keyword-mismatched objects. The quantitative test set in Table 1 is drawn from this same filtered encoder-degraded distribution, so the reported FID, CLIP, and DINO numbers are measured in a distribution that excludes severe encoder failures and text-conditioned artifacts such as broken geometry and Janus problems. The demonstrations on text-generated Shap-E objects in Figures 4 and 6 are qualitative only. The authors should either construct training pairs from actual Shap-E text-conditioned generations or, at minimum, report quantitative results on a held-out set of such generations, and should specify the filtering threshold used in Section 3.2.
  2. [§4.1, Table 1, §5] All quantitative metrics are computed on 2D multi-view images, not on the reconstructed 3D geometry. FID measures image-quality statistics, and the CLIP/DINO similarities compare enhanced views to ground-truth renderings; none of these validates the statement that the enriched multi-view set 'can be reconstructed into a high-quality 3D object.' The paper's Discussion candidly notes that the reconstruction method bakes lighting into texture, but no geometry-level metric (such as Chamfer distance, volume IoU, or a user study on reconstructed meshes) is reported. Adding reconstruction-based evaluation would directly test the central output of the pipeline and would also clarify whether the lighting baked into the enhanced views hurts reconstruction quality.
  3. [Equation (1), §4.1] The FID reference distribution is the paper's own filtered test set, which shares the same render style, degradation process, and lighting protocol as the training data. Because the model is trained to map exactly this kind of encoder-degraded rendering to clean renderings, the FID improvement over baselines may partly reflect the model's access to the target distribution's render statistics rather than its ability to recover high-frequency geometry and texture. Reporting FID on an external multi-view dataset or on text-generated Shap-E objects would strengthen the claim that the method generalizes beyond its training distribution.
minor comments (5)
  1. [Figures 2 and 7] The prompt text 'Torquoise' should be spelled 'Turquoise'.
  2. [§3.2, References [12], [41], [42]] The dataset description cites Objaverse as [12] while also referring to [41,42] for the source of the objects; since [12] is Objaverse-XL and [41,42] are captioning/view-selection papers, the exact dataset provenance should be clarified.
  3. [Table 2] The row 'W/o text prompt' reports CLIP 0.91, which is higher than the Full method's 0.89; the text states that 'all configurations are comparable aligned with the ground truth object,' but this should be reconciled with the claim that the full model is the best configuration.
  4. [§4.2] The citation 'Zero123++ [60, 72]' appears to cite InstantMesh [72] as part of Zero123++; the reference should be corrected.
  5. [§3.3] A brief description of how the six rendered views are arranged into the 3x2 grid, and how the grid is reverted after denoising, would improve reproducibility of the multi-view reconstruction step.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Sharp-It is an empirical supervised enhancement model trained on paired data and evaluated on a held-out split; the reported limitation is a domain-coverage concern, not a circular reduction.

full rationale

Sharp-It is a supervised multi-view enhancement model, not a derivation. Section 3.2 constructs paired (degraded, high-quality) multi-view sets by encoding Objaverse meshes with Shap-E's encoder and rendering the decoded shapes; Section 3.3 defines a standard diffusion loss on those pairs; Section 4.1 evaluates on a disjoint 6,000-object held-out test split built the same way. No predicted quantity is defined as, or algebraically reduces to, an input, and no parameter is fitted to the reported test metrics. The only near-concern is a domain-coverage gap that the paper itself acknowledges: the quantitative experiments are computed on the filtered encoder-degraded test distribution, while objects generated directly by Shap-E from text prompts are shown only qualitatively ('To demonstrate our method's generalization capability, we also present results on objects generated directly by Shap-E from text prompts', Section 4.1). That is an evaluation-validity limitation -- the training distribution may not match the deployment distribution of Shap-E text-to-3D outputs -- but it is not a circular reduction: the held-out degraded pairs are distinct from training pairs, the enhancement is not defined in terms of the evaluation target, and the central capability is still measured against external baselines. No load-bearing self-citation is present; the backbone models (Shap-E, Zero123++, InstantMesh) are independent prior works. The paper's central claim is an empirical one, and nothing in the construction makes the prediction equivalent to its input by definition.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests almost entirely on learned components and dataset assumptions rather than on hand-derived equations. The main hand-chosen parameters are dataset filtering and rendering choices; the main axioms concern the distribution match between training (encoder-degraded Objaverse) and inference (Shap-E generated) objects, the strength of Zero123++ priors, and the reliability of sparse-view reconstruction. No new physical or ontological entities are introduced.

free parameters (3)
  • Dataset filtering threshold = unspecified
    Section 3.2 removes objects where the degraded Shap-E rendering is 'significantly different' from the original; the threshold is chosen by hand and is not quantified, affecting the training and test distribution.
  • HDR lighting conditions = 3 HDR lighting conditions
    The number and type of lighting conditions used for rendering training pairs is a hand-selected choice; the ablation shows it affects output quality.
  • Training hyperparameters = 500,000 steps; batch size 3; CFG drop 0.1
    Training duration, batch size, and classifier-free-guidance drop probability are chosen by the authors; no sweep is reported, and learning rate is not stated.
assumptions (4)
  • domain assumption Shap-E's encoder applied to high-quality Objaverse objects produces a valid proxy for Shap-E's text-conditioned generator output.
    This is the core of the dataset construction in Section 3.2. If the encoder-degraded distribution differs from the generated distribution, the learned enhancement may not transfer to real Shap-E outputs.
  • domain assumption The pretrained Zero123++ model provides a strong multi-view diffusion prior that can be fine-tuned for enhancement.
    The method initializes from Zero123++ weights and relies on its multi-view consistency and image quality priors (Section 3.3).
  • domain assumption InstantMesh or similar sparse-view reconstruction can faithfully reconstruct a high-quality 3D mesh from the six enhanced views.
    The final 3D asset is produced by reconstruction rather than by the diffusion model itself; the paper acknowledges this as a limitation in Section 5 and does not jointly train the refiner with the reconstruction.
  • domain assumption Objaverse objects and BLIP2 captions form a suitable paired dataset for learning appearance and geometry enhancement.
    The training data and text conditioning rely on the quality and diversity of Objaverse and on automatic caption extraction (Section 3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sharp-It: A Multi-view to Multi-view Diffusion Model for 3D Synthesis and Manipulation." pith.science (2026). https://pith.science/paper/AL5DZNP5

@misc{pith2026241202631,
  author       = {Pith},
  title        = {Pith review of: Sharp-It: A Multi-view to Multi-view Diffusion Model for 3D Synthesis and Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AL5DZNP5}},
  note         = {Machine review of arXiv:2412.02631}
}
read the original abstract

Advancements in text-to-image diffusion models have led to significant progress in fast 3D content creation. One common approach is to generate a set of multi-view images of an object, and then reconstruct it into a 3D model. However, this approach bypasses the use of a native 3D representation of the object and is hence prone to geometric artifacts and limited in controllability and manipulation capabilities. An alternative approach involves native 3D generative models that directly produce 3D representations. These models, however, are typically limited in their resolution, resulting in lower quality 3D objects. In this work, we bridge the quality gap between methods that directly generate 3D representations and ones that reconstruct 3D objects from multi-view images. We introduce a multi-view to multi-view diffusion model called Sharp-It, which takes a 3D consistent set of multi-view images rendered from a low-quality object and enriches its geometric details and texture. The diffusion model operates on the multi-view set in parallel, in the sense that it shares features across the generated views. A high-quality 3D model can then be reconstructed from the enriched multi-view set. By leveraging the advantages of both 2D and 3D approaches, our method offers an efficient and controllable method for high-quality 3D content creation. We demonstrate that Sharp-It enables various 3D applications, such as fast synthesis, editing, and controlled generation, while attaining high-quality assets.

Figures

Figures reproduced from arXiv: 2412.02631 by the authors.

Figure 1
Figure 1. Sharp-It is a multi-view to multi-view model that enhances low-quality 3D shapes. It corrects fine-grained geometry details and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of 3D generation pipeline with Sharp-It. First, a 3D object is generated with Shap-E. Then, we render six views of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Self-attention maps for a query point (red) on the car’s [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Comparison of Sharp-It with other methods for 3D object enhancement (GaussianDreamer and MVEdit), and multi-view en [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative ablation study. The first column shows the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Text-to-MultiView qualitative results. Our method generates a multi-view set by creating a 3D object with Shap-E and refining it [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Our method enhances edits performed in Shap-E’s space [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 9
Figure 9. Figure 9: Sharp-It allows to edit the appearance of a Shap-E object [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

81 extracted references · 38 canonical work pages

  1. [1]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18208–18218, 2022. 3

    2022

  2. [2]

    Spatext: Spatio-textual representation for con- trollable image generation

    Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, and Xi Yin. Spatext: Spatio-textual representation for con- trollable image generation. In CVPR, 2023. 3

    2023

  3. [3]

    Mitra, and Peter Wonka

    Shariq Farooq Bhat, Niloy J. Mitra, and Peter Wonka. Loosecontrol: Lifting controlnet for generalized depth con- ditioning, 2023. 3

    work page 2023

  4. [4]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. In CVPR, 2023. 3, 4

    work page 2023

  5. [5]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV) , pages 22560–22570,

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 6

    2021

  7. [7]

    Single-stage diffusion nerf: A unified approach to 3d generation and reconstruction, 2023

    Hansheng Chen, Jiatao Gu, Anpei Chen, Wei Tian, Zhuowen Tu, Lingjie Liu, and Hao Su. Single-stage diffusion nerf: A unified approach to 3d generation and reconstruction, 2023. 2

    work page 2023

  8. [8]

    Generic 3d diffusion adapter using controlled multi-view editing, 2024

    Hansheng Chen, Ruoxi Shi, Yulin Liu, Bokui Shen, Ji- ayuan Gu, Gordon Wetzstein, Hao Su, and Leonidas Guibas. Generic 3d diffusion adapter using controlled multi-view editing, 2024. 3, 5

    work page 2024

Show all 81 references
  1. [9]

    Shap-editor: Instruction-guided latent 3d editing in seconds

    Minghao Chen, Junyu Xie, Iro Laina, and Andrea Vedaldi. Shap-editor: Instruction-guided latent 3d editing in seconds. arXiv preprint arXiv:2312.09246, 2023. 2, 3, 7, 8

    2023 arXiv

  2. [10]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22246–22256, 2023. 2

    2023

  3. [11]

    Slicedit: Zero- shot video editing with text-to-image diffusion models using spatio-temporal slices, 2024

    Nathaniel Cohen, Vladimir Kulikov, Matan Kleiner, Inbar Huberman-Spiegelglas, and Tomer Michaeli. Slicedit: Zero- shot video editing with text-to-image diffusion models using spatio-temporal slices, 2024. 3

    2024

  4. [12]

    Objaverse-xl: A universe of 10m+ 3d objects, 2023

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, Eli VanderBilt, Anirud- dha Kembhavi, Carl V ondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Obja...

    2023

  5. [13]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsanit, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVP...

    2023

  6. [14]

    Turboedit: Text-based image editing using few-step diffusion models, 2024

    Gilad Deutch, Rinon Gal, Daniel Garibi, Or Patashnik, and Daniel Cohen-Or. Turboedit: Text-based image editing using few-step diffusion models, 2024. 3

    2024

  7. [15]

    Renoise: Real image inversion through iterative noising, 2024

    Daniel Garibi, Or Patashnik, Andrey V oynov, Hadar Averbuch-Elor, and Daniel Cohen-Or. Renoise: Real image inversion through iterative noising, 2024. 3

    2024

  8. [16]

    3dgen: Triplane latent diffusion for textured mesh generation, 2023

    Anchit Gupta, Wenhan Xiong, Yixin Nie, Ian Jones, and Bar- las O˘guz. 3dgen: Triplane latent diffusion for textured mesh generation, 2023. 2

    2023

  9. [17]

    Efros, Aleksander Holynski, and Angjoo Kanazawa

    Ayaan Haque, Matthew Tancik, Alexei A. Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2nerf: Edit- ing 3d scenes with instructions, 2023. 3

    2023

  10. [18]

    Prompt-to-prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. 2022. 3

    2022

  11. [19]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium, 2017. 6

    2017

  12. [20]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. arXiv preprint arxiv:2006.11239,

    2006 arXiv

  13. [21]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2

    2023 arXiv

  14. [22]

    Lrm: Large reconstruction model for single image to 3d, 2024

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d, 2024. 3

    2024

  15. [23]

    Neural wavelet-domain diffusion for 3d shape gen- eration, inversion, and manipulation, 2023

    Jingyu Hu, Ka-Hei Hui, Zhengzhe Liu, Ruihui Li, and Chi- Wing Fu. Neural wavelet-domain diffusion for 3d shape gen- eration, inversion, and manipulation, 2023. 3

    2023

  16. [24]

    An edit friendly DDPM noise space: Inversion and manipulations

    Inbar Huberman-Spiegelglas, Vladimir Kulikov, and Tomer Michaeli. An edit friendly DDPM noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12469– 12478, 2024. 3, 8

    2024

  17. [25]

    Neural wavelet-domain diffusion for 3d shape generation, 2022

    Ka-Hei Hui, Ruihui Li, Jingyu Hu, and Chi-Wing Fu. Neural wavelet-domain diffusion for 3d shape generation, 2022. 2

    2022

  18. [26]

    Lvsm: A large view synthesis model with minimal 3d inductive bias, 2024

    Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias, 2024. 2, 3, 4, 8

    2024

  19. [27]

    Shap-e: Generating condi- tional 3d implicit functions, 2023

    Heewoo Jun and Alex Nichol. Shap-e: Generating condi- tional 3d implicit functions, 2023. 2, 3, 8

    2023

  20. [28]

    Noise-free score distillation, 2024

    Oren Katzir, Or Patashnik, Daniel Cohen-Or, and Dani Lischinski. Noise-free score distillation, 2024. 2 9

    2024

  21. [29]

    Posterior distil- lation sampling, 2024

    Juil Koo, Chanho Park, and Minhyuk Sung. Posterior distil- lation sampling, 2024. 3

    2024

  22. [30]

    BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML,

  23. [31]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. https://arxiv.org/abs/2311.06214, 2023. 2, 4

    2023 arXiv

  24. [32]

    Diffusion- sdf: Text-to-shape via voxelized diffusion, 2023

    Muheng Li, Yueqi Duan, Jie Zhou, and Jiwen Lu. Diffusion- sdf: Text-to-shape via voxelized diffusion, 2023. 2, 3

    2023

  25. [33]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. 2023. 3

    2023

  26. [34]

    Magic3d: High-resolution text-to-3d content creation, 2023

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation, 2023. 2

    2023

  27. [35]

    One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion

    Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Ji- ayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. arXiv preprint arXiv:2311.07885, 2023. 2

    2023 arXiv

  28. [36]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. Advances in Neural Information Processing Systems , 36, 2024. 2

    2024

  29. [37]

    Zero-1-to-3: Zero-shot one image to 3d object, 2023

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object, 2023. 2

    2023

  30. [38]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age, 2024

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age, 2024. 2

    2024

  31. [39]

    Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu

    Zhen Liu, Yao Feng, Michael J. Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. Meshdif- fusion: Score-based generative 3d mesh modeling, 2023. 3

    2023

  32. [40]

    Wonder3d: Single image to 3d using cross-domain diffusion,

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, and Wenping Wang. Wonder3d: Single image to 3d using cross-domain diffusion,

  33. [41]

    Scalable 3d captioning with pretrained models

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin John- son. Scalable 3d captioning with pretrained models. arXiv preprint arXiv:2306.07279, 2023. 3

    2023 arXiv

  34. [42]

    View selec- tion for 3d captioning via diffusion ranking

    Tiange Luo, Justin Johnson, and Honglak Lee. View selec- tion for 3d captioning via diffusion ranking. arXiv preprint arXiv:2404.07984, 2024. 3

    2024

  35. [43]

    Controllable mesh generation through sparse latent point diffusion models, 2023

    Zhaoyang Lyu, Jinyi Wang, Yuwei An, Ya Zhang, Dahua Lin, and Bo Dai. Controllable mesh generation through sparse latent point diffusion models, 2023. 2

    2023

  36. [44]

    SDEdit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equa- tions. In International Conference on Learning Representa- tions, 2022. 5

    2022

  37. [45]

    Latent-nerf for shape-guided generation of 3d shapes and textures

    Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. arXiv preprint arXiv:2211.07600,

    arXiv

  38. [46]

    Null-text inversion for editing real im- ages using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real im- ages using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038–6047, 2023. 3

    2023

  39. [47]

    Point-e: A system for generating 3d point clouds from complex prompts, 2022

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts, 2022. 2

    2022

  40. [48]

    On aliased resizing and surprising subtleties in gan evaluation

    Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In CVPR, 2022. 6

    2022

  41. [49]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–11, 2023. 3

    2023

  42. [50]

    Localizing object-level shape variations with text-to-image diffusion models

    Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch- Elor, and Daniel Cohen-Or. Localizing object-level shape variations with text-to-image diffusion models. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision (ICCV), 2023. 3

    2023

  43. [51]

    Consolidating attention features for multi-view image editing, 2024

    Or Patashnik, Rinon Gal, Daniel Cohen-Or, Jun-Yan Zhu, and Fernando De la Torre. Consolidating attention features for multi-view image editing, 2024. 3

    2024

  44. [52]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023. 5

    2023

  45. [53]

    Barron, and Ben Milden- hall

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv,

  46. [54]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 6

    2021

  47. [55]

    High-resolution image syn- thesis with latent diffusion models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2022. 2, 3, 4

    2022

  48. [56]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion mod...

    2022

  49. [57]

    Neural point cloud diffusion for disentangled 3d shape and appearance generation, 2024

    Philipp Schr ¨oppel, Christopher Wewer, Jan Eric Lenssen, Eddy Ilg, and Thomas Brox. Neural point cloud diffusion for disentangled 3d shape and appearance generation, 2024. 2 10

    2024

  50. [58]

    V ox-e: Text-guided voxel editing of 3d ob- jects

    Etai Sella, Gal Fiebelman, Peter Hedman, and Hadar Averbuch-Elor. V ox-e: Text-guided voxel editing of 3d ob- jects. arXiv preprint arXiv:2303.12048, 2023. 3

    2023 arXiv

  51. [59]

    Spice-e : Structural priors in 3d diffusion using cross- entity attention, 2024

    Etai Sella, Gal Fiebelman, Noam Atia, and Hadar Averbuch- Elor. Spice-e : Structural priors in 3d diffusion using cross- entity attention, 2024. 2, 3, 5, 8

    2024

  52. [60]

    Zero123++: a single image to consistent multi-view dif- fusion base model, 2023

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model, 2023. 2, 4, 5, 6, 7

    2023

  53. [61]

    Zero123++: a single image to consistent multi-view dif- fusion base model, 2023

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model, 2023. 2, 3

    2023

  54. [62]

    MVDream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3d gen- eration. In The Twelfth International Conference on Learn- ing Representations, 2024. 2, 4, 5, 7

    2024

  55. [63]

    Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, and Gordon Wetzstein

    J. Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, and Gordon Wetzstein. 3d neural field generation using triplane diffusion, 2022. 2

    2022

  56. [64]

    Viewset diffusion: (0-)image-conditioned 3d gener- ative models from 2d data

    Stanislaw Szymanowicz, Christian Rupprecht, and Andrea Vedaldi. Viewset diffusion: (0-)image-conditioned 3d gener- ative models from 2d data. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 8863–8873, 2023. 2

    2023

  57. [65]

    Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation, 2024

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation, 2024. 2

    2024

  58. [66]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1921–1930, 2023. 3

    1921

  59. [67]

    Yeh, and Greg Shakhnarovich

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lift- ing pretrained 2d diffusion models for 3d generation. arXiv preprint arXiv:2212.00774, 2022. 2

    2022 arXiv

  60. [68]

    Yeh, and Greg Shakhnarovich

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation, 2022. 2

    2022

  61. [69]

    Imagedream: Image-prompt multi-view diffusion for 3d generation

    Peng Wang and Yichun Shi. Imagedream: Image-prompt multi-view diffusion for 3d generation. arXiv preprint arXiv:2312.02201, 2023. 2, 4, 7

    2023 arXiv

  62. [70]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. arXiv preprint arXiv:2305.16213, 2023. 2

    2023 arXiv

  63. [71]

    Turboedit: Instant text-based image editing

    Zongze Wu, Nicholas Kolkin, Jonathan Brandt, Richard Zhang, and Eli Shechtman. Turboedit: Instant text-based image editing. ECCV, 2024. 3

    2024

  64. [72]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,

    arXiv

  65. [73]

    Paint by example: Exemplar-based image editing with diffusion mod- els

    Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion mod- els. arXiv preprint arXiv:2211.13227, 2022. 4

    2022 arXiv

  66. [74]

    Mosaic-sdf for 3d generative models, 2024

    Lior Yariv, Omri Puny, Natalia Neverova, Oran Gafni, and Yaron Lipman. Mosaic-sdf for 3d generative models, 2024. 2

    2024

  67. [75]

    Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models, 2024

    Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models, 2024. 2, 5

    2024

  68. [76]

    Lion: Latent point diffusion models for 3d shape generation

    xiaohui zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation. In Advances in Neural Information Processing Systems , pages 10021– 10039. Curran Associates, Inc., 2022. 2

    2022

  69. [77]

    Gaussiancube: A structured and explicit radiance represen- tation for 3d generative modeling, 2024

    Bowen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, and Baining Guo. Gaussiancube: A structured and explicit radiance represen- tation for 3d generative modeling, 2024. 2

    2024

  70. [78]

    Geolrm: Geometry-aware large re- construction model for high-quality 3d gaussian generation

    Chubin Zhang, Hongliang Song, Yi Wei, Yu Chen, Jiwen Lu, and Yansong Tang. Geolrm: Geometry-aware large re- construction model for high-quality 3d gaussian generation. arXiv preprint arXiv:2406.15333, 2024. 4

    2024 arXiv

  71. [79]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 3

    2023

  72. [80]

    3d shape generation and completion through point-voxel diffusion, 2021

    Linqi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through point-voxel diffusion, 2021. 2

    2021

  73. [81]

    Gtr: Improving large 3d reconstruction models through geometry and texture refinement, 2024

    Peiye Zhuang, Songfang Han, Chaoyang Wang, Aliak- sandr Siarohin, Jiaxu Zou, Michael Vasilkovsky, Vladislav Shakhrai, Sergey Korolev, Sergey Tulyakov, and Hsin-Ying Lee. Gtr: Improving large 3d reconstruction models through geometry and texture refinement, 2024. 2, 3, 4, 8 11

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.