REVIEW 4 major objections 5 minor 62 references
Instructive3D: Editing Large Reconstruction Models with Text Instructions
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Instructive3D shows that a frozen large reconstruction model can follow text instructions to edit 3D objects by running a latent diffusion process in triplane space.
desk verdict A compute-efficient triplane-latent editing adapter for frozen LRMs, with a genuinely novel combination but an evaluation that is circular with its own training-data pipeline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the triplane latent space of Real3D, a set of three 64x64x40 feature planes that jointly encode a 3D object's shape and appearance. The Tri-VAE compresses each plane independently to a 16x16x4 latent (about 68M parameters per plane encoder), and the LTriD diffusion UNet (about 870M parameters) denoises the stacked latents conditioned on a CLIP text embedding, with the original latent concatenated to the noisy latent to preserve the object's structure. Two training stages matter: Tri-VAE is trained first on 12,584 rendered objects, then LTriD is trained with null prompts for identity preservation and with InstructPix2Pix-generated edit pairs for prompt-conditioned edits. The whole scheme transfers edits into the triplane, so the frozen Real3D decoder renders them consistently from any viewpoint.
What would settle it
Render Instructive3D outputs for the same object and prompt (e.g., "add a flame design to the shoe") from all eight cube-corner viewpoints used in the paper's evaluation: if the flame design appears only in some views or the object's silhouette changes between views, the geometrically consistent triplane-editing claim is refuted. A sharper test compares the output mesh against a manually edited 3D ground truth rather than against a 2D-edited rendering.
Extended reading notes
Core claim
The paper's central claim is that fine-grained, text-guided editing can live inside a Large Reconstruction Model's latent space rather than in a post-processing mesh-editing stage. Concretely, the authors add a Triplane Variational Autoencoder and a latent triplane diffusion model (LTriD) on top of the frozen Real3D LRM. During inference, an input image is encoded to a triplane, the adapter denoises a latent version of that triplane conditioned on a CLIP text embedding of the edit prompt, and the frozen decoder turns the edited triplane into a 3D object. Against baselines that first generate a mesh with Real3D and then edit with Text2Mesh, Paint3D, or TEXTure, the paper reports that Instructive3D gives lower LPIPS, FID, and KID and higher PSNR, SSIM, and CLIP scores, with edits that hold across viewpoints because the modification happens in the shared triplane representation.
Load-bearing premise
The paper's edited 3D "ground truth" is actually a triplane computed from a 2D image edit, so the whole method stands on whether InstructPix2Pix edits are faithful 3D-consistent changes and whether the frozen LRM encodes them without loss.
Editorial extensions
If this is right
- Users can edit generated 3D objects with natural-language prompts, changing color, texture, and surface patterns while preserving the underlying geometry.
- Training only requires 2D edited images plus a frozen LRM encoder, so the expensive step of generating precisely edited 3D object pairs is avoided.
- The same frozen LRM decoder turns edited triplanes into view-consistent meshes, so a text-driven edit persists across different viewpoints.
- The adapter is trained on about 14.5K objects, making the approach compute- and data-efficient compared with fine-tuning a full LRM.
- Because the edit is applied inside the triplane representation, the method does not require separate mesh-editing tools or per-instance optimization at inference time.
Reading between the lines
- Because the adapter only touches the triplane space, the same trained weights could in principle be plugged into any LRM that produces Real3D-style triplanes, such as TripoSR or other triplane-based reconstructions, without re-training on their own decoders.
- The method inherits the edit distribution of InstructPix2Pix; prompts or edit types that InstructPix2Pix cannot render in 2D, such as precise geometric deformation, will not transfer to 3D. A testable extension is augmenting the paired dataset with procedural 3D material edits to broaden coverage.
- Null-prompt identity training hints that the adapter could double as a geometry-preserving denoiser, cleaning reconstruction artifacts from an LRM's triplane even without any edit instruction.
- The paper's consistency claim is bounded by Real3D's decoder; a direct comparison against manually edited 3D ground truths would separate triplane-edit fidelity from LRM reconstruction fidelity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Instructive3D, an adapter for a frozen large reconstruction model (Real3D) that performs text-prompted edits in the triplane latent space. A per-plane 2D VAE compresses triplane features, and a conditional latent diffusion model is trained on pairs obtained by rendering Objaverse objects, editing the renders with InstructPix2Pix, and encoding both original and edited images with Real3D. At inference, an input image and edit prompt are mapped through the Real3D encoder, the diffusion adapter, and the Real3D decoder. The experiments compare against Text2Mesh, Paint3D, and TEXTure on Objaverse LVIS using LPIPS, PSNR, SSIM, FID, KID, and CLIP scores, together with an ablation of per-plane 2D VAEs versus a 3D VAE.
Significance. If the evaluation were convincing, the paper would offer a practical and compute-efficient recipe for adding text-instruction editing to existing LRMs without collecting edited 3D object pairs. The architectural choice of freezing the LRM and training only a triplane adapter is sensible, and the per-plane 2D VAE ablation is a useful design data point. The qualitative figures show plausible edits across several object categories. However, the central quantitative claim is undermined by the evaluation protocol, which uses the same InstructPix2Pix-plus-Real3D pipeline to generate both training targets and test ground truth, so the numerical gains largely measure self-reconstruction of the training distribution rather than demonstrated generalization to genuine 3D instruction-following.
major comments (4)
- [§3.2 and §4.2, Table 1] The training targets x_edit in Section 3.2 are triplanes obtained by passing InstructPix2Pix-edited rendered images through Real3D, and the evaluation ground truth in Section 4.2 is described as 'the after-edit object generated by Real3D,' which is the same construction. Consequently, LPIPS, PSNR, SSIM, FID, and KID measure how well the adapter reproduces the exact pipeline that generated its training targets, not how well it generalizes to unseen edits or to genuinely novel 3D manipulations. The evaluation set is also sampled from the same Objaverse LVIS distribution used for training. The authors should add an evaluation that does not depend on this circular target: for example, a human perceptual study on edit fidelity and geometry preservation, held-out object categories and prompts, novel objects outside the LVIS distribution, and a direct baseline that applies InstructPix2Pix to the input render and reconstructs with Real3D. Without such controls, the numbers in Table 1 cannot support the abstract's claim of 'qualitatively superior 3D objects.'
- [Abstract and §5, Table 1] The claim that edits are 'geometrically consistent' or that the method 'preserves the geometry' is asserted rather than measured. No geometric metric—such as Chamfer distance, mesh IoU, normal consistency, or Hausdorff distance—appears in Table 1 or anywhere in the experiments; all reported metrics are computed on rendered images, which conflate appearance changes with geometric changes. The paper should report a direct geometric comparison between the input and edited meshes or triplanes, and ideally compare this quantity against the baselines under the same mesh-extraction protocol, to support the geometry-preservation claim.
- [§4.1 and §4.2, Table 1] The comparison with Text2Mesh, Paint3D, and TEXTure is asymmetric in a way that inflates the reported gap. Those baselines consume a mesh produced by Real3D, whereas Instructive3D's outputs are decoded by the same frozen Real3D that produced the ground-truth objects, so the LPIPS/PSNR/SSIM metrics favor the proposed method by construction. Additionally, the baselines are not trained on the synthetic triplane distribution and may fail for reasons unrelated to edit quality. The authors should add an image-space control baseline (e.g., InstructPix2Pix on the input render followed by Real3D reconstruction) and standardize the evaluation so that all methods are scored on the same rendered views and the same mesh-extraction pipeline.
- [§3.2, §3.6, and §4.3] The data-curation and training protocol is underspecified in ways that matter for generalization. The manual filtering of InstructPix2Pix pairs is not quantified (no number of generated versus retained pairs, no inter-annotator agreement, no criteria), no statistics on prompt types or per-prompt counts are given, and the split between training and evaluation objects is not described beyond '45 objects' for evaluation. The identity-preservation training with Didentity is also never evaluated quantitatively: there is no 'no-edit' test measuring how much the output changes when the prompt is empty or a period. Without these numbers, the reader cannot assess how much of the reported performance is due to filtering, prompt distribution, or identity shortcutting. The paper should report these statistics and include an identity-preservation experiment.
minor comments (5)
- [§4.2, Table 1 discussion] The text says the method 'consistently outperforms prior works across all key metrics,' but in Table 1 the CLIP score for Instructive3D (21.94) is slightly lower than TEXTure (21.98); the claim should be corrected to 'all metrics except CLIP' or similar.
- [§3.3, Eq. (2)] The VAE loss in Eq. (2) writes (x_k − x̃_k)^2 without an explicit sum over k or over spatial/channel dimensions; the notation should be clarified, for example by summing over the three planes and over all elements of each plane.
- [Figure 3 caption] The caption refers to 'The third column presents the meshes produced by the UNet model,' but the figure shows rows rather than columns; this should be corrected.
- [Throughout] There are several typographical errors: 'concatentate' (§3.3), 'boarder' (§1), 'randiance' (§2.1), 'turqoise' (Figure 17 caption), and inconsistent spacing in 'V AE.' A careful proofreading pass is needed.
- [§3.4, Eq. (4)] The conditioning mechanism in Eq. (4) is incompletely specified: it is not stated how the null prompt '.' is embedded, whether classifier-free guidance is used at inference, or how the concatenated latent [z_t, z] interacts with the cross-attention text conditioning. This information is needed to reproduce the method.
Circularity Check
Quantitative evaluation is circular: the ground-truth "after-edit object generated by Real3D" is the same InstructPix2Pix-to-Real3D triplane pipeline used to construct training targets, so the reported gains measure self-reconstruction rather than independent 3D editing quality.
-
fitted input called prediction
[Sec. 3.2 (Dataset for Conditional Diffusion Model), Eq. (4), Sec. 4.2 (Quantitative Results), Table 1]
"We utilize InstructPix2Pix [2], a conditional image-to-image diffusion model, to generate the edited images in this dataset. ... We obtain the triplane features of the original and edited images, x and xedit, respectively, by passing I and Iedit through the pre-trained LRM model M to create a paired triplane dataset with edit prompts ... We calculate the metrics by using the after-edit object generated by Real3D and compare it with the objects generated by our method and other baseline methods."
The diffusion loss in Eq. (4) trains the adapter to produce the edited triplane xedit, where xedit is Real3D's triplane encoding of an InstructPix2Pix-edited 2D render. The quantitative ground truth in Sec. 4.2 is the Real3D render of exactly that same xedit. Therefore PSNR, SSIM, LPIPS, FID, KID, and CLIP scores in Table 1 measure how well Instructive3D reproduces its own training targets. The baselines are evaluated against this same synthetic target distribution without having been trained on it, so the reported metric gaps do not independently establish superior instruction-following 3D editing; they largely establish self-consistency with the data-generation pipeline.
full rationale
The central circularity is confined to the evaluation protocol. The model is legitimately trained as a mapping from a Real3D triplane of an input image to a Real3D triplane of an InstructPix2Pix-edited image, and the diffusion objective in Eq. (4) is a standard denoising loss. No self-citation chain or imported uniqueness theorem is load-bearing here: Real3D, InstructPix2Pix, and CLIP are external models, and the adapter itself is a new learned component. However, the paper's quantitative claim of "qualitatively superior 3D objects with the properties specified by the edit prompts" rests on comparing outputs against ground truth that is generated by the same InstructPix2Pix-plus-Real3D pipeline used to make the training pairs. Consequently, low LPIPS, high PSNR/SSIM, and reduced FID/KID largely verify that the model can regenerate its own synthetic target distribution. This is a partial circularity of the fitted-input-called-prediction type, not a complete collapse of the method: the adapter still learns a real conditional mapping in triplane space, and the qualitative figures provide some independent visual evidence. The geometry-preservation claim is asserted from operating in triplane latent space but is not supported by any geometric metric, which is a correctness and evaluation gap rather than a circular-reasoning step. Overall, the evaluation is circular with respect to training-data construction, warranting a score of 6.
Assumptions & free parameters
free parameters (3)
- Per-plane triplane normalization bounds xmin_k and xmax_k =
Computed from the triplane feature dataset Dbase
- KL weight lambda_KL =
Not reported
- Latent channel count C2 and latent resolution H2 x W2 =
C2=12, H2=W2=16
assumptions (4)
- domain assumption Real3D is a frozen, reliable triplane encoder-decoder whose latent space is semantically editable.
- ad hoc to paper InstructPix2Pix-edited images, when re-encoded by Real3D, provide valid ground-truth triplanes x_edit for the desired text edit.
- domain assumption The Tri-VAE latent space is smooth enough for a latent diffusion model to denoise into valid triplane latents.
- domain assumption CLIP text embeddings provide sufficient semantic conditioning for fine-grained material and color edits.
Cite this review
Pith. "Pith review of Instructive3D: Editing Large Reconstruction Models with Text Instructions." pith.science (2026). https://pith.science/paper/6VLUOFEQ
@misc{pith2026250104374,
author = {Pith},
title = {Pith review of: Instructive3D: Editing Large Reconstruction Models with Text Instructions},
year = {2026},
howpublished = {\url{https://pith.science/paper/6VLUOFEQ}},
note = {Machine review of arXiv:2501.04374}
}
read the original abstract
Transformer based methods have enabled users to create, modify, and comprehend text and image data. Recently proposed Large Reconstruction Models (LRMs) further extend this by providing the ability to generate high-quality 3D models with the help of a single object image. These models, however, lack the ability to manipulate or edit the finer details, such as adding standard design patterns or changing the color and reflectance of the generated objects, thus lacking fine-grained control that may be very helpful in domains such as augmented reality, animation and gaming. Naively training LRMs for this purpose would require generating precisely edited images and 3D object pairs, which is computationally expensive. In this paper, we propose Instructive3D, a novel LRM based model that integrates generation and fine-grained editing, through user text prompts, of 3D objects into a single model. We accomplish this by adding an adapter that performs a diffusion process conditioned on a text prompt specifying edits in the triplane latent space representation of 3D object models. Our method does not require the generation of edited 3D objects. Additionally, Instructive3D allows us to perform geometrically consistent modifications, as the edits done through user-defined text prompts are applied to the triplane latent representation thus enhancing the versatility and precision of 3D objects generated. We compare the objects generated by Instructive3D and a baseline that first generates the 3D object meshes using a standard LRM model and then edits these 3D objects using text prompts when images are provided from the Objaverse LVIS dataset. We find that Instructive3D produces qualitatively superior 3D objects with the properties specified by the edit prompts.
Figures
Figures from the paper (30 more)
Reference graph
Works this paper leans on
-
[1]
Sutherland, Michael Arbel, and Arthur Gretton
Mikołaj Bi ´nkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans, 2021. 6, 7
work page 2021
-
[2]
Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions,
-
[3]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Li...
arXiv 2005
-
[4]
Emerg- ing properties in self-supervised vision transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Herv’e J’egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9630–9640, 2021. 2
work page 2021
-
[5]
pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis
Eric R Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, and Gordon Wetzstein. pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5799–5809, 2021. 3
2021
-
[6]
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from im- age pairs for scalable generalizable 3d reconstruction.ArXiv, abs/2312.12337, 2023. 1
arXiv 2023
-
[7]
Anydoor: Zero-shot object-level im- age customization
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level im- age customization. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 6593–6602, 2024. 5
work page 2024
-
[8]
3d-r2n2: A unified approach for single and multi-view 3d object reconstruction
Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In Com- puter Vision–ECCV 2016: 14th European Conference, Am- sterdam, The Netherlands, October 11-14, 2016, Proceed- ings, Part VIII 14, pages 628–644. Springer, 2016. 3
work page 2016
Show all 62 references
-
[9]
Objaverse: A universe of annotated 3d objects, 2022
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects, 2022. 4
2022
-
[10]
Objaverse: A universe of annotated 3d objects
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...
2023
-
[11]
A point set generation network for 3d object reconstruction from a single image
Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 605–613, 2017. 3
2017
-
[12]
Mesh r-cnn
Georgia Gkioxari, Jitendra Malik, and Justin Johnson. Mesh r-cnn. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 9785–9795, 2019. 3
2019
-
[13]
Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In Neural Information Processing Systems, 2014. 1
2014
-
[14]
Gans trained by a two time-scale update rule converge to a local nash equilib- rium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 6, 7
2017
-
[15]
Denoising dif- fusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3, 5
2020
-
[16]
Lrm: Large reconstruction model for single image to 3d
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. ArXiv, abs/2311.04400, 2023. 2, 3
2023 arXiv
-
[17]
Barron, Pieter Abbeel, and Ben Poole
Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, and Ben Poole. Zero-shot text-guided object gen- eration with dream fields. 2022. 3
2022
-
[18]
Real3d: Scaling up large reconstruction models with real- world images
Hanwen Jiang, Qixing Huang, and Georgios Pavlakos. Real3d: Scaling up large reconstruction models with real- world images. arXiv preprint arXiv:2406.08479, 2024. 4, 5, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39
2024 arXiv
-
[19]
Gal: Geometric adversarial loss for single-view 3d-object recon- struction
Li Jiang, Shaoshuai Shi, Xiaojuan Qi, and Jiaya Jia. Gal: Geometric adversarial loss for single-view 3d-object recon- struction. In Proceedings of the European conference on computer vision (ECCV), pages 802–816, 2018. 3
2018
-
[20]
Efros, and Jitendra Malik
Angjoo Kanazawa, Shubham Tulsiani, Alexei A. Efros, and Jitendra Malik. Learning category-specific mesh reconstruc- tion from image collections. ArXiv, abs/1803.07549, 2018. 1
2018 arXiv
-
[21]
Elucidating the design space of diffusion-based generative models, 2022
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models, 2022. 5
2022
-
[22]
3d gaussian splatting for real-time radiance field rendering
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,
-
[23]
Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 3
2023
-
[24]
Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 3
2022
-
[25]
Zero-1-to- 3: Zero-shot one image to 3d object
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 3 9
2023
-
[26]
Soft ras- terizer: A differentiable renderer for image-based 3d reason- ing
Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. Soft ras- terizer: A differentiable renderer for image-based 3d reason- ing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 7708–7717, 2019. 3
2019
-
[27]
Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age
Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. arXiv preprint arXiv:2309.03453, 2023. 3
2023 arXiv
-
[28]
Decoupled weight decay regularization, 2019
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. 5
2019
-
[29]
Occupancy networks: Learning 3d reconstruction in function space
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Se- bastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019. 3
2019
-
[30]
Text2mesh: Text-driven neural stylization for meshes
Oscar Michel, Roi Bar-On, Richard Liu, Sagie Benaim, and Rana Hanocka. Text2mesh: Text-driven neural stylization for meshes. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13482–13492, 2021. 6, 12
2022
-
[31]
Local light field fusion: Practical view syn- thesis with prescriptive sampling guidelines
Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view syn- thesis with prescriptive sampling guidelines. ACM Transac- tions on Graphics (ToG), 38(4):1–14, 2019. 3
2019
-
[32]
Srinivasan, Matthew Tancik, Jonathan T
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis, 2020. 3
2020
-
[33]
Nerf: Representing scenes as neural radiance fields for view syn- thesis
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 1, 3
2021
-
[34]
Deepsdf: Learning con- tinuous signed distance functions for shape representation
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning con- tinuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 165–174, 2019. 3
2019
-
[35]
Dreamfusion: Text-to-3d using 2d diffusion
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 1, 2, 3
2022 arXiv
-
[36]
Learning transferable visual models from natural language supervi- sion
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...
2021
-
[37]
Sharf: Shape-conditioned radiance fields from a sin- gle view
Konstantinos Rematas, Ricardo Martin-Brualla, and Vittorio Ferrari. Sharf: Shape-conditioned radiance fields from a sin- gle view. arXiv preprint arXiv:2102.08860, 2021. 3
2021 arXiv
-
[38]
Texture: Text-guided texturing of 3d shapes, 2023
Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or. Texture: Text-guided texturing of 3d shapes, 2023. 6, 12
2023
-
[39]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3, 4, 5
2022
-
[40]
U-net: Convolutional networks for biomedical image segmentation,
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation,
-
[41]
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...
2023
-
[42]
Photorealistic text-to-image diffusion models with deep language understanding
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...
2022
-
[43]
Scene representation networks: Continuous 3d- structure-aware neural scene representations
Vincent Sitzmann, Michael Zollh ¨ofer, and Gordon Wet- zstein. Scene representation networks: Continuous 3d- structure-aware neural scene representations. Advances in Neural Information Processing Systems, 32, 2019. 3
2019
-
[44]
Splatter image: Ultra-fast single-view 3d recon- struction
Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d recon- struction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10208– 10217, 2024. 3
2024
-
[45]
Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024. 3
2024 arXiv
-
[46]
Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation, 2024
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation, 2024. 1, 2, 3
2024
-
[47]
Triposr: Fast 3d object reconstruction from a single image, 2024
Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image, 2024. 3
2024
-
[48]
Single-view view synthe- sis with multiplane images
Richard Tucker and Noah Snavely. Single-view view synthe- sis with multiplane images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 551–560, 2020. 3
2020
-
[49]
Multi-view supervision for single-view re- construction via differentiable ray consistency
Shubham Tulsiani, Tinghui Zhou, Alexei A Efros, and Ji- tendra Malik. Multi-view supervision for single-view re- construction via differentiable ray consistency. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 2626–2634, 2017. 3
2017
-
[50]
Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12619–12629, 2023. 3
2023
-
[51]
Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. ArXiv, abs/2305.16213, 2023. 1
2023 arXiv
-
[52]
Crm: Single image to 3d textured mesh with convolu- tional reconstruction model, 2024
Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun 10 Zhu. Crm: Single image to 3d textured mesh with convolu- tional reconstruction model, 2024. 3
2024
-
[53]
Agg: Amor- tized generative 3d gaussians for single image to 3d, 2024
Dejia Xu, Ye Yuan, Morteza Mardani, Sifei Liu, Jiaming Song, Zhangyang Wang, and Arash Vahdat. Agg: Amor- tized generative 3d gaussians for single image to 3d, 2024. 1, 2
2024
-
[54]
Point- nerf: Point-based neural radiance fields
Qiangeng Xu, Zexiang Xu, Julien Philip, Sai Bi, Zhixin Shu, Kalyan Sunkavalli, and Ulrich Neumann. Point- nerf: Point-based neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5438–5448, 2022. 3
2022
-
[55]
Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation, 2024
Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation, 2024. 3
2024
-
[56]
Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models, 2024
Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models, 2024. 1, 2, 3
2024
-
[57]
pixelnerf: Neural radiance fields from one or few images
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4578–4587, 2021. 1, 3
2021
-
[58]
Paint3d: Paint anything 3d with lighting-less texture diffusion models,
Xianfang Zeng, Xin Chen, Zhongqi Qi, Wen Liu, Zibo Zhao, Zhibin Wang, Bin Fu, Yong Liu, and Gang Yu. Paint3d: Paint anything 3d with lighting-less texture diffusion models,
-
[59]
Yeo, Bo Dai, and Chen Change Loy
Junzhe Zhang, Daxuan Ren, Zhongang Cai, C. Yeo, Bo Dai, and Chen Change Loy. Monocular 3d object reconstruction with gan inversion. ArXiv, abs/2207.10061, 2022. 1
2022 arXiv
-
[60]
The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6, 7
2018
-
[61]
Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting, 2024
Xiaoyu Zhou, Xingjian Ran, Yajiao Xiong, Jinlin He, Zhi- wei Lin, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting, 2024. 1, 2
2024
-
[62]
change color to red
Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. arXiv preprint arXiv:2312.03594, 2023. 5 11 Contents A . Introduction 12 B . Implementation Details 12 C . Experi...
2023 arXiv
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.