Pith. sign in

REVIEW 3 major objections 4 minor 105 references

ARM: Appearance Reconstruction Model for Relightable 3D Generation

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By moving appearance reconstruction into UV texture space, ARM produces sharper, relightable 3D assets from sparse-view images than previous feed-forward methods.

desk verdict A competent but provisional feed-forward systems paper: the UV-space InstantAlbedo stage is the real new idea, and the evaluation protocol needs tightening before the 'outperforms' claim can be fully trusted. read the letter →

arxiv 2411.10825 v1 pith:XUTUMZSQ submitted 2024-11-16 cs.CV cs.GR

Xiang Feng , Chang Yu , Zoubin Bi , Yintong Shang , Feng Gao , Hongzhi Wu , Kun Zhou , Chenfanfu Jiang
show 1 more author
Yin Yang
This is my paper · ORCID
classification cs.CVcs.GR
keywords image-to-3DreconstructionrelightableappearanceUVtexturespacePhysically-BasedRenderingmapsmaterialdecompositionsparse-viewlargemodelprior
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ARM is a feed-forward system that turns sparse-view images of an object into a ready-to-use 3D mesh with physically based appearance. Its central claim is that geometry and appearance should be handled by separate networks, and that appearance should be reconstructed directly in UV texture space rather than decoded from triplanes at fixed resolution. On that basis it back-projects the input views, masks, positions, view directions, and surface normals onto the texture atlas, fuses them with a global-receptive-field inpainting network, and adds a material-aware image prior to separate illumination from albedo. The paper reports that this yields sharper textures, spatially varying roughness and metalness, and better relighting than prior feed-forward models such as InstantMesh, MeshFormer, and SF3D, trained on 8 H100 GPUs in about five days. A sympathetic reader would take away that texture resolution and material disentanglement, not geometry, are the remaining bottleneck for practical single-image 3D generation.

What carries the argument

The load-bearing mechanism is the UV texture-space appearance stage, which ARM calls InstantAlbedo. Instead of querying a triplane with an MLP, ARM first unwraps the predicted mesh into atlas charts, then explicitly back-projects each input view and its auxiliary data (mask, position, texture coordinate, view direction, normal, and material-encoded features) onto the texture map. Six per-view U-Net feature maps are fused by max-pooling, and an FFC-Net with a global receptive field inpaints unseen texels and refines the result; a material prior from a material-aware image encoder is injected through the back-projection so the network can tell material boundaries apart from lighting gradients. This machinery is what lets ARM produce spatially varying albedo, roughness, and metalness at texture resolution without scaling up the triplane transformer.

What would settle it

One decisive test is to take a GeoRM mesh, unwrap it with two different atlas parameterizations, and run InstantAlbedo on both; if the predicted albedo and relit renders differ substantially in sharpness or show misaligned seams, the central assumption of transferable texel correspondence fails. A sharper test: at inference, replace the unwrapping code used to build the training atlas with an independent UV unwrapper, and check whether PSNR and LPIPS on relit renders degrade on GSO.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the blurriness and baked-in lighting that plague feed-forward image-to-3D models come from decoding appearance from triplanes, and that both problems disappear when appearance is processed in UV texture space. ARM's GeoRM predicts a density field and extracts a mesh with differentiable marching cubes; the mesh is unwrapped into atlas charts, and InstantAlbedo back-projects the six input views together with masks, positions, texture coordinates, view directions, and normals onto those charts. A per-view U-Net extracts features, max-pooling fuses the views, and an FFC-Net with a global receptive field inpaints unseen regions, producing both a lighting-baked color map and a decomposed diffuse albedo map. Roughness and metalness are predicted separately per vertex by GlossyRM, and a material-aware image encoder (a DINO ViT initialized on semantic material maps and fine-tuned jointly) is back-projected as a material prior so that strong lighting does not bake into the albedo. The authors report that this design outperforms LGM, CRM, InstantMesh, MeshFormer, and SF3D on GSO and OmniObject3D for both geometry and appearance, and beats SF3D on relit renders under novel lighting.

Load-bearing premise

The method assumes that the flattening of a newly predicted mesh onto a 2D texture atlas at inference matches the flattening that was used to prepare the pre-unwrapped training shapes; if those two unwrappings disagree, the image pixels and material features back-projected onto the texture land in the wrong spots, and the sharpness gains would not transfer.

Editorial extensions

If this is right

  • Single-image 3D generation can produce assets with true PBR maps, not just vertex colors, so downstream engines can relight them under arbitrary environments.
  • Texture sharpness no longer scales with triplane resolution, because appearance is resolved on the UV atlas where measurements are directly visible.
  • A learned material prior can separate illumination from albedo even with only six views, reducing the baked-in-lighting artifacts that inverse-rendering losses leave behind.
  • Geometry and appearance can be trained separately, which keeps the memory footprint low enough that triplane resolution can be raised to 256 and the whole system trained on 8 GPUs.
  • The reported gains on GSO, OmniObject3D, and the relighting dataset imply that the UV-space pipeline transfers across object categories outside the Objaverse training subset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Pith inference: the UV-space stage is modular: any mesh with a consistent unwrapping could receive InstantAlbedo's texture refinement, so ARM's appearance gains may transfer to other geometry generators or to user-provided meshes.
  • Pith inference: the material prior's separation of albedo and lighting suggests a testable extension to unseen material categories or to in-the-wild photos with unknown lighting, where the prior would need to generalize beyond Objaverse's material distribution.
  • Pith inference: because the training atlas and the inference mesh are unwrapped by different code paths, the method's robustness to mesh perturbation and re-meshing is a risk worth measuring directly; a geometry generator that produces a slightly different triangulation could break texel correspondence.
  • Pith inference: pairing this texture-space pipeline with a video-based multi-view generator instead of Zero123++ could reduce the cross-view inconsistencies the paper names as its main limitation, since back-projection would then average over more consistent measurements.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ARM, a feed-forward image-to-3D reconstruction system that separates geometry from appearance. GeoRM predicts a density field through an LRM-style triplane synthesizer and extracts a mesh with differentiable marching cubes; GlossyRM predicts per-vertex roughness and metalness; InstantAlbedo back-projects six multi-view images and auxiliary data into an atlas UV texture space and uses a U-Net plus an FFC-Net with global receptive field to predict lighting-baked color and decomposed diffuse albedo, aided by a material-aware image encoder. Training uses rendered Objaverse subsets, including 55K pre-unwrapped GeoRM meshes for InstantAlbedo. Experiments on GSO and OmniObject3D report improved geometry and appearance metrics over LGM, CRM, InstantMesh, SF3D, and MeshFormer, and ablations show benefits from the back-projected measurements, the material prior, and the FFC-Net. A supplementary relighting benchmark compares ARM with SF3D on 100 PBR objects.

Significance. Relightable appearance reconstruction from sparse views is an important and timely problem, and ARM's architecture is a credible engineering contribution: it moves appearance synthesis into UV texture space to circumvent triplane resolution limits, fuses multi-view information with max-pooling, inpaints unseen texture regions with a global-receptive-field network, and addresses material/lighting ambiguity with a learned material prior. The design choices are individually motivated and the ablations in Table 3 and Figure 6 support their contribution. Training on 8 H100 GPUs is also a practical strength. If the evaluation protocol is made fully rigorous, the method would be a solid addition to the feed-forward 3D generation literature, with concrete value for relighting applications.

major comments (3)
  1. [§7.2 and Supp. §14] The alignment protocol is load-bearing for the central 'outperforms' claim and is underspecified. Supp. §14 states that rotations and scales are sampled, refined with ICP, and that the authors 'select the alignment with the highest evaluation score,' but it does not state how many restarts are used, which evaluation score (F-Score, CD, PSNR, LPIPS, or a combination) governs the selection, or whether the same selected alignment is used for all reported metrics. The Table 1 margins over MeshFormer are small (e.g., GSO F-Score 0.968 vs 0.966; PSNR 21.692 vs 20.500; LPIPS 0.137 vs 0.141), so a per-metric or test-set-tuned alignment search can plausibly account for the reported gaps. Please specify the exact number of restarts, the selection criterion, and whether the chosen alignment is fixed across all geometry and appearance metrics; if possible, report results under a single pre-registered, metric-independent alignment.
  2. [§7.2 and Table 2] The relightable-appearance superiority claim rests on an author-created benchmark whose protocol is not independently checkable. The 100-object PBR set, the environment maps, the object selection criteria, and the rendering pipeline are not fully specified, only SF3D is used as a baseline, and no error bars or per-object distributions are reported. Moreover, Supp. Table 5 explicitly labels the PBR-map comparison as 'only indicative,' yet the main text uses these results to support strong conclusions about material decomposition. Please release the benchmark or evaluate on an existing public PBR dataset, add at least one additional relightable baseline, and provide variance information or significance tests for the metrics in Tables 1 and 2.
  3. [§5.2 and §6] The UV-space approach assumes that the atlas unwrapping used to pre-unwrap the 55K training shapes from GeoRM matches the unwrapping applied to a new GeoRM mesh at inference. The paper does not state whether the same unwrapping implementation, chart-cutting parameters, and texel sampling are used in both places. If the inference-time unwrapping differs, the back-projected image measurements, auxiliary maps, and learned inpainting cues are misaligned with the texel-to-surface correspondence seen in training, and the reported texture-quality gains would not transfer. Please state explicitly that the training and inference unwrappings are identical, or train InstantAlbedo with the same on-the-fly unwrapping used at test time.
minor comments (4)
  1. [Throughout] The manuscript contains several typographical errors (e.g., 'Conventionly' in the Introduction, 'illustred' in Section 4, 'latter processing' for 'later processing', and 'reflective properties' for 'reflectance properties'); a copyedit pass would improve readability.
  2. [§6 and §7.1] The relationship between the number of input views and the loss views is unclear: Section 6 says 'with 6 used as input' while Section 7.1 sets N = 10 for view selection, and Equations (2)–(4) sum over N views. Please clarify whether N refers to the number of supervision views, input views, or both, and how the six input views are chosen.
  3. [Supp. Table 5 and §7.2] Since the supplementary PBR-map comparison is explicitly described as 'only indicative,' the main text should carry this caveat when discussing material-decomposition quality in Section 7.2 and Figure 5, rather than presenting the comparison as a definitive quantitative result.
  4. [Fig. 2 caption] The caption lists 'Density ρ m Roughness & Metalness' as outputs of the same MLP, but the pipeline assigns density to GeoRM and roughness/metalness to GlossyRM; the caption should separate the two models to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ARM's reconstruction claims are empirically supervised and benchmarked against external datasets; no prediction reduces to its inputs by construction.

full rationale

ARM is an empirical feed-forward reconstruction system. Its geometry and appearance stages are trained with supervised losses on ground-truth renders (Eqs. 2-4) and evaluated on external GSO and OmniObject3D benchmarks, so the reported geometry and lighting-baked appearance numbers are not defined in terms of the method's own outputs. InstantAlbedo is trained on 55K shapes generated by GeoRM, but that matches its inference-time input distribution (Sec. 6) rather than fitting any evaluation target; the UV-space back-projection is a consistent training/evaluation convention, not a prediction equal to its input. The material prior is an encoder initialized from external weights [58] and fine-tuned against ground-truth materials; the FFC-Net and measurement back-projection contributions are supported by ablations (Tab. 3), so their role is empirical. Self-citations ([1], [88]) occur in related-work and dataset-similarity contexts and are not load-bearing for the central claim. The Supp. Sec. 14 alignment protocol ('We select the alignment with the highest evaluation score') is a legitimate benchmark-validity concern: metric-driven alignment search can inflate small margins in Tables 1-2, and the search details are unspecified. This is an evaluation-protocol risk, not a circular derivation, because the predicted meshes are produced without access to ground truth and the alignment is applied post-hoc to all methods. The paper's own caveats (multi-view inconsistencies in Sec. 8; 'indicative scores' for PBR maps in Supp. Tab. 5) further weaken the strength of some secondary claims but do not make any central result equivalent to its inputs by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a supervised training pipeline rather than a mathematical derivation. The main unverified premises are the stability of UV atlas correspondence between training and inference, the consistency of diffusion-generated multi-view inputs, and the transfer of the material prior. The hand-chosen loss weights and input view count are free parameters. No invented entities are introduced.

free parameters (3)
  • Loss balancing coefficients λz, λM, λn, λ1, λ2, λ3 = 0.5, 1.0, 0.2, 0.7, 0.3, 0.1
    Hand-chosen weights for geometry and material losses (Eqs. 2 to 4); no sensitivity study is reported, yet they determine the trained models that produce all benchmark numbers.
  • Input view count (6 views) = 6
    The pipeline uses six back-projected views for appearance; the choice is fixed by the Zero123++ setting and is not varied in ablations.
  • Evaluation alignment and metric thresholds = F-score threshold 0.1; 100,000 sampled points; scale range 0.7 to 1.4
    Hand-chosen evaluation settings. Alignment that maximizes the score is selected, which can inflate reported numbers.
assumptions (5)
  • domain assumption Appearance is described by a microfacet SVBRDF with GGX distribution (Eq. 1), with only diffuse albedo, roughness, and metalness.
    Section 3 and Eq. 1; no global illumination, interreflection, transmission, or subsurface effects are modeled, so the network can only encode what this model can express.
  • domain assumption The six input views produced by Zero123++ from a single image are sufficiently consistent for pixel-accurate back-projection.
    Sections 4 and 5.2. The authors themselves note in Section 8 that inconsistencies in upstream multi-view images introduce artifacts.
  • domain assumption The DINO-based material prior, pretrained on a semantic material dataset, transfers to in-the-wild images after fine-tuning on synthetic Objaverse renders.
    Section 5.3. The encoder's pretraining data and the synthetic training distribution may not cover all real-world materials, which affects albedo-lighting separation.
  • ad hoc to paper Offline pre-unwrapped atlas charts of GeoRM-output shapes have the same UV correspondence as atlas charts computed for new meshes at inference.
    Sections 5.2 and 6. 'Since unwrapping is time-consuming and impractical to perform during training, we synthesize a pre-unwrapped training dataset offline.' A mismatch between training and inference unwrapping would break texture back-projection.
  • standard math Differentiable marching cubes and rasterizer correctly extract and render the iso-surface from GeoRM's density grid.
    Section 5.1 relies on DiffMC [76] and a differentiable rasterizer [27]; these are established tools.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ARM: Appearance Reconstruction Model for Relightable 3D Generation." pith.science (2026). https://pith.science/paper/XUTUMZSQ

@misc{pith2026241110825,
  author       = {Pith},
  title        = {Pith review of: ARM: Appearance Reconstruction Model for Relightable 3D Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XUTUMZSQ}},
  note         = {Machine review of arXiv:2411.10825}
}
read the original abstract

Recent image-to-3D reconstruction models have greatly advanced geometry generation, but they still struggle to faithfully generate realistic appearance. To address this, we introduce ARM, a novel method that reconstructs high-quality 3D meshes and realistic appearance from sparse-view images. The core of ARM lies in decoupling geometry from appearance, processing appearance within the UV texture space. Unlike previous methods, ARM improves texture quality by explicitly back-projecting measurements onto the texture map and processing them in a UV space module with a global receptive field. To resolve ambiguities between material and illumination in input images, ARM introduces a material prior that encodes semantic appearance information, enhancing the robustness of appearance decomposition. Trained on just 8 H100 GPUs, ARM outperforms existing methods both quantitatively and qualitatively.

Figures

Figures reproduced from arXiv: 2411.10825 by the authors.

Figure 1
Figure 1. ARM generates high-quality, relightable 3D content from a single image input. This figure presents sample results generated from different input images, demonstrating ARM’s ability to reconstruct a variety of objects with spatially-varying appearance. Please refer to our supplementary video for results under dynamic view and lighting. Abstract Recent image-to-3D reconstruction models have greatly advanced geometry g… view at source ↗
Figure 2
Figure 2. Overview of our pipeline. (left) Starting from sparse-view input images generated by a diffusion model [59], ARM separates shape and appearance generation into two stages. In the geometry stage, ARM uses GeoRM to predict a 3D shape from the input images. In the appearance stage, ARM employs InstantAlbedo and GlossyRM to reconstruct PBR maps, enabling realistic relighting under varied lighting conditions. (right) Bot… view at source ↗
Figure 3
Figure 3. Overview of InstantAlbedo. InstantAlbedo operates in the texture UV space. This process begins by converting all necessary data to UV texture space. Given the unwrapped mesh from GeoRM, we back-project images, material encodings, and auxiliary data into UV texture space, resulting in six sets of inputs corresponding to the six input views. InstantAlbedo then processes these maps using a U-Net and an inpainting-speci… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison. We present examples of single-image 3D generation across different methods. While other methods exhibit blurriness, ARM reconstructs complex patterns with sharp details. Please zoom in to examine the texture quality. Full results, including comp…
Figure 5
Figure 5. Figure 5: PBR comparison. We compare reconstructed PBR maps and relit images under novel lighting to SF3D [4]. While SF3D produces constant roughness and material with lighting baked into the diffuse color (highlighted in the figure), our method generates spatially-varying appea…
Figure 6
Figure 6. Figure 6: Ablation results for FFC-Net and material prior. FFC-Net aids in inpainting unseen regions (top), while the ma￾terial prior improves the diffuse decomposition (bottom), both for the InstantAlbedo stage. 7.4. Ablations We perform both visual and quantitative ablation ex…
Figure 7
Figure 7. Figure 7: Architecture of tri￾plane synthesizer. A pretrained ViT image encoder [6] converts multi￾view input images into im￾age tokens. To make the network aware of camera pose, we add AdaLN cam￾era pose modulation lay￾ers to the ViT encoder, fol￾lowing Instant3D [29], en￾ablin…
Figure 9
Figure 9. Figure 9: Qualitative comparison. We present examples of single-image 3D generation across different methods. While other methods exhibit blurriness, ARM reconstructs complex patterns with sharp details. Please zoom in to examine the texture quality. 15. Additional results In Ta…
Figure 10
Figure 10. Figure 10: PBR comparison. We compare reconstructed PBR maps and relit images under novel lighting to SF3D [4]. While SF3D produces constant roughness and material with lighting baked into the diffuse color (highlighted in the figure), our method generates spatially-varying appe…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

105 extracted references · 38 canonical work pages

  1. [1]

    GS^3: Efficient Relighting with Triple Gaussian Splatting

    Zoubin Bi, Yixin Zeng, Chong Zeng, Fan Pei, Xiang Feng, Kun Zhou, and Hongzhi Wu. Gsˆ 3: Efficient relighting with triple gaussian splatting. arXiv preprint arXiv:2410.11419,

    work page Pith review arXiv

  2. [2]

    Nerd: Neural reflectance decomposition from image collections

    Mark Boss, Raphael Braun, Varun Jampani, Jonathan T Bar- ron, Ce Liu, and Hendrik Lensch. Nerd: Neural reflectance decomposition from image collections. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12684–12694, 2021. 3

    2021

  3. [3]

    Samurai: Shape and material from uncon- strained real-world arbitrary image collections

    Mark Boss, Andreas Engelhardt, Abhishek Kar, Yuanzhen Li, Deqing Sun, Jonathan Barron, Hendrik Lensch, and Varun Jampani. Samurai: Shape and material from uncon- strained real-world arbitrary image collections. Advances in Neural Information Processing Systems, 35:26389–26403,

  4. [4]

    Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement

    Mark Boss, Zixuan Huang, Aaryaman Vasishta, and Varun Jampani. Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement. arXiv preprint arXiv:2408.00653, 2024. 3, 6, 7, 8, 15, 17

    arXiv 2024

  5. [5]

    Physically-based shading at disney

    Brent Burley and Walt Disney Animation Studios. Physically-based shading at disney. In Acm Siggraph, pages 1–7. vol. 2012, 2012. 3

    2012

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021. 5, 14

    2021

  7. [7]

    Effi- cient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Effi- cient geometry-aware 3d generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 2

    2022

  8. [8]

    Generative novel view synthesis with 3d-aware diffusion models

    Eric R Chan, Koki Nagano, Matthew A Chan, Alexander W Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. Generative novel view synthesis with 3d-aware diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4217–4229, 2023. 2

    2023

Show all 105 references
  1. [9]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22246–22256, 2023. 2, 3

    2023

  2. [10]

    3dtopia-xl: Scaling high- quality 3d asset generation via primitive diffusion

    Zhaoxi Chen, Jiaxiang Tang, Yuhao Dong, Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie, Tong Wu, Shunsuke Saito, et al. 3dtopia-xl: Scaling high- quality 3d asset generation via primitive diffusion. arXiv preprint arXiv:2409.12957, 2024. 3

    2024 arXiv

  3. [11]

    Fast fourier convolu- tion

    Lu Chi, Borui Jiang, and Yadong Mu. Fast fourier convolu- tion. Advances in Neural Information Processing Systems, 33:4479–4488, 2020. 5

    2020

  4. [12]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  5. [13]

    Objaverse- xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse- xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  6. [14]

    Google scanned objects: A high-quality dataset of 3d scanned household items

    Laura Downs, Anthony Francis, Nate Koenig, Brandon Kin- man, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. In 2022 International Conference on Robotics and Automation (ICRA), ...

    2022

  7. [15]

    Shi- nobi: Shape and illumination using neural object decompo- sition via brdf optimization in-the-wild

    Andreas Engelhardt, Amit Raj, Mark Boss, Yunzhi Zhang, Abhishek Kar, Yuanzhen Li, Deqing Sun, Ricardo Martin Brualla, Jonathan T Barron, Hendrik Lensch, et al. Shi- nobi: Shape and illumination using neural object decompo- sition via brdf optimization in-the-wild. In Proceedin...

    2024

  8. [16]

    Make-it-real: Unleashing large multimodal model’s ability for painting 3d objects with realistic materials

    Ye Fang, Zeyi Sun, Tong Wu, Jiaqi Wang, Ziwei Liu, Gor- don Wetzstein, and Dahua Lin. Make-it-real: Unleashing large multimodal model’s ability for painting 3d objects with realistic materials. arXiv preprint arXiv:2404.16829, 2024. 3

    2024 arXiv

  9. [17]

    Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray trac- ing

    Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray trac- ing. arXiv preprint arXiv:2311.16043, 2023. 3

    2023 arXiv

  10. [18]

    Shape, light, and material decomposition from images using monte carlo rendering and denoising

    Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. Shape, light, and material decomposition from images using monte carlo rendering and denoising. Advances in Neural Information Processing Systems, 35:22856–22869, 2022. 3

    2022

  11. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 15

    2016

  12. [20]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  13. [21]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2

    2023 arXiv

  14. [22]

    Mvd-fusion: Single-view 3d via depth-consistent multi-view generation

    Hanzhe Hu, Zhizhuo Zhou, Varun Jampani, and Shubham Tulsiani. Mvd-fusion: Single-view 3d via depth-consistent multi-view generation. In Proceedings of the IEEE/CVF 9 Conference on Computer Vision and Pattern Recognition, pages 9698–9707, 2024. 2

    2024

  15. [23]

    Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces

    Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaox- iao Long, Wenping Wang, and Yuexin Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 53...

    2024

  16. [24]

    Shap-e: Generat- ing conditional 3d implicit functions

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 2

    2023 arXiv

  17. [25]

    Learn- ing efficient illumination multiplexing for joint capture of reflectance and shape

    Kaizhang Kang, Cihui Xie, Chengan He, Mingqi Yi, Minyi Gu, Zimin Chen, Kun Zhou, and Hongzhi Wu. Learn- ing efficient illumination multiplexing for joint capture of reflectance and shape. ACM Trans. Graph., 38(6):165:1– 165:12, 2019. 2

    2019

  18. [26]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  19. [27]

    Modular primi- tives for high-performance differentiable rendering

    Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primi- tives for high-performance differentiable rendering. ACM Transactions on Graphics, 39(6), 2020. 15

    2020

  20. [28]

    Photo- realistic single image super-resolution using a generative ad- versarial network

    Christian Ledig, Lucas Theis, Ferenc Husz´ar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo- realistic single image super-resolution using a generative ad- versarial network. In Proceedings of the IE...

  21. [29]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 2, 14

    2023 arXiv

  22. [30]

    Gs-ir: 3d gaussian splatting for inverse rendering

    Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21644–21653, 2024. 3

    2024

  23. [31]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2023

  24. [32]

    Consistent123: One image to highly consistent 3d asset using case-aware diffusion priors

    Yukang Lin, Haonan Han, Chaoqun Gong, Zunnan Xu, Yachao Zhang, and Xiu Li. Consistent123: One image to highly consistent 3d asset using case-aware diffusion priors. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 6715–6724, 2024. 2

    2024

  25. [33]

    One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion

    Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Ji- ayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. arXiv preprint arXiv:2311.07885, 2023. 2

    2023 arXiv

  26. [34]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  27. [35]

    Meshformer: High-quality mesh generation with 3d-guided reconstruction model.arXiv preprint arXiv:2408.10198, 2024

    Minghua Liu, Chong Zeng, Xinyue Wei, Ruoxi Shi, Ling- hao Chen, Chao Xu, Mengqi Zhang, Zhaoning Wang, Xi- aoshuai Zhang, Isabella Liu, et al. Meshformer: High-quality mesh generation with 3d-guided reconstruction model.arXiv preprint arXiv:2408.10198, 2024. 2, 6, 7, 15

    2024 arXiv

  28. [36]

    Zero-1-to-3: Zero-shot one image to 3d object, 2023

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object, 2023. 2

    2023

  29. [37]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. arXiv preprint arXiv:2309.03453, 2023. 2

    2023 arXiv

  30. [38]

    Text-guided texturing by synchronized multi-view diffusion

    Yuxin Liu, Minshan Xie, Hanyuan Liu, and Tien-Tsin Wong. Text-guided texturing by synchronized multi-view diffusion. arXiv preprint arXiv:2311.12891, 2023. 2

    2023 arXiv

  31. [39]

    Unidream: Unifying diffusion priors for re- lightable text-to-3d generation

    Zexiang Liu, Yangguang Li, Youtian Lin, Xin Yu, Sida Peng, Yan-Pei Cao, Xiaojuan Qi, Xiaoshui Huang, Ding Liang, and Wanli Ouyang. Unidream: Unifying diffusion priors for re- lightable text-to-3d generation. In European Conference on Computer Vision, pages 74–91. Springer, 2025. 3

    2025

  32. [40]

    Wonder3d: Sin- gle image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  33. [41]

    Att3d: Amortized text-to-3d object synthesis

    Jonathan Lorraine, Kevin Xie, Xiaohui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming- Yu Liu, Sanja Fidler, and James Lucas. Att3d: Amortized text-to-3d object synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pag...

    2023

  34. [42]

    Deep relightable textures: volumetric performance capture with neural rendering

    Abhimitra Meka, Rohit Pandey, Christian Haene, Sergio Orts-Escolano, Peter Barnum, Philip David-Son, Daniel Er- ickson, Yinda Zhang, Jonathan Taylor, Sofien Bouaziz, et al. Deep relightable textures: volumetric performance capture with neural rendering. ACM Transactions on Gra...

    2020

  35. [43]

    Latent-nerf for shape-guided generation of 3d shapes and textures

    Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12663–12673, 2023. 2

    2023

  36. [44]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 1, 2

    2021

  37. [45]

    Extracting Triangular 3D Models, Materials, and Lighting From Images

    Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas M ¨uller, and Sanja Fidler. Extracting Triangular 3D Models, Materials, and Lighting From Images. In Proceedings of the IEEE/CVF 10 Conference on Computer Vision and Pattern Recognition ...

    2022

  38. [46]

    Extracting triangular 3d models, materials, and lighting from images

    Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas M¨uller, and Sanja Fi- dler. Extracting triangular 3d models, materials, and lighting from images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2022

  39. [47]

    Practical svbrdf acquisition of 3d objects with unstructured flash photography

    Giljoo Nam, Joo Ho Lee, Diego Gutierrez, and Min H Kim. Practical svbrdf acquisition of 3d objects with unstructured flash photography. ACM Transactions on Graphics (ToG), 37(6):1–12, 2018. 2

    2018

  40. [48]

    Scalable diffusion mod- els with transformers

    William Peebles and Saining Xie. Scalable diffusion mod- els with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195– 4205, 2023. 3

    2023

  41. [49]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 2

    2022 arXiv

  42. [50]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. arXiv preprint arXiv:1612.00593, 2016. 5

    2016 arXiv

  43. [51]

    Magic123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors.arXiv preprint arXiv:2306.17843,

    Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin-Ying Lee, Ivan Sko- rokhodov, Peter Wonka, Sergey Tulyakov, et al. Magic123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors.arXiv preprint arXiv:2306.17843,

    arXiv

  44. [52]

    Diligent102: A photometric stereo benchmark dataset with controlled shape and material vari- ation

    Jieji Ren, Feishi Wang, Jiahao Zhang, Qian Zheng, Mingjun Ren, and Boxin Shi. Diligent102: A photometric stereo benchmark dataset with controlled shape and material vari- ation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages ...

    2022

  45. [53]

    Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies

    Xuanchi Ren, Jiahui Huang, Xiaohui Zeng, Ken Museth, Sanja Fidler, and Francis Williams. Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4209–4219, 2024. 2

    2024

  46. [54]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  47. [55]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  48. [56]

    Relightable gaussian codec avatars

    Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130–141, 2024. 2

    2024

  49. [57]

    Let 2d diffusion model know 3d- consistency for robust text-to-3d generation

    Junyoung Seo, Wooseok Jang, Min-Seop Kwak, Hyeonsu Kim, Jaehoon Ko, Junho Kim, Jin-Hwa Kim, Jiyoung Lee, and Seungryong Kim. Let 2d diffusion model know 3d- consistency for robust text-to-3d generation. arXiv preprint arXiv:2303.07937, 2023. 2

    2023 arXiv

  50. [58]

    Materialistic: Se- lecting similar materials in images

    Prafull Sharma, Julien Philip, Micha ¨el Gharbi, Bill Freeman, Fredo Durand, and Valentin Deschaintre. Materialistic: Se- lecting similar materials in images. ACM Trans. Graph., 42 (4), 2023. 5, 15

    2023

  51. [59]

    Zero123++: a single image to consistent multi-view dif- fusion base model

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model. arXiv preprint arXiv:2310.15110, 2023. 2, 3, 6

    arXiv

  52. [60]

    Mvdream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. arXiv preprint arXiv:2308.16512, 2023. 2

    2023 arXiv

  53. [61]

    Meta 3d assetgen: Text-to-mesh generation with high- quality geometry, texture, and pbr materials

    Yawar Siddiqui, Tom Monnier, Filippos Kokkinos, Mahen- dra Kariya, Yanir Kleiman, Emilien Garreau, Oran Gafni, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, et al. Meta 3d assetgen: Text-to-mesh generation with high- quality geometry, texture, and pbr materials. arXiv pr...

    2024 arXiv

  54. [62]

    Nerv: Neural reflectance and visibility fields for relight- ing and view synthesis

    Pratul P Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T Barron. Nerv: Neural reflectance and visibility fields for relight- ing and view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pag...

    2021

  55. [63]

    Dreamcraft3d: Hierarchi- cal 3d generation with bootstrapped diffusion prior

    Jingxiang Sun, Bo Zhang, Ruizhi Shao, Lizhen Wang, Wen Liu, Zhenda Xie, and Yebin Liu. Dreamcraft3d: Hierarchi- cal 3d generation with bootstrapped diffusion prior. arXiv preprint arXiv:2310.16818, 2023. 2

    2023 arXiv

  56. [64]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF winte...

    2022

  57. [65]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024. 2, 6, 7

    2024 arXiv

  58. [66]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024. 2

    2024 arXiv

  59. [67]

    Collaborative control for geometry- conditioned pbr image generation

    Shimon Vainer, Mark Boss, Mathias Parger, Konstantin Kutsy, Dante De Nigris, Ciara Rowles, Nicolas Perony, and Simon Donn ´e. Collaborative control for geometry- conditioned pbr image generation. In European Conference on Computer Vision, pages 127–145. Springer, 2025. 3

    2025

  60. [68]

    Sv3d: Novel multi-view synthesis and 3d generation from a single image using la- tent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from a single image using la- tent video diffusion. In European Conference on Computer...

    2025

  61. [69]

    Microfacet models for refraction through rough surfaces

    Bruce Walter, Stephen R Marschner, Hongsong Li, and Ken- neth E Torrance. Microfacet models for refraction through rough surfaces. Rendering techniques, 2007:18th, 2007. 14

    2007

  62. [70]

    Score jacobian chaining: Lift- ing pretrained 2d diffusion models for 3d generation

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lift- ing pretrained 2d diffusion models for 3d generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12619–12629, 2023. 2

    2023

  63. [71]

    Imagedream: Image-prompt multi-view diffusion for 3d generation

    Peng Wang and Yichun Shi. Imagedream: Image-prompt multi-view diffusion for 3d generation. arXiv preprint arXiv:2312.02201, 2023. 2

    2023 arXiv

  64. [72]

    Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021. 1, 2

    2021 arXiv

  65. [73]

    Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction

    Peng Wang, Hao Tan, Sai Bi, Yinghao Xu, Fujun Luan, Kalyan Sunkavalli, Wenping Wang, Zexiang Xu, and Kai Zhang. Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction. arXiv preprint arXiv:2311.12024, 2023. 2

    2023 arXiv

  66. [74]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  67. [75]

    Crm: Single image to 3d textured mesh with convolutional reconstruction model

    Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xi- ang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convolutional reconstruction model. arXiv preprint arXiv:2403.05034, 2024. 2, 6, 7

    2024 arXiv

  68. [76]

    Neumanifold: Neural watertight manifold reconstruction with efficient and high- quality rendering support

    Xinyue Wei, Fanbo Xiang, Sai Bi, Anpei Chen, Kalyan Sunkavalli, Zexiang Xu, and Hao Su. Neumanifold: Neural watertight manifold reconstruction with efficient and high- quality rendering support. arXiv preprint arXiv:2305.17134,

    arXiv

  69. [77]

    Meshlrm: Large reconstruction model for high- quality mesh

    Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, and Zex- iang Xu. Meshlrm: Large reconstruction model for high- quality mesh. arXiv preprint arXiv:2404.12385, 2024. 2

    2024 arXiv

  70. [78]

    Unique3d: High-quality and efficient 3d mesh generation from a single image

    Kailu Wu, Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu, Yueqi Duan, and Kaisheng Ma. Unique3d: High-quality and efficient 3d mesh generation from a single image. arXiv preprint arXiv:2405.20343, 2024. 2

    2024 arXiv

  71. [79]

    Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation

    Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Liang Pan Jiawei Ren, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu. Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. In IEEE/CVF Conference on Computer V...

    2023

  72. [80]

    Latte3d: Large-scale amortized text-to-enhanced3d synthe- sis

    Kevin Xie, Jonathan Lorraine, Tianshi Cao, Jun Gao, James Lucas, Antonio Torralba, Sanja Fidler, and Xiaohui Zeng. Latte3d: Large-scale amortized text-to-enhanced3d synthe- sis. arXiv preprint arXiv:2403.15385, 2024. 2

    2024 arXiv

  73. [81]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191,

    arXiv

  74. [82]

    Mat- laber: Material-aware text-to-3d via latent brdf auto-encoder

    Xudong Xu, Zhaoyang Lyu, Xingang Pan, and Bo Dai. Mat- laber: Material-aware text-to-3d via latent brdf auto-encoder. arXiv preprint arXiv:2308.09278, 2023. 3

    2023 arXiv

  75. [83]

    Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model

    Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Ji- ahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model. arXiv preprint arXiv:2311.09217, 2023. 2

    2023 arXiv

  76. [84]

    Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation. arXiv preprint arXiv:2403.14621, 2024. 2

    2024 arXiv

  77. [85]

    Consistent-1-to-3: Consistent image to 3d view synthesis via geometry-aware diffusion models

    Jianglong Ye, Peng Wang, Kejie Li, Yichun Shi, and Heng Wang. Consistent-1-to-3: Consistent image to 3d view synthesis via geometry-aware diffusion models. In 2024 International Conference on 3D Vision (3DV), pages 664–

    2024

  78. [86]

    Paint- it: Text-to-texture synthesis via deep convolutional tex- ture map optimization and physically-based rendering

    Kim Youwang, Tae-Hyun Oh, and Gerard Pons-Moll. Paint- it: Text-to-texture synthesis via deep convolutional tex- ture map optimization and physically-based rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4347–4356, 2024. 3

    2024

  79. [87]

    Relighting neural radiance fields with shadow and highlight hints

    Chong Zeng, Guojun Chen, Yue Dong, Pieter Peers, Hongzhi Wu, and Xin Tong. Relighting neural radiance fields with shadow and highlight hints. In ACM SIGGRAPH 2023 Conference Proceedings, 2023. 3

    2023

  80. [88]

    Dilightnet: Fine-grained light- ing control for diffusion-based image generation

    Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained light- ing control for diffusion-based image generation. In ACM SIGGRAPH 2024 Conference Papers, 2024. 3, 6

    2024

  81. [89]

    Physg: Inverse rendering with spherical gaussians for physics-based material editing and relighting

    Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, and Noah Snavely. Physg: Inverse rendering with spherical gaussians for physics-based material editing and relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5453–5462, 2021. 3

    2021

  82. [90]

    Gs-lrm: Large re- construction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large re- construction model for 3d gaussian splatting. In European Conference on Computer Vision, pages 1–19. Springer,

  83. [91]

    Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 3

    2024

  84. [92]

    Ner- factor: Neural factorization of shape and reflectance under an unknown illumination

    Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul De- bevec, William T Freeman, and Jonathan T Barron. Ner- factor: Neural factorization of shape and reflectance under an unknown illumination. ACM Transactions on Graphics (ToG), 40(6):1–18, 2021. 3

    2021

  85. [93]

    Dreammat: High-quality pbr material generation with geometry-and light-aware diffusion models

    Yuqing Zhang, Yuan Liu, Zhiyu Xie, Lei Yang, Zhongyuan Liu, Mengzhou Yang, Runze Zhang, Qilong Kou, Cheng Lin, Wenping Wang, et al. Dreammat: High-quality pbr material generation with geometry-and light-aware diffusion models. ACM Transactions on Graphics (TOG), 43(4):1–18, 2024. 3 12

    2024

  86. [94]

    Locally attentional sdf diffusion for controllable 3d shape generation

    Xin-Yang Zheng, Hao Pan, Peng-Shuai Wang, Xin Tong, Yang Liu, and Heung-Yeung Shum. Locally attentional sdf diffusion for controllable 3d shape generation. ACM Transactions on Graphics (ToG), 42(4):1–13, 2023. 2

    2023

  87. [95]

    Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers

    Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  88. [96]

    1 ARM models the appearance of object by a spatially vary- ing BRDF described in Eq

    Detailed explanation of Eq. 1 ARM models the appearance of object by a spatially vary- ing BRDF described in Eq. 1. For the microfacet nor- mal distribution term D, we use isotropic GGX distribu- tion [69]: D(n, h, α) = α2 π((n · h)(α2 − 1) + 1)2 , α= ρ2, where n is the half-w...

  89. [97]

    Details on GeoRM and GlossyRM GeoRM and GlossyRM are built on the LRM frame- work, with a super-resolution upsampler added to the triplane synthesizer, as shown in Fig. 7. … Image Tokens Triplane Tokens Transformer Upsampler Encoder Triplane Synthesizer 512 3 32 2048 3 32 Figu...

  90. [98]

    Unified material prediction ARM separates PBR parameter prediction into two net- works: InstantAlbedo for diffuse albedo and GlossyRM for roughness and metalness. Although predicting all material properties within InstantAlbedo might seem more straight- forward, our experiment...

  91. [99]

    The material-aware image encoder is based on [58], excluding the user reference injection and cross- attention layers

    Details on InstantAlbedo The InstantAlbedo framework comprises three main net- works: a material-aware image encoder, a U-Net, and an FFC-Net. The material-aware image encoder is based on [58], excluding the user reference injection and cross- attention layers. For the FFC-Net...

  92. [100]

    This subset is carefully curated based on the following criteria to ensure high-quality train- ing data:

    Dataset selection GeoRM and GlossyRM are trained on a 150K subset of the Objaverse dataset [12]. This subset is carefully curated based on the following criteria to ensure high-quality train- ing data:

  93. [101]

    This requirement ensures that the objects have sufficient material data for training GlossyRM

    Each selected object must include a roughness map or a metalness map. This requirement ensures that the objects have sufficient material data for training GlossyRM

  94. [102]

    The object must not be a point cloud, nor a sparse or small object with low occupancy (fewer than 10 pixels per rendered view)

  95. [103]

    Low-quality objects, such as scanned indoor data or large scenes with multiple objects, are excluded

  96. [104]

    Fol- lowing MeshFormer [35], we use a two-step alignment based on the evaluation metric

    Shape alignment During evaluation, we align each method’s predicted meshes to the ground truth meshes before calculating met- rics, as coordinate frames may differ across methods. Fol- lowing MeshFormer [35], we use a two-step alignment based on the evaluation metric. First, w...

  97. [105]

    5, We report quantitative metrics comparing the pre- dicted PBR maps with ground truth, using SF3D and our method

    Additional results In Tab. 5, We report quantitative metrics comparing the pre- dicted PBR maps with ground truth, using SF3D and our method. Due to the high ambiguity in appearance decom- position, where multiple valid decompositions can explain the same shaded image, we only...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.