Pith. sign in

REVIEW 2 major objections 6 minor 109 references

LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields

T0 review · 2 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new transformer turns 3–6 photos into 3D objects in under a second

desk verdict Serious feed-forward inverse-rendering system with a genuinely useful progressive-update module; the real-data claim of beating optimization methods is plausible but rests on masked crops and 18 images rather than the advertised 3–6. read the letter →

arxiv 2504.20026 v1 pith:7JWCRKZX submitted 2025-04-28 cs.CV cs.AI

Zhengqin Li , Dilin Wang , Ka Chen , Zhaoyang Lv , Thu Nguyen-Phuoc , Milim Lee , Jia-Bin Huang , Lei Xiao
show 6 more authors
Cheng Zhang Yufeng Zhu Carl S. Marshall Yufeng Ren Richard Newcombe Zhao Dong
This is my paper · ORCID
classification cs.CVcs.AI
keywords inverserenderinglargereconstructionmodelprogressivehexa-planeneuralSDFdirectionalencodingview-dependentradiancefieldsrelightingsparse-view3D
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LIRM is a feed-forward transformer that takes three to six posed images of an object and, in under a second, outputs geometry, PBR material maps (albedo, metallic, roughness), and a view-dependent radiance field suitable for relighting in standard graphics pipelines. The paper's central claim is that sparse-view inverse rendering no longer needs dense captures or long optimization: with three progressive sets of six images, LIRM reaches reconstruction quality on par with, and on geometry metrics better than, optimization-based methods that run for hours on dense views. A sympathetic reader would care because this is the difference between an asset-creation workflow that takes minutes or hours and one that feels interactive, while still producing relightable content rather than view-baked radiance. The authors report that additional views can be added progressively without increasing GPU memory, and identify two remaining limits: mirror reflections and very fine texture detail.

What carries the argument

The load-bearing representation is the hexa-plane neural SDF: six feature planes divide the bounding volume into eight octants, each with its own tri-plane, so front and back textures do not share a single plane and leak into each other. From these features, small MLPs predict signed distance, albedo, metallic, roughness, normal, and view-dependent color, with a signed-distance bias $\|x\|-0.1R$ to stabilize training. View dependence is handled by neural directional encoding (NDE): the transformer outputs multiple 128x128 feature panoramas, and at each surface point a small network selects which panorama to query with the reflection direction, modeling near-field and occluded reflections without a second-bounce computation. The update module feeds the previous set of hexa-plane and NDE tokens back through self-attention together with new image tokens, so extra views refine the reconstruction at constant GPU memory.

What would settle it

Run LIRM on Stanford-ORB full frames without foreground-mask cropping: if relighting and view-synthesis metrics degrade markedly relative to the masked-crop numbers in Table 5, the posed-image claim depends on segmentation, not just pose.

Watch

Extended reading notes

Core claim

The paper claims that a single transformer, trained on 38 million rendered images from 600k objects with ground-truth PBR materials, jointly predicts a hexa-plane neural SDF and four NDE panoramas that encode view-dependent appearance. At inference, volume rendering from the SDF yields a mesh via marching cubes, and BRDF maps are extracted on surface points; relighting is then direct. On Stanford-ORB, LIRM with three sets of input images achieves the lowest chamfer distance among compared methods (0.31 vs 0.43 for the best optimization baseline) and competitive relighting and view-synthesis PSNR, while running in about 0.3 seconds per update step on an A100. The authors state this is 'on par and even better than state-of-the-art optimization-based methods, which take dense views as inputs and several hours to run,' and they explicitly acknowledge that mirror-like reflections and the finest texture details remain unsolved.

Load-bearing premise

On real data, LIRM is evaluated with inputs cropped and resized to the object's foreground mask, so the practical claim of reconstruction from arbitrary posed images assumes foreground segmentation is available or accurate at deployment.

Editorial extensions

If this is right

  • A capture loop can be interactive: after an initial reconstruction from 3–6 images, users add views of unseen regions and the same transformer refines the asset without retraining or memory growth.
  • Relightable assets become cheap enough for mass production: mesh plus PBR maps come out of a feed-forward pass, so standard game, film, and VFX pipelines can consume them directly.
  • Sparse-view inverse rendering closes much of the quality gap to dense-view optimization: on Stanford-ORB, geometry (chamfer distance) is the best among compared methods, suggesting hours-long optimization may be avoidable for many objects.
  • Because background images are fed alongside masked object images, the model can separate lighting color from material color, improving relighting fidelity under novel environments.
  • The same progressive-update mechanism may extend to scenes too large for one capture, as the paper demonstrates on a changing scene and on egocentric captures without fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the real-world protocol is switched to full images without foreground-mask cropping, the gap between LIRM and optimization-based methods may widen; a direct masked-versus-unmasked comparison would settle whether the 'posed images' promise includes busy backgrounds.
  • The multi-panorama NDE design suggests a general recipe for feed-forward models: replace spherical-harmonic bottlenecks with learned directional feature volumes, which could improve other large reconstruction models whenever shiny materials matter.
  • The update module's robustness to camera-trajectory order (shown in supplementary sequential-sampling experiments) hints that the same network could serve as an online object refiner, updating a 3D asset as a person walks around it.
  • A stress test the authors did not run: evaluate on a dataset with explicit mirror-like BRDFs to quantify how much the NDE panoramas can be pushed before requiring a second-bounce term.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. LIRM is a transformer-based feed-forward inverse rendering model that reconstructs shape (as a neural SDF), BRDF materials, and view-dependent radiance from sparse posed images in under a second. The paper makes three main technical contributions: a progressive update module that refines reconstruction as additional input view sets are provided without growing the transformer token count; a hexa-plane neural SDF representation that improves texture detail relative to tri-planes; and a neural directional encoding (NDE) mechanism with multiple predicted panoramas for view-dependent appearance. The model is trained on a large synthetic dataset (600k objects, 38M rendered images) with PBR materials and HDR environment lighting. Experiments on GSO, ABO, and DTC show strong view synthesis and inverse rendering results, and on Stanford-ORB the paper claims reconstruction quality on par with or better than dense-view optimization-based inverse rendering methods at a fraction of the runtime.

Significance. If the claims hold, LIRM would be a notable advance toward practical sparse-view relightable 3D reconstruction, combining the speed of large reconstruction models with material decomposition and view-dependent radiance. The paper has considerable strengths: each of the three architectural components is ablated (update stages, hexa-plane versus tri-plane, NDE versus diffuse-only, camera trajectories); the training dataset is large and designed to mimic real capture conditions; the training pipeline avoids the stability issues of differentiable marching cubes; and the authors include an honest limitations section. The main risk is that the central real-data comparison rests on an evaluation protocol that is narrower than the abstract's promise, as detailed in the major comments.

major comments (2)
  1. [Sec. 4, Table 5, Abstract] The headline claim that LIRM 'achieves reconstruction quality on par and even better than state-of-the-art optimization-based methods' from '3 to 6 posed images' is not supported by the real-data experiment as reported. The best LIRM rows in Table 5 use 18 images (three sets of six), not 3-6; the one-set rows are consistently below Neural-PBIR on relighting (e.g., LIRM-NDE 1st PSNR-H 24.25 vs Neural-PBIR 26.01). In addition, the inputs are masked images plus background images, and Sec. 4 states that the authors 'only crop and resize the region of interest based on the object's foreground mask,' so a ground-truth foreground mask is required and LIRM receives an object-centric crop at higher effective resolution than the optimization baselines, which process the full images. This preprocessing asymmetry alone could explain part of the Chamfer-distance improvement (0.31 vs 0.43). The paper reports no experiment without masks or with matched preprocessing, so the practical scope of the central claim is narrower than the abstract states. Please add such experiments or explicitly restrict and rephrase the claim.
  2. [Sec. 4, Tables 1-2; Supplementary Sec. 8] The rows labeled 'MeshLRM [85]' in Tables 1 and 2 are not the official released model (which has not been open-sourced); Supplementary Sec. 8 discloses that the authors trained their own 'LRM-V olSDF' model with the same architecture on the authors' Shutterstock-based dataset. The main text does not disclose this provenance. Because MeshLRM is the primary LRM baseline and the comparison is the basis for the claim that LIRM outperforms the prior state of the art, the main text should state that these numbers come from a re-implementation trained on a different dataset, and the comparison should be framed accordingly. If official numbers become available, they should be used or the discrepancy discussed.
minor comments (6)
  1. [Table 5] The row header 'LIRM-NDE 3nd' should read 'LIRM-NDE 3rd'.
  2. [Sec. 5, Supplementary paragraph] The main-text paragraph beginning 'Supplementary We will include more implementation details...' contains placeholder language ('We will include', 'We will add comparisons'), even though the arXiv supplementary already contains these items; this text should be replaced with a description of the actual supplementary contents.
  3. [Supplementary Sec. 6] The overview states that 'Our supplementary material consists of three parts' but then lists four bullet points; the enumeration should be corrected.
  4. [Supplementary Sec. 7] In the paragraph on testing datasets, the sentence beginning 'esting datasets captured under uniform lighting...' is missing a leading 'T'; fix the typo.
  5. [Tables 1-2 and Supplementary Sec. 8] The naming of the re-implemented baseline is inconsistent: the main text uses 'MeshLRM [85]' while the supplementary calls it 'LRM-V olSDF'; use one name and clearly identify it as a re-implementation.
  6. [Table 5] The 'CD' entry for MetaLRM is listed as '-'; clarify whether geometry was not evaluated for this method or the value is unavailable.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: LIRM's claims are empirical and benchmarked externally; self-citations and mask preprocessing are not definitional reductions.

full rationale

The paper's derivation chain is a trained feed-forward transformer, and the core claims are empirical, not analytic. Shape, materials, and radiance are decoded from hexa-plane and NDE tokens (Eqs. 8-21) and supervised by rendering losses against independent synthetic and real benchmarks (GSO, ABO, Stanford-ORB). No equation defines an output in terms of the quantity it is claimed to predict, and no fitted parameter is renamed as a prediction. The recurrence in Eqs. 5-6 feeds previous output tokens back as input for progressive refinement; this is a standard self-conditioning/refinement design, not a circular definition. The main self-citations (DTC dataset [17] used as a test set, Neural-PBIR [73] used as an optimization baseline) are not load-bearing for the method's derivation; the central comparison on Stanford-ORB is external, and the same authors' prior work is used as a comparator, not as justification for a premise. The mask-crop preprocessing on Stanford-ORB (Sec. 4) is a legitimate external-validity confound — the abstract promises input of 'posed images' while the real-data evaluation additionally uses a foreground mask and up to 18 images — but it is a benchmarking issue, not a circular reduction. The paper itself flags limitations (mirror reflections, texture detail, and pending supplementary comparisons with MeshLRM and BRDF reconstruction), which further shows the claims are not definitionally forced. Verdict: no significant circularity; score 2 reflects only the presence of non-load-bearing self-citations and the unmentioned mask input, not a circular derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim is an empirical system validated by experiments; no new physical entities are introduced. The listed free parameters are hand-chosen hyperparameters (NDE count, SDF bias coefficient, plane resolutions, sharpness schedule) that the reported quality depends on. The axioms are modeling and data assumptions the architecture relies on; the real-data evaluations provide partial independent support, but the synthetic-to-real transfer and the convexity prior remain untested at scale.

free parameters (4)
  • Number of NDE panoramas N = 4
    Selected by ablation (Fig. 5) to balance near-field/occlusion modeling and capacity. The view-dependent radiance quality, part of the central claim, depends on this hand-set choice.
  • SDF bias scale factor in sbias(x) = ||x|| - 0.1R = 0.1
    Hand-chosen in Eq. (12) and stated as important for fast convergence of the SDF; affects geometry and material decomposition.
  • Hexa-plane token resolution = 48x48x6
    Chosen in Sec 3.5 as a cost-quality trade-off for detailed texture recovery; the main claim of high texture fidelity depends on it.
  • SDF sharpness schedule 1/beta = 1e to 2e2 (stage 1), 2e2 to 2.5e2 (stage 2), 2.5e2 (stage 3)
    Designed coarse-to-fine schedule (Tab. 6) that controls surface sharpness; a hand-set training hyperparameter.
assumptions (5)
  • domain assumption The hexa-plane representation assumes the target object is roughly convex and located in the center of the 3D volume.
    Stated in Sec 3.3: 'This representation utilizes the prior that the target object is likely to be roughly convex and located in the center of the 3D volume.' Non-convex or off-center objects may stress the 8-volume decomposition.
  • domain assumption Plücker ray coordinates are sufficient for the transformer to infer camera poses across diverse settings.
    Eq. (1) tokenizes image patches with (v, v x o); the model inherits this generalization mechanism from MeshLRM (Sec 3.1). If Plücker encoding is lossy, the cross-camera claims weaken.
  • domain assumption Synthetic training renders transfer to real-world images.
    The model is trained on 38M images rendered from Shutterstock objects with HDR environments (Sec 4) and tested on real Stanford-ORB/Aria without fine-tuning. The practical claim depends on this transfer.
  • domain assumption SDF-based volume rendering with the density conversion in Eq. (9) is sufficient for joint geometry and material supervision.
    Uses SDF volume ray marching from VolSDF [93] with a beta sharpness parameter. This modeling choice determines how geometry and BRDF maps are optimized together.
  • domain assumption Multiple NDE panoramas with a predicted index can approximate near-field reflections and occlusion.
    Sec 3.4: near-field effects are handled by N>=4 panoramas and a small MLP predicting the query index (Eqs. 19-20). Ablation Fig. 5 supports this, but it remains a learned approximation, not a physical model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields." pith.science (2026). https://pith.science/paper/7JWCRKZX

@misc{pith2026250420026,
  author       = {Pith},
  title        = {Pith review of: LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7JWCRKZX}},
  note         = {Machine review of arXiv:2504.20026}
}
read the original abstract

We present Large Inverse Rendering Model (LIRM), a transformer architecture that jointly reconstructs high-quality shape, materials, and radiance fields with view-dependent effects in less than a second. Our model builds upon the recent Large Reconstruction Models (LRMs) that achieve state-of-the-art sparse-view reconstruction quality. However, existing LRMs struggle to reconstruct unseen parts accurately and cannot recover glossy appearance or generate relightable 3D contents that can be consumed by standard Graphics engines. To address these limitations, we make three key technical contributions to build a more practical multi-view 3D reconstruction framework. First, we introduce an update model that allows us to progressively add more input views to improve our reconstruction. Second, we propose a hexa-plane neural SDF representation to better recover detailed textures, geometry and material parameters. Third, we develop a novel neural directional-embedding mechanism to handle view-dependent effects. Trained on a large-scale shape and material dataset with a tailored coarse-to-fine training scheme, our model achieves compelling results. It compares favorably to optimization-based dense-view inverse rendering methods in terms of geometry and relighting accuracy, while requiring only a fraction of the inference time.

Figures

Figures reproduced from arXiv: 2504.20026 by the authors.

Figure 1
Figure 1. Given small sets of images (e.g., 4 to 8), LIRM progressively reconstructs view-dependent radiance fields, geometry and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The network architecture of LIRM. The inputs are masked images [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Visualization of the initial and updated reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Comparisons between our tri-plane and hexa-plane re [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Comparisons of different strategies to model view [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: LIRM-NDE can generalize to real images to recover [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Inverse rendering results on DTC [17] (row 2 to 4) and ABO [11] (row 1) datasets. Material ground-truth are included in insets. and LIRM-NDE can improve reconstruction quality with more inputs [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Comparisons with prior works on the Stanford-ORB dataset [ [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparisons of geometry reconstruction [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Two limitations of LIRM. (a) LIRM-NDE cannot han [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Reconstructing a changing scene with the LIRM update [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: LIRM reconstruction from images casually captured [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparisons of view synthesis results after [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 15
Figure 15. Figure 15: Comparisons of reconstruction results under different [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 14
Figure 14. Figure 14: Two different camera trajectories to test LIRM’s update [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 16
Figure 16. Figure 16: BRDF prediction results. LIRM better recovers BRDF [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

109 extracted references · 45 canonical work pages

  1. [85]

    Meshlrm: Large reconstruction model for high- quality mesh

    Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, and Zex- iang Xu. Meshlrm: Large reconstruction model for high- quality mesh. arXiv preprint arXiv:2404.12385, 2024. 2, 3, 4, 5, 6, 7, 10, 1

    arXiv 2024

  2. [1]

    https://www.shutterstock.com/ search/3d

    Shutterstock. https://www.shutterstock.com/ search/3d. 6, 2

  3. [2]

    Intrinsic scene prop- erties from a single rgb-d image

    Jonathan T Barron and Jitendra Malik. Intrinsic scene prop- erties from a single rgb-d image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 17–24, 2013. 3

    2013

  4. [3]

    Shape, illumination, and reflectance from shading

    Jonathan T Barron and Jitendra Malik. Shape, illumination, and reflectance from shading. IEEE transactions on pattern analysis and machine intelligence, 37(8):1670–1687, 2014. 3

    2014

  5. [4]

    Nerd: Neural re- flectance decomposition from image collections

    Mark Boss, Raphael Braun, Varun Jampani, Jonathan T Barron, Ce Liu, and Hendrik Lensch. Nerd: Neural re- flectance decomposition from image collections. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 12684–12694, 2021. 1, 2

    2021

  6. [5]

    Neural-pil: Neu- ral pre-integrated lighting for reflectance decomposition

    Mark Boss, Varun Jampani, Raphael Braun, Ce Liu, Jonathan Barron, and Hendrik Lensch. Neural-pil: Neu- ral pre-integrated lighting for reflectance decomposition. Advances in Neural Information Processing Systems , 34: 10691–10704, 2021

    2021

  7. [6]

    Samurai: Shape and material from uncon- strained real-world arbitrary image collections.Advances in Neural Information Processing Systems, 35:26389–26403,

    Mark Boss, Andreas Engelhardt, Abhishek Kar, Yuanzhen Li, Deqing Sun, Jonathan Barron, Hendrik Lensch, and Varun Jampani. Samurai: Shape and material from uncon- strained real-world arbitrary image collections.Advances in Neural Information Processing Systems, 35:26389–26403,

  8. [7]

    Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement

    Mark Boss, Zixuan Huang, Aaryaman Vasishta, and Varun Jampani. Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement. arXiv preprint, 2024. 3

    2024

Show all 109 references
  1. [8]

    Intrinsic image decompo- sition via ordinal shading

    Chris Careaga and Ya ˘gız Aksoy. Intrinsic image decompo- sition via ordinal shading. ACM Transactions on Graphics, 43(1):1–24, 2023. 3

    2023

  2. [9]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021. 4

    2021

  3. [10]

    Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo

    Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In Proceedings of the IEEE/CVF international con- ference on computer vision, pages 14124–14133, 2021. 3

    2021

  4. [11]

    Abo: Dataset and benchmarks for real-world 3d ob- ject understanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achlesh- war Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d ob- ject understanding. In Proceedings of the IEEE/CVF con- ference ...

    2022

  5. [12]

    Flashattention-2: Faster attention with bet- ter parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with bet- ter parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023. 4

    2023 arXiv

  6. [13]

    Acquiring the reflectance field of a human face

    Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. Acquiring the reflectance field of a human face. InProceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 145–156, 2000. 1

    2000

  7. [14]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  8. [15]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Chris- tian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  9. [16]

    Single-image svbrdf cap- ture with a rendering-aware deep network

    Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and Adrien Bousseau. Single-image svbrdf cap- ture with a rendering-aware deep network. ACM Transac- tions on Graphics (ToG), 37(4):1–15, 2018. 3

    2018

  10. [17]

    Digital twin catalog: A large-scale photorealistic 3d object digital twin dataset

    Zhao Dong, Ka Chen, Zhaoyang Lv, Hong-Xing Yu, Yun- zhi Zhang, Cheng Zhang, Yufeng Zhu, Stephen Tian, Zhengqin Li, Geordie Moffatt, Sean Christofferson, James Fort, Xiaqing Pan, Mingfei Yan, Jiajun Wu, Carl Yuheng Ren, and Richard Newcombe. Digital twin catalog: A large-scale ...

    2025

  11. [18]

    Dreamcraft: Text-guided generation of functional 3d environments in minecraft

    Sam Earle, Filippos Kokkinos, Yuhe Nie, Julian Togelius, and Roberta Raileanu. Dreamcraft: Text-guided generation of functional 3d environments in minecraft. In Proceedings of the 19th International Conference on the Foundations of Digital Games, pages 1–15, 2024. 3

    2024

  12. [19]

    Project aria: A new tool for ego- centric multi-modal ai research, 2023

    Jakob Engel, Kiran Somasundaram, Michael Goesele, Al- bert Sun, Alexander Gamino, Andrew Turner, Arjang Ta- lattof, Arnie Yuan, Bilal Souti, Brighid Meredith, Cheng Peng, Chris Sweeney, Cole Wilson, Dan Barnes, Daniel DeTone, David Caruso, Derek Valleroy, Dinesh Ginjupalli, Du...

    2023

  13. [20]

    Shi- nobi: Shape and illumination using neural object decompo- sition via brdf optimization in-the-wild

    Andreas Engelhardt, Amit Raj, Mark Boss, Yunzhi Zhang, Abhishek Kar, Yuanzhen Li, Deqing Sun, Ricardo Martin Brualla, Jonathan T Barron, Hendrik Lensch, et al. Shi- nobi: Shape and illumination using neural object decompo- sition via brdf optimization in-the-wild. In Proceedin...

    2024

  14. [21]

    K- planes: Explicit radiance fields in space, time, and appear- ance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K- planes: Explicit radiance fields in space, time, and appear- ance. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 12479–12488,

  15. [22]

    Kaolin: A pytorch library for accelerating 3d deep learning research

    Clement Fuji Tsang, Maria Shugrina, Jean Francois Lafleche, Towaki Takikawa, Jiehan Wang, Charles Loop, Wenzheng Chen, Krishna Murthy Jatavallabhula, Edward Smith, Artem Rozantsev, Or Perel, Tianchang Shen, Jun Gao, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xiang, Jiani...

    2022

  16. [23]

    Learning to predict in- door illumination from a single image

    Marc-Andr ´e Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagn ´e, and Jean-Franc ¸ois Lalonde. Learning to predict in- door illumination from a single image. arXiv preprint arXiv:1704.00090, 2017. 3

    2017 arXiv

  17. [24]

    Fast spatially-varying in- door lighting estimation

    Mathieu Garon, Kalyan Sunkavalli, Sunil Hadap, Nathan Carr, and Jean-Franc ¸ois Lalonde. Fast spatially-varying in- door lighting estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6908–6917, 2019. 3

    2019

  18. [25]

    Shape, light, and material decomposition from images us- ing monte carlo rendering and denoising

    Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. Shape, light, and material decomposition from images us- ing monte carlo rendering and denoising. Advances in Neural Information Processing Systems, 35:22856–22869,

  19. [26]

    Openlrm: Open-source large reconstruction models

    Zexin He and Tengfei Wang. Openlrm: Open-source large reconstruction models. https://github.com/ 3DTopia/OpenLRM, 2023. 2, 3

    2023

  20. [27]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural informa- tion processing systems, 33:6840–6851, 2020. 1, 3

    2020

  21. [28]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2, 3, 1

    2023 arXiv

  22. [29]

    Putting nerf on a diet: Semantically consistent few-shot view synthesis

    Ajay Jain, Matthew Tancik, and Pieter Abbeel. Putting nerf on a diet: Semantically consistent few-shot view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5885–5894, 2021. 3

    2021

  23. [30]

    Gaussian- shader: 3d gaussian splatting with shading functions for re- flective surfaces

    Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xi- aoxiao Long, Wenping Wang, and Yuexin Ma. Gaussian- shader: 3d gaussian splatting with shading functions for re- flective surfaces. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , p...

    2024

  24. [31]

    Tensoir: Tensorial inverse rendering

    Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Song- fang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 165–174, 2023. 1, 2

    2023

  25. [32]

    Geonerf: Generalizing nerf with geometry priors

    Mohammad Mahdi Johari, Yann Lepoittevin, and Franc ¸ois Fleuret. Geonerf: Generalizing nerf with geometry priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18365–18375, 2022. 3

    2022

  26. [33]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  27. [34]

    Infonerf: Ray entropy minimization for few-shot neural volume ren- dering

    Mijeong Kim, Seonguk Seo, and Bohyung Han. Infonerf: Ray entropy minimization for few-shot neural volume ren- dering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12912– 12921, 2022. 3

    2022

  28. [35]

    Stanford-orb: a real- world 3d object inverse rendering benchmark

    Zhengfei Kuang, Yunzhi Zhang, Hong-Xing Yu, Samir Agarwala, Elliott Wu, Jiajun Wu, et al. Stanford-orb: a real- world 3d object inverse rendering benchmark. Advances in Neural Information Processing Systems, 36, 2024. 1, 2, 3, 8, 9, 11

    2024

  29. [36]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 2, 3, 4, 1

    2023 arXiv

  30. [37]

    Ner- facc: A general nerf acceleration toolbox

    Ruilong Li, Matthew Tancik, and Angjoo Kanazawa. Ner- facc: A general nerf acceleration toolbox. arXiv preprint arXiv:2210.04847, 2022. 6, 1

    2022 arXiv

  31. [38]

    Differentiable monte carlo ray tracing through edge sampling

    Tzu-Mao Li, Miika Aittala, Fr ´edo Durand, and Jaakko Lehtinen. Differentiable monte carlo ray tracing through edge sampling. ACM Transactions on Graphics (TOG), 37 (6):1–11, 2018. 2

    2018

  32. [39]

    Model- ing surface appearance from a single photograph using self- augmented convolutional neural networks

    Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. Model- ing surface appearance from a single photograph using self- augmented convolutional neural networks. ACM Transac- tions on Graphics (ToG), 36(4):1–11, 2017. 3

    2017

  33. [40]

    Cgintrinsics: Better intrinsic image decomposition through physically-based rendering

    Zhengqi Li and Noah Snavely. Cgintrinsics: Better intrinsic image decomposition through physically-based rendering. In Proceedings of the European conference on computer vi- sion (ECCV), pages 371–387, 2018. 3

    2018

  34. [41]

    Materials for masses: Svbrdf acquisition with a single mobile phone image

    Zhengqin Li, Kalyan Sunkavalli, and Manmohan Chan- draker. Materials for masses: Svbrdf acquisition with a single mobile phone image. In Proceedings of the Euro- pean conference on computer vision (ECCV), pages 72–87,

  35. [42]

    Learning to recon- struct shape and spatially-varying reflectance from a single image

    Zhengqin Li, Zexiang Xu, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Learning to recon- struct shape and spatially-varying reflectance from a single image. ACM Transactions on Graphics (TOG), 37(6):1–11,

  36. [43]

    Inverse 12 rendering for complex indoor scenes: Shape, spatially- varying lighting and svbrdf from a single image

    Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Inverse 12 rendering for complex indoor scenes: Shape, spatially- varying lighting and svbrdf from a single image. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2020

  37. [44]

    Openrooms: An end-to-end open framework for photorealistic indoor scene datasets

    Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gun- davarapu, Jia Shi, et al. Openrooms: An end-to-end open framework for photorealistic indoor scene datasets. arXiv preprint arXiv:2007.12868, 2020. 3

    2007 arXiv

  38. [45]

    Physically-based editing of indoor scene lighting from a single image

    Zhengqin Li, Jia Shi, Sai Bi, Rui Zhu, Kalyan Sunkavalli, Miloˇs Ha ˇsan, Zexiang Xu, Ravi Ramamoorthi, and Man- mohan Chandraker. Physically-based editing of indoor scene lighting from a single image. In European Confer- ence on Computer Vision, pages 555–572. Springer, 2022. 3

    2022

  39. [46]

    Neuralangelo: High-fidelity neural surface reconstruction

    Zhaoshuo Li, Thomas M ¨uller, Alex Evans, Russell H Tay- lor, Mathias Unberath, Ming-Yu Liu, and Chen-Hsuan Lin. Neuralangelo: High-fidelity neural surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8456–8465, 2023. 6

    2023

  40. [47]

    Gs-ir: 3d gaussian splatting for inverse rendering

    Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 21644–21653, 2024. 1, 2

    2024

  41. [48]

    Magic3d: High- resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High- resolution text-to-3d content creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2023

  42. [49]

    One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion

    Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Ji- ayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. In Proceedings of the IEEE/CVF Conference on Co...

    2024

  43. [50]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 9298–9309, 2023. 3

    2023

  44. [51]

    Soft rasterizer: A differentiable renderer for image-based 3d rea- soning

    Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. Soft rasterizer: A differentiable renderer for image-based 3d rea- soning. In Proceedings of the IEEE/CVF international con- ference on computer vision, pages 7708–7717, 2019. 2

    2019

  45. [52]

    Syncdreamer: Generating multiview-consistent images from a single-view image

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Generating multiview-consistent images from a single-view image. arXiv preprint arXiv:2309.03453, 2023. 3

    2023 arXiv

  46. [53]

    Reflectance and illumi- nation recovery in the wild

    Stephen Lombardi and Ko Nishino. Reflectance and illumi- nation recovery in the wild. IEEE transactions on pattern analysis and machine intelligence, 38(1):129–141, 2015. 3

    2015

  47. [54]

    Neural volumes: Learning dynamic renderable volumes from im- ages

    Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. Neural volumes: Learning dynamic renderable volumes from im- ages. arXiv preprint arXiv:1906.07751, 2019. 1, 2

    1906 arXiv

  48. [55]

    Sparseneus: Fast generalizable neural sur- face reconstruction from sparse views

    Xiaoxiao Long, Cheng Lin, Peng Wang, Taku Komura, and Wenping Wang. Sparseneus: Fast generalizable neural sur- face reconstruction from sparse views. InEuropean Confer- ence on Computer Vision, pages 210–227. Springer, 2022. 3

    2022

  49. [56]

    Wonder3d: Single image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Single image to 3d using cross-domain diffusion. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  50. [57]

    Image- based brdf measurement including human skin

    Stephen R Marschner, Stephen H Westin, Eric PF Lafor- tune, Kenneth E Torrance, and Donald P Greenberg. Image- based brdf measurement including human skin. In Ren- dering Techniques’ 99: Proceedings of the Eurographics Workshop in Granada, Spain, June 21–23, 1999 10 , pages 13...

    1999

  51. [58]

    A data-driven reflectance model

    Wojciech Matusik. A data-driven reflectance model. PhD thesis, Massachusetts Institute of Technology, 2003. 1

    2003

  52. [59]

    Lime: Live intrinsic material esti- mation

    Abhimitra Meka, Maxim Maximov, Michael Zollhoefer, Avishek Chatterjee, Hans-Peter Seidel, Christian Richardt, and Christian Theobalt. Lime: Live intrinsic material esti- mation. In Proceedings of the IEEE conference on com- puter vision and pattern recognition , pages 6315–6324,

  53. [60]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 1, 2

    2021

  54. [61]

    Instant neural graphics primitives with a multiresolution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022. 1, 2

    2022

  55. [62]

    Extracting triangular 3d models, materials, and lighting from images

    Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas M ¨uller, and Sanja Fidler. Extracting triangular 3d models, materials, and lighting from images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , p...

    2022

  56. [63]

    Regnerf: Regularizing neural radiance fields for view syn- thesis from sparse inputs

    Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan. Regnerf: Regularizing neural radiance fields for view syn- thesis from sparse inputs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , ...

    2022

  57. [64]

    Scalable diffusion mod- els with transformers

    William Peebles and Saining Xie. Scalable diffusion mod- els with transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 4195– 4205, 2023. 3

    2023

  58. [65]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 3

    2022 arXiv

  59. [66]

    Zeronvs: Zero-shot 360-degree view synthesis from a single real im- age

    Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Her- rmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, et al. Zeronvs: Zero-shot 360-degree view synthesis from a single real im- age. arXiv preprint arXiv:2310.17994, 2023. 3 13

    arXiv

  60. [67]

    Zerorf: Fast sparse view 360deg reconstruction with zero pretrain- ing

    Ruoxi Shi, Xinyue Wei, Cheng Wang, and Hao Su. Zerorf: Fast sparse view 360deg reconstruction with zero pretrain- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 21114–21124,

  61. [68]

    Mvdream: Multi-view diffusion for 3d generation

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308.16512, 2023. 3

    2023 arXiv

  62. [69]

    Gir: 3d gaussian inverse ren- dering for relightable scene factorization

    Yahao Shi, Yanmin Wu, Chenming Wu, Xing Liu, Chen Zhao, Haocheng Feng, Jingtuo Liu, Liangjun Zhang, Jian Zhang, Bin Zhou, et al. Gir: 3d gaussian inverse ren- dering for relightable scene factorization. arXiv preprint arXiv:2312.05133, 2023. 1, 2

    2023 arXiv

  63. [70]

    Meta 3d assetgen: Text-to-mesh generation with high- quality geometry, texture, and pbr materials

    Yawar Siddiqui, Tom Monnier, Filippos Kokkinos, Mahen- dra Kariya, Yanir Kleiman, Emilien Garreau, Oran Gafni, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, et al. Meta 3d assetgen: Text-to-mesh generation with high- quality geometry, texture, and pbr materials. arXiv pr...

    2024 arXiv

  64. [71]

    Score- based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score- based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020. 1, 3

    2011 arXiv

  65. [72]

    Lighthouse: Predicting lighting volumes for spatially- coherent illumination

    Pratul P Srinivasan, Ben Mildenhall, Matthew Tancik, Jonathan T Barron, Richard Tucker, and Noah Snavely. Lighthouse: Predicting lighting volumes for spatially- coherent illumination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages ...

    2020

  66. [73]

    Neural-pbir reconstruction of shape, mate- rial, and illumination

    Cheng Sun, Guangyan Cai, Zhengqin Li, Kai Yan, Cheng Zhang, Carl Marshall, Jia-Bin Huang, Shuang Zhao, and Zhao Dong. Neural-pbir reconstruction of shape, mate- rial, and illumination. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 18046– 1...

    2023

  67. [75]

    Mvdiffusion++: A dense high- resolution multi-view diffusion model for single or sparse- view 3d object reconstruction

    Shitao Tang, Jiacheng Chen, Dilin Wang, Chengzhou Tang, Fuyang Zhang, Yuchen Fan, Vikas Chandra, Yasutaka Fu- rukawa, and Rakesh Ranjan. Mvdiffusion++: A dense high- resolution multi-view diffusion model for single or sparse- view 3d object reconstruction. In European Conferen...

    2025

  68. [76]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024. 2, 3

    2024 arXiv

  69. [77]

    Objects with lighting: A real-world dataset for evaluating reconstruc- tion and rendering for object relighting

    Benjamin Ummenhofer, Sanskar Agrawal, Rene Sepul- veda, Yixing Lao, Kai Zhang, Tianhang Cheng, Stephan Richter, Shenlong Wang, and German Ros. Objects with lighting: A real-world dataset for evaluating reconstruc- tion and rendering for object relighting. In 2024 Inter- nation...

    2024

  70. [78]

    Neus: Learning neural implicit surfaces by volume rendering for multi-view recon- struction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view recon- struction. arXiv preprint arXiv:2106.10689, 2021. 5

    2021 arXiv

  71. [79]

    Steindreamer: Variance re- duction for text-to-3d score distillation via stein identity

    Peihao Wang, Zhiwen Fan, Dejia Xu, Dilin Wang, Sreyas Mohan, Forrest Iandola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, et al. Steindreamer: Variance re- duction for text-to-3d score distillation via stein identity. arXiv preprint arXiv:2401.00604, 2023. 3

    2023 arXiv

  72. [80]

    Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction

    Peng Wang, Hao Tan, Sai Bi, Yinghao Xu, Fujun Luan, Kalyan Sunkavalli, Wenping Wang, Zexiang Xu, and Kai Zhang. Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction. arXiv preprint arXiv:2311.12024, 2023. 2, 3

    2023 arXiv

  73. [81]

    Taming mode collapse in score distillation for text-to-3d generation

    Peihao Wang, Dejia Xu, Zhiwen Fan, Dilin Wang, Sreyas Mohan, Forrest Iandola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, et al. Taming mode collapse in score distillation for text-to-3d generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- ...

    2024

  74. [82]

    Ibrnet: Learning multi-view image-based rendering

    Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul P Srinivasan, Howard Zhou, Jonathan T Barron, Ricardo Martin-Brualla, Noah Snavely, and Thomas Funkhouser. Ibrnet: Learning multi-view image-based rendering. In Proceedings of the IEEE/CVF conference on computer vi- sion and pa...

    2021

  75. [83]

    Learning indoor inverse rendering with 3d spatially-varying lighting

    Zian Wang, Jonah Philion, Sanja Fidler, and Jan Kautz. Learning indoor inverse rendering with 3d spatially-varying lighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 12538–12547,

  76. [84]

    Prolificdreamer: High- fidelity and diverse text-to-3d generation with variational score distillation

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongx- uan Li, Hang Su, and Jun Zhu. Prolificdreamer: High- fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Process- ing Systems, 36, 2024. 3

    2024

  77. [86]

    Neural directional encoding for efficient and accurate view-dependent appearance modeling

    Liwen Wu, Sai Bi, Zexiang Xu, Fujun Luan, Kai Zhang, Iliyan Georgiev, Kalyan Sunkavalli, and Ravi Ramamoor- thi. Neural directional encoding for efficient and accurate view-dependent appearance modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2024

  78. [87]

    Hyperdreamer: Hyper-realistic 3d content generation and editing from a single image

    Tong Wu, Zhibing Li, Shuai Yang, Pan Zhang, Xingang Pan, Jiaqi Wang, Dahua Lin, and Ziwei Liu. Hyperdreamer: Hyper-realistic 3d content generation and editing from a single image. In SIGGRAPH Asia 2023 Conference Papers, pages 1–10, 2023. 3

    2023

  79. [88]

    Sparp: Fast 3d object reconstruc- tion and pose estimation from sparse views

    Chao Xu, Ang Li, Linghao Chen, Yulin Liu, Ruoxi Shi, Hao Su, and Minghua Liu. Sparp: Fast 3d object reconstruc- tion and pose estimation from sparse views. arXiv preprint arXiv:2408.10195, 2024. 3 14

    2024 arXiv

  80. [89]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,

    arXiv

  81. [90]

    Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model

    Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Ji- ahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model. arXiv preprint arXiv:2311.09217, 2023. 2, 3

    2023 arXiv

  82. [91]

    Deep image-based relighting from optimal sparse samples

    Zexiang Xu, Kalyan Sunkavalli, Sunil Hadap, and Ravi Ramamoorthi. Deep image-based relighting from optimal sparse samples. ACM Transactions on Graphics (ToG), 37 (4):1–13, 2018. 3

    2018

  83. [92]

    Sire-ir: Inverse rendering for brdf reconstruction with shadow and illumi- nation removal in high-illuminance scenes

    Ziyi Yang, Yanzhen Chen, Xinyu Gao, Yazhen Yuan, Yu Wu, Xiaowei Zhou, and Xiaogang Jin. Sire-ir: Inverse rendering for brdf reconstruction with shadow and illumi- nation removal in high-illuminance scenes. arXiv preprint arXiv:2310.13030, 2023. 1, 2

    arXiv

  84. [93]

    V olume rendering of neural implicit surfaces

    Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. V olume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 34:4805–4815,

  85. [94]

    Plenoctrees for real-time rendering of neural radiance fields

    Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5752– 5761, 2021. 5

    2021

  86. [95]

    pixelnerf: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4578– 4587, 2021. 3

    2021

  87. [96]

    Mvimgnet: A large-scale dataset of multi-view images

    Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, et al. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF conference on computer vision and pat- tern recogn...

    2023

  88. [97]

    A dif- ferential theory of radiative transfer

    Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. A dif- ferential theory of radiative transfer. ACM Transactions on Graphics (TOG), 38(6):1–16, 2019. 2

    2019

  89. [98]

    Physg: Inverse rendering with spherical gaussians for physics-based material editing and relighting

    Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, and Noah Snavely. Physg: Inverse rendering with spherical gaussians for physics-based material editing and relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5453–5462, 2021. 1, 2

    2021

  90. [99]

    Arf: Artistic radiance fields

    Kai Zhang, Nick Kolkin, Sai Bi, Fujun Luan, Zexiang Xu, Eli Shechtman, and Noah Snavely. Arf: Artistic radiance fields. In European Conference on Computer Vision, pages 717–733. Springer, 2022. 1

    2022

  91. [100]

    Iron: Inverse rendering by optimizing neural sdfs and ma- terials from photometric images

    Kai Zhang, Fujun Luan, Zhengqi Li, and Noah Snavely. Iron: Inverse rendering by optimizing neural sdfs and ma- terials from photometric images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5565–5574, 2022. 1, 2

    2022

  92. [101]

    Gs-lrm: Large reconstruction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large reconstruction model for 3d gaussian splatting. InEuropean Conference on Computer Vision , pages 1–19. Springer,

  93. [102]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6

    2018

  94. [103]

    Relitlrm: Generative relightable radiance for large reconstruction models

    Tianyuan Zhang, Zhengfei Kuang, Haian Jin, Zexiang Xu, Sai Bi, Hao Tan, He Zhang, Yiwei Hu, Milos Hasan, William T Freeman, et al. Relitlrm: Generative relightable radiance for large reconstruction models. arXiv preprint arXiv:2410.06231, 2024. 3

    arXiv

  95. [104]

    Ner- factor: Neural factorization of shape and reflectance under an unknown illumination

    Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul Debevec, William T Freeman, and Jonathan T Barron. Ner- factor: Neural factorization of shape and reflectance under an unknown illumination. ACM Transactions on Graphics (ToG), 40(6):1–18, 2021. 1, 2

    2021

  96. [105]

    Modeling indirect il- lumination for inverse rendering

    Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia, and Xiaowei Zhou. Modeling indirect il- lumination for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18643–18652, 2022. 8

    2022

  97. [106]

    Nemf: Inverse volume rendering with neural microflake field

    Youjia Zhang, Teng Xu, Junqing Yu, Yuteng Ye, Yanqing Jing, Junle Wang, Jingyi Yu, and Wei Yang. Nemf: Inverse volume rendering with neural microflake field. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 22919–22929, 2023. 1, 2 15 LIRM: ...

    2023

  98. [107]

    • Implementation details, including multi-stage training, accelerated deferred rendering, training and testing datasets creation

    Overview Our supplementary material consists of three parts. • Implementation details, including multi-stage training, accelerated deferred rendering, training and testing datasets creation. • Ablation studies on synthetic dataset, including impacts of multi-stage coarse-to-fi...

  99. [108]

    We first train with large batch sizes but small res- olutions for fast convergence, and later train with high res- olutions but small batch sizes for better details

    Implementation details Coarse-to-fine training Our training consists of three stages. We first train with large batch sizes but small res- olutions for fast convergence, and later train with high res- olutions but small batch sizes for better details. The hyper- parameters for...

  100. [109]

    is used in all prior volume-based LRM methods [28, 36, 85] to reduce GPU memory consumption. The basic idea is to cache the gradients so that we can render an im- age patch-by-patch while still computing a perceptual loss like LPIPS on the whole image, which is essential for r...

  101. [110]

    We run the experiments on the GSO dataset rendered with uni- form lighting

    Experiments on Synthetic Data Impacts of coarse-to-fine training We test the network’s reconstruction quality after different stages of training. We run the experiments on the GSO dataset rendered with uni- form lighting. The quantitative results are summarized in Tab. 7. We r...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.