Pith. sign in

REVIEW 4 major objections 6 minor 46 references

CrossModalityDiffusion: Multi-Modal Novel View Synthesis with Unified Intermediate Representation

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Joint training with random modality pairs makes a single 3D feature volume serve every imaging sensor, enabling cross-modal novel view synthesis without scene geometry.

desk verdict A clean modular extension of GeNVS to multi-modal NVS, but the central claim of a unified modality-agnostic representation is asserted rather than demonstrated. read the letter →

arxiv 2501.09838 v1 pith:5K62OOLQ submitted 2025-01-16 cs.CV cs.AIeess.IV

Alex Berian , Daniel Brignac , JhihYang Wu , Natnael Daba , Abhijit Mahalanobis This is my paper · ORCID
classification cs.CVcs.AIeess.IV
keywords multi-modalnovelviewsynthesiscross-modalimagegenerationneuralradiancefieldsdiffusionmodelsfeaturevolumesjointtrainingmodality-agnosticrepresentationgeospatialimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CrossModalityDiffusion aims to synthesize a novel image of a scene in any target sensor modality—electro-optical (EO), synthetic aperture radar (SAR), or two LiDAR variants—from a handful of input images that may come from any of those sensors, with no prior knowledge of scene geometry. The paper's central claim is that by jointly training the modality-specific encoders and diffusion denoisers while randomly sampling input and target modalities, the encoders learn a unified, modality-agnostic intermediate representation: all input modalities map into one shared 3D feature volume space, and any denoiser can decode a rendered feature image into its own modality. This would matter because it removes the need for per-modality retraining or explicit geometric calibration in multi-modal novel view synthesis, and it lets information from different sensors be fused into a single representation. The framework is validated on the synthetic ShapeNet cars dataset, where it generates consistent novel views across all modality pairs and improves as more input images from varied sensors are fused.

What carries the argument

The central object is the shared feature volume space. Each input image is processed by a modality-specific encoder (initialized from a pretrained EO GeNVS encoder, a prior single-modality novel-view-synthesis architecture) into a geometry-aware feature volume oriented in its camera's frustum; all volumes are overlapped and averaged along each ray, then a single shared MLP turns the averaged features into color and density, and volume rendering produces a 16-channel feature image from the target viewpoint. That feature image conditions a modality-specific diffusion U-Net denoiser that synthesizes the output image. The key training mechanism is joint training: randomly selecting one to three input images from random views and modalities and a random target view and modality forces the encoders to align their feature volumes in the common space without any cross-modal supervision. For range-angle LiDAR, the feature image is also rendered in range-angle coordinates so non-perspective sensors can be handled.

What would settle it

Render feature images for the same scene and novel pose from EO-only and LiDAR-only inputs after training: if the representation is truly unified, the two feature images should be near-identical; if the encoders learned modality-specific shortcuts, the feature images will visibly diverge and cross-modal outputs will degrade. The same test can be run on real multi-modal captures to see whether the alignment transfers outside synthetic cars.

Watch

Extended reading notes

Core claim

The core discovery is that a single implicitly learned 3D feature volume can serve as a common ground for all imaging modalities, and that this unification can be achieved purely by joint training with random input and target modality samples. Each modality-specific encoder turns its input images into geometry-aware feature volumes placed in the camera frustum; a shared MLP and volumetric renderer produce feature images from new poses; and a modality-specific diffusion denoiser generates the final image. Because training randomly mixes modalities, the encoders are incentivized to produce the same feature field for the same scene regardless of sensor, yielding what the paper calls a modality-agnostic intermediate representation. On the ShapeNet cars dataset, the framework demonstrates any-to-any cross-modal novel view synthesis—e.g., EO to SAR, LiDAR to EO—and shows that fusing multiple input images, even from different sensors at the same viewpoint, improves output quality.

Load-bearing premise

The load-bearing premise is that jointly training with randomly sampled input and target modalities—without any cross-modal supervision, geometric calibration, or explicit consistency loss—is enough to make feature volumes from very different sensors align into one coherent 3D space, a premise tested only on synthetic ShapeNet cars.

Editorial extensions

If this is right

  • Any combination of trained input and output modalities can be used at test time: an EO input can drive a SAR output, and a LiDAR input can drive an EO output, with no retraining per pair.
  • Adding more input images, even from different sensors at the same viewpoint, monotonically improves novel-view quality because the model fuses geometric evidence in the shared volume.
  • The framework can be extended to non-perspective sensor images, such as range-angle LiDAR, by rendering the feature image in that coordinate frame.
  • A new modality can be added by training a new encoder/denoiser module initialized from the EO pretrained model, keeping the shared MLP and volume space fixed.
  • The unified representation is not limited to view synthesis and could support other downstream multi-modal tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the unified representation holds beyond synthetic data, the same feature volumes could be plugged into downstream tasks such as detection or segmentation, as the paper suggests, but this remains untested.
  • The joint-training alignment hypothesis could be tested directly: for a fixed scene and novel pose, the feature images rendered from different input modalities should be nearly identical if the representation is truly modality-agnostic; if they diverge, the encoders have learned modality shortcuts.
  • Real sensor data adds noise, misalignment, and different resolution; the assumption that random modality sampling suffices without a geometric consistency loss is likely to fail on real captures, so an explicit alignment regularizer would be a natural extension.
  • The improvement from fusing multiple sensors at the same viewpoint suggests the model is combining complementary information, not just averaging; a concrete follow-up is to measure how much each sensor contributes to the final novel view.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CrossModalityDiffusion, a modular extension of GeNVS for multi-modal novel view synthesis (MMNVS). The framework uses modality-specific encoders to produce geometry-aware feature volumes from input images of different modalities (EO, SAR, LiDAR in range-angle and perspective forms), a shared MLP that fuses overlapping volumes, and a modality-specific diffusion denoiser to synthesize the target image. The authors claim that joint training with randomly sampled input and target modalities yields a unified, modality-agnostic intermediate representation that can be decoded into any output modality. They validate the approach on a synthetic ShapeNet-cars dataset rendered in four modalities, reporting single-modality in/out results (Table 1), fusion improvements with more input views (Table 2), and combination of different sensors from the same viewpoint (Table 3), along with qualitative examples in Figures 1 and 4.

Significance. If the central claim of a unified, modality-agnostic intermediate representation holds, the modular framework would be a useful step toward cross-modal novel view synthesis for geospatial imaging, where real multi-sensor data is heterogeneous and ground-truth geometry is often unavailable. The paper has several strengths: the code is publicly linked; the dataset generation pipeline combines existing renderers (BLAINDER, RaySAR) with SRN-Cars; the architecture cleanly separates modality-specific adapters from a shared 3D representation; and the experiments cover many input-output modality combinations, including non-perspective LiDAR range-angle images. However, the significance is currently bounded by the absence of baselines, the lack of error bars or significance testing, the single-object synthetic evaluation, and the lack of any direct measurement of feature-volume alignment across modalities. The paper therefore presents a promising system with an unvalidated central claim rather than a demonstrated method.

major comments (4)
  1. [§6, Tables 1–3] The experiments evaluate no baseline methods. Without comparisons to, for example, a per-modality GeNVS model, an image-retrieval baseline, or a simple copy-from-nearest-view baseline, the reported absolute metrics (FID, LPIPS, DISTS, PSNR, SSIM) cannot establish that CrossModalityDiffusion is effective for MMNVS or that the modular design provides any benefit over monolithic alternatives.
  2. [§6.3, Table 2] The fusion improvements are small and likely within noise. For example, at S=2 the FID changes from 29.99 to 27.47 (an 8.4% relative improvement) and PSNR from 16.40 to 16.83 (a 2.7% relative improvement), yet no error bars, confidence intervals, or significance tests are reported, and the authors themselves note variance in the 'Separate' row. The claim that more input images improve output quality is therefore not statistically supported.
  3. [§5.3] The central claim of a unified, modality-agnostic intermediate representation is not directly tested. The training objective is only the final image reconstruction loss; there is no explicit loss that aligns or compares feature volumes produced from different input modalities. The paper provides no quantitative measurement of feature-volume or feature-image consistency across modalities at the same novel viewpoint, so joint training is shown to produce usable cross-modal outputs, but the stronger claim of a shared representation remains an interpretation without direct evidence.
  4. [§5.1 and §6] Evaluation is limited to a single synthetic object class (ShapeNet cars) where all modalities are rendered from the same underlying 3D models with known poses. The stated motivation is real geospatial EO/LiDAR/SAR data, but the paper provides no experiments on real data, no held-out category, and no test of whether the learned representation generalizes beyond the training distribution. This leaves open the possibility that the model exploits category-specific appearance shortcuts rather than a sensor-agnostic geometric representation.
minor comments (6)
  1. [§5.3] The phrase "modality agonistic" appears to be a typo and should read "modality-agnostic."
  2. [§6.1, Table 1] The table header appears garbled: "In Modality Out ModalityFID↓" should be split into separate columns with proper spacing, e.g., "Input Modality | Output Modality | FID↓ ...".
  3. [§6.3, Table 2] The caption text "S= 2 input viewsSeparate" is missing a space; also, the distinction between "Separate" (one random source image) and "Fused" (S images) should be stated more clearly in the caption.
  4. [§4] The phrase "encodes each source image xsi image into feature volumes" contains a duplicated word; it should be "encodes each source image xsi into feature volumes."
  5. [§5.1 and References] The simulator is spelled "BLAINDER" in the text but "Blainder" in the reference list [30]; please use consistent spelling.
  6. [§6.2] The modification for range-angle feature-image rendering is described but not evaluated in isolation; a small ablation showing the quantitative effect of this rendering choice would strengthen the claim that the framework handles non-perspective projections.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the shared-representation claim is an empirical architecture hypothesis evaluated on held-out synthetic views, not a derivation equivalent to its inputs.

full rationale

The paper contains no derivation chain that reduces to its own inputs. The central claim—that joint training of modality-specific encoders with a shared MLP yields a modality-agnostic intermediate representation—is an empirical architecture claim supported by held-out test results on ShapeNet cars. The shared feature-volume space is a design construction, not a fitted parameter renamed as a prediction. The reported numbers are actual outputs of a trained model on a test split, not quantities forced by the training objective. There are no self-citations, no imported uniqueness theorems, and no ansatz smuggled in through prior work by the same authors. The most relevant concern is that the paper does not directly measure feature-volume alignment across input modalities and evaluates only on a single synthetic object class, so the modality-agnostic claim may be under-validated; however, an unvalidated empirical premise is a correctness or evidence limitation, not circularity. Under the stated rules, no circular step can be quoted and exhibited, so the appropriate score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

This is an empirical deep learning paper, so the ledger mostly records standard methods and domain assumptions. The central claim does not follow from an analytic derivation; it is a learned behavior. Most 'free parameters' are inherited from GeNVS or chosen by hand, and the main risk is the unvalidated assumption that synthetic renders are representative of real sensor data.

free parameters (3)
  • feature volume latent dimension = 16
    Inherited from GeNVS; no ablation or justification for this choice.
  • number of random input images during training = 1-3
    Chosen by hand; no sensitivity analysis is provided.
  • znear to zfar sampling range for volume rendering = dataset-specific
    Mentioned as dataset-specific in GeNVS, but the exact values for the ShapeNet cars are not reported.
assumptions (4)
  • standard math NeRF volume rendering equations (weighted sum of sampled colors and densities)
    Used to render feature images from feature volumes, taken from NeRF [25].
  • standard math Diffusion model denoising process (iterative refinement of latent or pixel space)
    The output generation uses a diffusion U-Net following GeNVS and related diffusion literature.
  • domain assumption BLAINDER and RaySAR renderers produce realistic LiDAR and SAR images from ShapeNet CAD models
    The entire cross-modal evaluation rests on the validity of these synthetic renders as proxies for real sensor data; no validation against real EO, LiDAR, or SAR imagery is provided.
  • domain assumption The SRN-Cars dataset provides accurate camera poses and a representative distribution of car shapes
    Used without further geometric verification; errors here would propagate to the rendered LiDAR and SAR views.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CrossModalityDiffusion: Multi-Modal Novel View Synthesis with Unified Intermediate Representation." pith.science (2026). https://pith.science/paper/5K62OOLQ

@misc{pith2026250109838,
  author       = {Pith},
  title        = {Pith review of: CrossModalityDiffusion: Multi-Modal Novel View Synthesis with Unified Intermediate Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5K62OOLQ}},
  note         = {Machine review of arXiv:2501.09838}
}
read the original abstract

Geospatial imaging leverages data from diverse sensing modalities-such as EO, SAR, and LiDAR, ranging from ground-level drones to satellite views. These heterogeneous inputs offer significant opportunities for scene understanding but present challenges in interpreting geometry accurately, particularly in the absence of precise ground truth data. To address this, we propose CrossModalityDiffusion, a modular framework designed to generate images across different modalities and viewpoints without prior knowledge of scene geometry. CrossModalityDiffusion employs modality-specific encoders that take multiple input images and produce geometry-aware feature volumes that encode scene structure relative to their input camera positions. The space where the feature volumes are placed acts as a common ground for unifying input modalities. These feature volumes are overlapped and rendered into feature images from novel perspectives using volumetric rendering techniques. The rendered feature images are used as conditioning inputs for a modality-specific diffusion model, enabling the synthesis of novel images for the desired output modality. In this paper, we show that jointly training different modules ensures consistent geometric understanding across all modalities within the framework. We validate CrossModalityDiffusion's capabilities on the synthetic ShapeNet cars dataset, demonstrating its effectiveness in generating accurate and consistent novel views across multiple imaging modalities and perspectives.

Figures

Figures reproduced from arXiv: 2501.09838 by the authors.

Figure 1
Figure 1. CrossModalityDiffusion input/output examples. Three examples of CrossModalityDiffusion with two input im￾ages, and an output from a different modality. From the top down: EO to SAR, LiDAR(RA) to LiDAR(P), LiDAR(P) to EO. The cor￾responding intermediate feature image is shown in the middle, and the ground truth target image is shown on the right. ical features of the Earth’s landscape, making them indis￾pensable for … view at source ↗
Figure 2
Figure 2. Dataset Generation. We begin with the SRN-Cars dataset for EO images and corresponding camera pose matrices. We then use the pose matrix and ShapeNet object file for gener￾ating LiDAR(RA) and LiDAR(P) images with BLAINDER, and SAR images with RaySAR images of it from different viewpoints. From each view￾point, we need to use different sensors to capture the scene (EO, LiDAR, SAR, etc). In our experiments, we use the… view at source ↗
Figure 3
Figure 3. Architecture of CrossModalityDiffusion. 5.3. Joint Training To allow our framework to input multiple images from different modalities at the same time then output a novel view of the scene in any modality, we jointly train the en￾coder and denoiser modules for each modality at the same time. During training, we randomly select between one and three input images from random views and modalities, as well as one random… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Demonstration of any modality to any modality through unified intermediate representation. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 21 canonical work pages

  1. [1]

    Raysar- 3d sar simulator: Now open source

    Stefan Auer, Richard Bamler, and Peter Reinartz. Raysar- 3d sar simulator: Now open source. In 2016 IEEE In- ternational Geoscience and Remote Sensing Symposium (IGARSS), pages 6730–6733. IEEE, 2016. 3, 4

    work page 2016

  2. [2]

    Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 5855–5864,

  3. [3]

    Zip-nerf: Anti-aliased grid-based neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19697–19705, 2023. 2

    2023

  4. [4]

    Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement

    Mark Boss, Zixuan Huang, Aaryaman Vasishta, and Varun Jampani. Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement. arXiv preprint, 2024. 2

    work page 2024

  5. [5]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 2

    2022

  6. [6]

    Generative novel view synthesis with 3d-aware diffusion models

    Eric R Chan, Koki Nagano, Matthew A Chan, Alexander W Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. Generative novel view synthesis with 3d-aware diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4217–4229, 2023. 1, 2, 3

    work page 2023

  7. [7]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 2

    arXiv 2015

  8. [8]

    Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Deeplab: Semantic im- age segmentation with deep convolutional nets, atrous con- volution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence , 40(4):834–848,

Show all 46 references
  1. [9]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818, 2018. 3

    2018

  2. [10]

    Big data geospatial processing for massive aerial lidar datasets

    David Deibe, Margarita Amor, and Ram ´on Doallo. Big data geospatial processing for massive aerial lidar datasets. Re- mote Sensing, 12(4), 2020. 1

    2020

  3. [11]

    Simoncelli

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. Image quality assessment: Unifying structure and texture similarity. CoRR, abs/2004.07728, 2020. 4

    2004 arXiv

  4. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  5. [13]

    Sur la distance de deux lois de probabilit´e

    Maurice Fr ´echet. Sur la distance de deux lois de probabilit´e. In Annales de l’ISUP, volume 6, pages 183–198, 1957. 4

    1957

  6. [14]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 12479–12488, 2023. 2

    2023

  7. [15]

    Cat3d: Create anything in 3d with multi-view diffusion models

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405.10314, 2024. 2

    2024 arXiv

  8. [16]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019. 1

    2019

  9. [17]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 2

    2020

  10. [18]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2

    2023 arXiv

  11. [19]

    Segmentation models pytorch

    Pavel Iakubovskii. Segmentation models pytorch. https: //github.com/qubvel/segmentation_models. pytorch, 2019. 3

    2019

  12. [20]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 1, 2, 3

    2022

  13. [21]

    Analyzing and improving the training dynamics of diffusion models

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24174–24184, 2024. 2

    2024

  14. [22]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4401–4410, 2019. 2

    2019

  15. [23]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  16. [24]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 2

    2023

  17. [25]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 1, 2, 3 8

    2021

  18. [26]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR,

  19. [27]

    The terrasar-x satel- lite

    Wolfgang Pitz and David Miller. The terrasar-x satel- lite. IEEE Transactions on Geoscience and Remote Sensing, 48(2):615–622, 2010. 1

    2010

  20. [28]

    Barron, and Ben Milden- hall

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv,

  21. [29]

    Neural implicit surface reconstruction using imaging sonar

    Mohamad Qadri, Michael Kaess, and Ioannis Gkioulekas. Neural implicit surface reconstruction using imaging sonar. In 2023 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 1040–1047. IEEE, 2023. 2

    2023

  22. [30]

    Blainder—a blender ai add-on for generation of semantically labeled depth-sensing data

    Stefan Reitmann, Lorenzo Neumann, and Bernhard Jung. Blainder—a blender ai add-on for generation of semantically labeled depth-sensing data. Sensors, 21(6):2144, 2021. 3, 4

    2021

  23. [31]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  24. [32]

    Zero123++: a single image to consistent multi-view dif- fusion base model, 2023

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model, 2023. 2

    2023

  25. [33]

    Mvdream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. arXiv preprint arXiv:2308.16512, 2023. 2

    2023 arXiv

  26. [34]

    Scene representation networks: Continuous 3d- structure-aware neural scene representations

    Vincent Sitzmann, Michael Zollh ¨ofer, and Gordon Wet- zstein. Scene representation networks: Continuous 3d- structure-aware neural scene representations. Advances in Neural Information Processing Systems, 32, 2019. 3, 4

    2019

  27. [35]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 2

    2015

  28. [36]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 2

    2011 arXiv

  29. [37]

    Learned initializations for optimizing coordinate-based neural representations

    Matthew Tancik, Ben Mildenhall, Terrance Wang, Divi Schmidt, Pratul P Srinivasan, Jonathan T Barron, and Ren Ng. Learned initializations for optimizing coordinate-based neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2021

  30. [38]

    Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation. arXiv preprint arXiv:2309.16653,

    arXiv

  31. [39]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024. 2

    2024 arXiv

  32. [40]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Il- lia Polosukhin. Attention is all you need. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett, editors,Advances in N...

    2017

  33. [41]

    Barron, and Pratul P

    Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T. Barron, and Pratul P. Srinivasan. Ref-NeRF: Structured view-dependent appearance for neural radiance fields. CVPR, 2022. 2

    2022

  34. [42]

    Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. NeurIPS, 2021. 2

    2021

  35. [43]

    Reconfusion: 3d reconstruction with diffusion priors

    Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P Srinivasan, Dor Verbin, Jonathan T Barron, Ben Poole, et al. Reconfusion: 3d reconstruction with diffusion priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  36. [44]

    pixelnerf: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4578–4587, 2021. 2

    2021

  37. [45]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 4

    2018

  38. [46]

    Sparsefusion: Dis- tilling view-conditioned diffusion for 3d reconstruction

    Zhizhuo Zhou and Shubham Tulsiani. Sparsefusion: Dis- tilling view-conditioned diffusion for 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 12588–12597, 2023. 2 9

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.