pith. sign in

arxiv: 2605.31535 · v1 · pith:T7OKCLLSnew · submitted 2026-05-29 · 💻 cs.CV · cs.AI· cs.LG

RayDer: Scalable Self-Supervised Novel View Synthesis from Real-World Video

Ulrich Prestel , Stefan Andreas Baumann , Nick Stracke , Bj\"orn Ommer This is my paper

Pith reviewed 2026-06-28 22:48 UTC · model grok-4.3

classification 💻 cs.CV cs.AIcs.LG
keywords novel view synthesisself-supervised learningtransformer modelvideo datascaling laws3D scene reconstructioncamera pose estimation
0
0 comments X

The pith

RayDer consolidates camera estimation, reconstruction and rendering into one transformer for scalable self-supervised novel view synthesis from video.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents RayDer as a single feed-forward transformer that unifies camera estimation, scene reconstruction, and rendering for self-supervised novel view synthesis. By treating time-varying content as a minimal dynamic nuisance state, it enables stable training on real-world videos while focusing on static scenes. This design turns NVS into a single-model scaling problem that exhibits clean power-law behavior with data and compute. It achieves zero-shot performance competitive with supervised methods on various benchmarks. Readers would care because it shows how abundant video data can drive high-quality view synthesis without supervision or brittle multi-network setups.

Core claim

RayDer is a unified feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering into a single backbone. A minimal dynamic state treated as a nuisance factor absorbs time-varying content, enabling stable training on unconstrained real-world video while keeping static-scene NVS as the target task. The model exhibits clean power-law scaling with data and compute and outperforms static-scene data mixtures, achieving strong zero-shot open-set performance competitive with state-of-the-art supervised approaches.

What carries the argument

Unified feed-forward transformer backbone integrating camera estimation, scene reconstruction and rendering, with minimal dynamic state as nuisance factor for handling video dynamics.

If this is right

  • RayDer exhibits clean power-law scaling with increasing data and compute.
  • It outperforms training on static-scene data mixtures alone.
  • It achieves competitive zero-shot open-set performance with supervised SOTA on multiple benchmarks.
  • Training on unconstrained real-world video becomes stable for static NVS.
  • Self-supervised NVS becomes a well-posed single-model scaling problem.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the scaling holds, larger models trained on more video could surpass current supervised methods without labels.
  • The nuisance state approach might apply to other tasks where dynamics are not the focus but available in data.
  • Consolidating multiple components into one model could simplify other 3D vision pipelines.

Load-bearing premise

The minimal dynamic state sufficiently absorbs time-varying content to enable stable training on real-world video without compromising the static scene NVS objective.

What would settle it

Observing that training becomes unstable or scaling breaks when the dynamic state is removed on real-world video datasets would falsify the central claim.

Figures

Figures reproduced from arXiv: 2605.31535 by Bj\"orn Ommer, Nick Stracke, Stefan Andreas Baumann, Ulrich Prestel.

Figure 1
Figure 1. Figure 1: Training Static-scene Novel View Synthesis from Abundant Video. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: NVS performance across sections, training on general video (here, SA-B). Our goal is to make self-supervised novel view synthesis (NVS) scalable in data, model size, and compute, without introducing task-specific supervision or brittle system design. Starting from a modern feed-forward baseline (§3.1), we identify three bottlenecks that prevent scaling: §3.2 Data: existing methods assume static scenes for … view at source ↗
Figure 3
Figure 3. Figure 3: Preliminaries: RayZer [28]. RayZer uses three models responsible for different tasks: a) Camera Estimation, b) Reconstruc￾tion, c) Rendering. We start our exploration with RayZer [28], a feed-forward NVS method trained in a self-supervised manner on unposed, uncalibrated videos of static scenes with camera motion. Extending upon LVSM [29], RayZer consists of three distinct ViT [10] subnetworks ( [PITH_FUL… view at source ↗
Figure 4
Figure 4. Figure 4: Training RayZer directly on dynamic videos leads to instabilities and stalled training. Scaling self-supervised NVS faces an immediate data bottleneck: truly static￾scene videos, as required by current methods [28, 46, 68, 73], are a tiny subset of what is available at scale. However, training RayZer directly on dynamic video leads to gradient spikes and instabilities: the original RayZer [28] diverges con… view at source ↗
Figure 5
Figure 5. Figure 5: Consolidation. We combine RayZer’s three networks (a) into one (b). Single-Network Consolidation (Config D). To reduce scaling decisions to a single network, which can allocate capacity between tasks as needed, and improve performance by sharing features, we unify all three components – camera/dynamic state estimation, scene reconstruction, and rendering (see [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Our at￾tention mask. Parallel-target Attention (Config E). Naively treating the consolidated model as decoder-only [29] reprocesses input views for each target view, which is prohibitively expensive. We factorize attention such that input tokens only attend to each other, while target tokens attend to themselves and input tokens (see [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Many input views (a) allow encoding camera poses via an implicit “time” axis; sparse views (b) require true relative camera poses. Autoregressive Pose Learning (Config F, G). When training on video frames, many input views make pose prediction easy to solve by using frame-order shortcuts rather than actual geometry (Fig. 7a). We find that in practice, this results in predicted poses primarily encoding time… view at source ↗
Figure 8
Figure 8. Figure 8: Final Architecture Overview. RayDer unifies camera estimation (a) and novel view synthesis (b) in a single transformer backbone. Lightweight local intra-frame encoder and decoder layers handle high-resolution processing. train-test gap, since standard NVS settings do not condition on and generate frames in temporal order. Randomizing the autoregression order instead (CONFIG G) closes this gap and further i… view at source ↗
Figure 9
Figure 9. Figure 9: Zero-shot qualitative samples of RayDer compared with E-RayZer [89] in (a) typical (non-dense view) NVS settings, (b) an extreme setting with ∼zero context view overlap, and (c) settings evaluated in Tab. 5. Our RayDer model, trained on large-scale non-static-constrained video data, outperforms E-RayZer – a prior model trained on a multi static dataset mixture – by a wide margin. 26 28 PSNR (dB,↑) 1% Data … view at source ↗
Figure 10
Figure 10. Figure 10: Scaling Across Data and Model Size. We evaluate models trained on SpatialVid (2.7M total samples) at different model scales (visualized as shades of green) and dataset fractions (shades of blue), on RE-10k [92]. Left: Increasing data scale consistently improves performance, as long as model scale is not a limit. At small data scales, large models tend to overfit, resulting in worse performance than smalle… view at source ↗
Figure 11
Figure 11. Figure 11: Compute-Optimal Scaling Analysis. RayDer’s compute-optimal performance (i.e., the compute-quality Pareto frontier) on unseen datasets (here, RE10K [92]) across both compute and train dataset size is well-approximated by a single power law. Model Scale Model Scale Data Scale Data Scale Ground Truth Reference [PITH_FULL_IMAGE:figures/full_fig_p008_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative Scaling. RayDer’s qualitative behavior follows the trends seen in quantitative evals ( [PITH_FULL_IMAGE:figures/full_fig_p008_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Learned Camera Geometry Scales with Data, Model Size, and Compute. [PITH_FULL_IMAGE:figures/full_fig_p010_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Limitations. Both main failure modes arise from the regression objective collapsing under-constrained content to a low-frequency average, dashed boxes mark affected regions. (a) content unseen in any input view is rendered as a blurry mean estimate. (b) in presence of dynamic content, the static scene is rendered correctly from the novel pose; moving content is averaged. to a mixture of blur and loose int… view at source ↗
read the original abstract

Self-supervised novel view synthesis (NVS) remains challenging to scale, despite the abundance of video data, largely due to the brittleness of training on realistic videos and the hard-to-predict scaling behavior of multi-network system designs. We introduce RayDer, a unified, feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering into a single backbone, turning self-supervised NVS into a well-posed single-model scaling problem. A minimal dynamic state, treated as a nuisance factor, absorbs time-varying content and enables stable training on unconstrained real-world video. Importantly, RayDer keeps static-scene NVS as its target task: dynamic content is leveraged purely as scalable supervision, not reconstructed as in dynamic-scene (4D) NVS. Across multiple model sizes and orders of magnitude in data, RayDer exhibits clean power-law scaling with data and compute, and outperforms static-scene data mixtures. On a large number of benchmarks, RayDer achieves strong zero-shot open-set performance competitive with state-of-the-art supervised approaches. Project Page: https://compvis.github.io/rayder

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces RayDer, a unified feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering into a single backbone for self-supervised novel view synthesis (NVS) from real-world video. A minimal dynamic state is treated as a nuisance factor to absorb time-varying content, enabling stable training while keeping the target strictly static-scene NVS (dynamics used only as scalable supervision, not reconstructed). The model exhibits clean power-law scaling with data and compute across sizes, outperforms static-scene data mixtures, and achieves strong zero-shot open-set performance competitive with supervised SOTA on multiple benchmarks.

Significance. If the scaling behavior and benchmark results hold under the stated assumptions, the work would be significant for scaling self-supervised 3D vision: it reframes NVS as a single-model empirical scaling problem rather than a brittle multi-network design, potentially allowing better leverage of abundant unlabeled video while avoiding the full complexity of 4D dynamic reconstruction.

major comments (2)
  1. [Abstract] Abstract: the central claim that a minimal dynamic state suffices to absorb all time-varying content (non-rigid motion, lighting variation, partial occlusions) without destabilizing static-scene NVS training or causing the backbone to allocate capacity to dynamic reconstruction is load-bearing, yet the abstract supplies no parameterization, capacity, or regularization details for this state, leaving the assumption unanchored and the stability claim unevaluable.
  2. [Abstract] Abstract: the assertion of 'clean power-law scaling with data and compute' across model sizes and orders of magnitude in data is presented as a key empirical result, but no quantitative details (model sizes, data volumes, fitted exponents, or goodness-of-fit metrics) are supplied, making it impossible to assess whether the scaling is genuinely parameter-free or merely consistent with prior scaling literature.
minor comments (1)
  1. The project page URL is a useful addition for readers seeking implementation details or visualizations.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and will revise the abstract to improve self-containment while preserving the manuscript's focus.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that a minimal dynamic state suffices to absorb all time-varying content (non-rigid motion, lighting variation, partial occlusions) without destabilizing static-scene NVS training or causing the backbone to allocate capacity to dynamic reconstruction is load-bearing, yet the abstract supplies no parameterization, capacity, or regularization details for this state, leaving the assumption unanchored and the stability claim unevaluable.

    Authors: We agree the abstract would benefit from brief anchoring details. The parameterization (32-dimensional per-frame latent with explicit L2 regularization to enforce minimality and prevent capacity allocation to dynamics) is fully specified in Section 3.2 and Appendix B. We will revise the abstract to note the state as a low-capacity nuisance factor under L2 regularization, making the claim evaluable without expanding length substantially. revision: yes

  2. Referee: [Abstract] Abstract: the assertion of 'clean power-law scaling with data and compute' across model sizes and orders of magnitude in data is presented as a key empirical result, but no quantitative details (model sizes, data volumes, fitted exponents, or goodness-of-fit metrics) are supplied, making it impossible to assess whether the scaling is genuinely parameter-free or merely consistent with prior scaling literature.

    Authors: The quantitative details (model sizes 10M–1B parameters, data volumes up to 10^6 video hours, fitted exponents ~0.35 for data and ~0.25 for compute, R^2 > 0.95) appear in Section 4.3 and Figure 3. We acknowledge the abstract is overly terse. We will revise it to include a concise reference to the observed scaling ranges and exponents. revision: yes

Circularity Check

0 steps flagged

No circularity; empirical scaling claims with no derivation chain

full rationale

The paper presents RayDer as a unified feed-forward transformer consolidating camera estimation, reconstruction and rendering, with a minimal dynamic state treated as nuisance to enable training on real video while targeting static-scene NVS. All performance claims (zero-shot competitiveness, power-law scaling with data/compute) are stated as empirical observations across model sizes and benchmarks. No equations, uniqueness theorems, fitted parameters renamed as predictions, or self-citation load-bearing steps appear in the provided text. The design choice of the dynamic state is presented as an architectural decision rather than a derived result, and no reduction of any claim to its own inputs by construction is identifiable. The work is therefore self-contained against external benchmarks with no detectable circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract supplies no equations, training details, or architectural specifics, preventing identification of free parameters, axioms, or invented entities.

pith-pipeline@v0.9.1-grok · 5738 in / 1060 out tokens · 24180 ms · 2026-06-28T22:48:14.944340+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

95 extracted references · 29 canonical work pages · 14 internal anchors

  1. [1]

    [Accessed 09-11-2025]

    YouTube for Press — blog.youtube.https://blog.youtube/press/. [Accessed 09-11-2025]

    2025

  2. [2]

    Mip-nerf 360: Unbounded anti- aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti- aliased neural radiance fields. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5470–5479, 2022

    2022

  3. [3]

    ImageHash: A python perceptual image hashing module — github.com

    Johannes Buchner. ImageHash: A python perceptual image hashing module — github.com. https://github.com/ JohannesBuchner/imagehash, 2025

    2025

  4. [4]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19457–19467, 2024

    2024

  5. [5]

    Dbarf: Deep bundle-adjusting generalizable neural radiance fields

    Yu Chen and Gim Hee Lee. Dbarf: Deep bundle-adjusting generalizable neural radiance fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24–34, 2023

    2023

  6. [6]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. InEuropean Conference on Computer Vision, pages 370–386. Springer, 2024

    2024

  7. [7]

    Mvsplat360: Feed-forward 360 scene synthesis from sparse views.Advances in Neural Information Processing Systems, 37:107064–107086, 2024

    Yuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang, Andrea Vedaldi, Tat-Jen Cham, and Jianfei Cai. Mvsplat360: Feed-forward 360 scene synthesis from sparse views.Advances in Neural Information Processing Systems, 37:107064–107086, 2024

    2024

  8. [8]

    Scalable high-resolution pixel-space image synthesis with hourglass diffusion transformers

    Katherine Crowson, Stefan Andreas Baumann, Alex Birch, Tanishq Mathew Abraham, Daniel Z Kaplan, and Enrico Shippole. Scalable high-resolution pixel-space image synthesis with hourglass diffusion transformers. InProceedings of the 41st International Conference on Machine Learning, pages 9550–9575. PMLR, 2024

    2024

  9. [9]

    VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences

    Kai Deng, Zexin Ti, Jiawei Xu, Jian Yang, and Jin Xie. Vggt-long: Chunk it, loop it, align it–pushing vggt’s limits on kilometer-scale long rgb sequences.arXiv preprint arXiv:2507.16443, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on Learning Representations, 2021

    2021

  11. [11]

    Novel view synthesis with pixel-space diffusion models

    Noam Elata, Bahjat Kawar, Yaron Ostrovsky-Berman, Miriam Farber, and Ron Sokolovsky. Novel view synthesis with pixel-space diffusion models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 26756–26766, 2025

    2025

  12. [12]

    IncVGGT: Incremental VGGT for memory-bounded long-range 3d reconstruction

    Keyu Fang, Changchun Zhou, Yuzhe Fu, Hai Helen Li, and Yiran Chen. IncVGGT: Incremental VGGT for memory-bounded long-range 3d reconstruction. InThe Fourteenth International Conference on Learning Representations, 2026

    2026

  13. [13]

    Quantized visual geometry grounded transformer.arXiv preprint arXiv:2509.21302, 2025

    Weilun Feng, Haotong Qin, Mingqiang Wu, Chuanguang Yang, Yuqi Li, Xiangqi Li, Zhulin An, Libo Huang, Yulun Zhang, Michele Magno, et al. Quantized visual geometry grounded transformer.arXiv preprint arXiv:2509.21302, 2025

    work page arXiv 2025

  14. [14]

    Colmap-free 3d gaussian splatting

    Yang Fu, Sifei Liu, Amey Kulkarni, Jan Kautz, Alexei A Efros, and Xiaolong Wang. Colmap-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20796–20805, 2024

    2024

  15. [15]

    Monocular dynamic view synthesis: A reality check

    Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthesis: A reality check. InNeurIPS, 2022

    2022

  16. [16]

    Scaling laws for neural machine translation

    Behrooz Ghorbani, Orhan Firat, Markus Freitag, Ankur Bapna, Maxim Krikun, Xavier Garcia, Ciprian Chelba, and Colin Cherry. Scaling laws for neural machine translation. InInternational Conference on Learning Representations, 2022

    2022

  17. [17]

    Neighborhood attention transformer

    Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi. Neighborhood attention transformer. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6185–6194, 2023

    2023

  18. [18]

    Scaling Laws for Autoregressive Generative Modeling

    Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling.arXiv preprint arXiv:2010.14701, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  19. [19]

    Improving neural networks by preventing co-adaptation of feature detectors

    Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors.arXiv preprint arXiv:1207.0580, 2012. 13

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  20. [20]

    An empirical analysis of compute-optimal large language model training

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Oriol Vinyals, Jack William Rae, and Laur...

    2022

  21. [21]

    arXiv preprint arXiv:2410.22128 (2024)

    Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jisang Han, Jiaolong Yang, Chong Luo, and Seungryong Kim. Pf3plat: Pose-free feed-forward 3d gaussian splatting.arXiv preprint arXiv:2410.22128, 2024

    work page arXiv 2024

  22. [22]

    LRM: Large Reconstruction Model for Single Image to 3D

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d.arXiv preprint arXiv:2311.04400, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  23. [23]

    MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  24. [24]

    No pose at all: Self-supervised pose-free 3d gaussian splatting from sparse views

    Ranran Huang and Krystian Mikolajczyk. No pose at all: Self-supervised pose-free 3d gaussian splatting from sparse views. arXiv preprint arXiv:2508.01171, 2025

    work page arXiv 2025

  25. [25]

    Spfsplatv2: Efficient self-supervised pose-free 3d gaussian splatting from sparse views.arXiv preprint arXiv:2509.17246, 2025

    Ranran Huang and Krystian Mikolajczyk. Spfsplatv2: Efficient self-supervised pose-free 3d gaussian splatting from sparse views.arXiv preprint arXiv:2509.17246, 2025

    work page arXiv 2025

  26. [26]

    Arbitrary style transfer in real-time with adaptive instance normalization

    Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. InProceedings of the IEEE international conference on computer vision, pages 1501–1510, 2017

    2017

  27. [27]

    Large scale multi-view stereopsis evaluation

    Rasmus Jensen, Anders Dahl, George V ogiatzis, Engil Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 406–413. IEEE, 2014

    2014

  28. [28]

    Rayzer: A self-supervised large view synthesis model

    Hanwen Jiang, Hao Tan, Peng Wang, Haian Jin, Yue Zhao, Sai Bi, Kai Zhang, Fujun Luan, Kalyan Sunkavalli, Qixing Huang, and Georgios Pavlakos. Rayzer: A self-supervised large view synthesis model. 2025

    2025

  29. [29]

    Lvsm: A large view synthesis model with minimal 3d inductive bias

    Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  30. [30]

    Selfsplat: Pose-free and 3d prior-free generalizable 3d gaussian splatting

    Gyeongjin Kang, Jisang Yoo, Jihyeon Park, Seungtae Nam, Hyeonsoo Im, Sangheon Shin, Sangpil Kim, and Eunbyung Park. Selfsplat: Pose-free and 3d prior-free generalizable 3d gaussian splatting. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 22012–22022, 2025

    2025

  31. [31]

    Scaling Laws for Neural Language Models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  32. [32]

    MapAnything: Universal Feed-Forward Metric 3D Reconstruction

    Nikhil Keetha, Norman Müller, Johannes Schönberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weber, Nelson Antunes, et al. Mapanything: Universal feed-forward metric 3d reconstruction.arXiv preprint arXiv:2509.13414, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

    2023

  34. [34]

    pHash: The open source perceptual hash library.https://www.phash.org/, 2010

    Evan Klinger and David Starkweather. pHash: The open source perceptual hash library.https://www.phash.org/, 2010

    2010

  35. [35]

    Tanks and temples: Benchmarking large-scale scene reconstruction.ACM Transactions on Graphics, 36(4), 2017

    Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction.ACM Transactions on Graphics, 36(4), 2017

    2017

  36. [36]

    Video autoencoder: self-supervised disentanglement of static 3d structure and motion

    Zihang Lai, Sifei Liu, Alexei A Efros, and Xiaolong Wang. Video autoencoder: self-supervised disentanglement of static 3d structure and motion. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9730–9740, 2021

    2021

  37. [37]

    Grounding image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. InEuropean Conference on Computer Vision, pages 71–91. Springer, 2024

    2024

  38. [38]

    Vicasplat: A single run is all you need for 3d gaussian splatting and camera estimation from unposed video frames.arXiv preprint arXiv:2503.10286, 2025

    Zhiqi Li, Chengrui Dong, Yiming Chen, Zhangchi Huang, and Peidong Liu. Vicasplat: A single run is all you need for 3d gaussian splatting and camera estimation from unposed video frames.arXiv preprint arXiv:2503.10286, 2025

    work page arXiv 2025

  39. [39]

    Megasam: Accurate, fast and robust structure and motion from casual dynamic videos

    Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast and robust structure and motion from casual dynamic videos. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 10486–10496, 2025. 14

    2025

  40. [40]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22160–22169, 2024

    2024

  41. [41]

    Scaling Sequence-to-Sequence Generative Neural Rendering

    Shikun Liu, Kam Woh Ng, Wonbong Jang, Jiadong Guo, Junlin Han, Haozhe Liu, Yiannis Douratsos, Juan C Pérez, Zijian Zhou, Chi Phung, et al. Scaling sequence-to-sequence generative neural rendering.arXiv preprint arXiv:2510.04236, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  42. [42]

    Zhang, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, and David Novotny

    Xingchen Liu, Piyush Tayal, Jianyuan Wang, Jesus Zarzar, Tom Monnier, Konstantinos Tertikas, Jiali Duan, Antoine Toisoul, Jason Y . Zhang, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, and David Novotny. Uncommon objects in 3d. In arXiv, 2024

    2024

  43. [43]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Represen- tations, 2019

    2019

  44. [44]

    Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar

    Ben Mildenhall, Pratul P. Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines.ACM Transactions on Graphics (TOG), 2019

    2019

  45. [45]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. InEuropean Conference on Computer Vision, pages 405–421. Springer, 2020

    2020

  46. [46]

    arXiv preprint arXiv:2510.13063 (2025)

    Thomas W Mitchel, Hyunwoo Ryu, and Vincent Sitzmann. True self-supervised novel view synthesis is transferable.arXiv preprint arXiv:2510.13063, 2025

    work page arXiv 2025

  47. [47]

    Scaling transformer-based novel view synthesis with models token disentanglement and synthetic data

    Nithin Gopalakrishnan Nair, Srinivas Kaza, Xuan Luo, Vishal M Patel, Stephen Lombardi, and Jungyeon Park. Scaling transformer-based novel view synthesis with models token disentanglement and synthetic data. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 28567–28576, 2025

    2025

  48. [48]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  49. [49]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 724–732, 2016

    2016

  50. [50]

    Julius Plucker. Xvii. on a new geometry of space.Philosophical Transactions of the Royal Society of London, (155):725–791, 1865

  51. [51]

    SAM 2: Segment Anything in Images and Videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  52. [52]

    Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. InInternational Conference on Computer Vision, 2021

    2021

  53. [53]

    Geometry-free view synthesis: Transformers and no 3d priors

    Robin Rombach, Patrick Esser, and Björn Ommer. Geometry-free view synthesis: Transformers and no 3d priors. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 14356–14366, 2021

    2021

  54. [54]

    Aleksandr Safin, Daniel Duckworth, and Mehdi S. M. Sajjadi. Repast: Relative pose attention scene representation transformer. 2023

    2023

  55. [55]

    Object scene representation transformer.Advances in neural information processing systems, 35:9512–9524, 2022

    Mehdi SM Sajjadi, Daniel Duckworth, Aravindh Mahendran, Sjoerd Van Steenkiste, Filip Pavetic, Mario Lucic, Leonidas J Guibas, Klaus Greff, and Thomas Kipf. Object scene representation transformer.Advances in neural information processing systems, 35:9512–9524, 2022

    2022

  56. [56]

    Scene representation transformer: Geometry-free novel view synthesis through set-latent scene representations

    Mehdi SM Sajjadi, Henning Meyer, Etienne Pot, Urs Bergmann, Klaus Greff, Noha Radwan, Suhani V ora, Mario Luˇci´c, Daniel Duckworth, Alexey Dosovitskiy, et al. Scene representation transformer: Geometry-free novel view synthesis through set-latent scene representations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2022

  57. [57]

    Rust: Latent neural scene representations from unposed imagery

    Mehdi SM Sajjadi, Aravindh Mahendran, Thomas Kipf, Etienne Pot, Daniel Duckworth, Mario Lu ˇci´c, and Klaus Greff. Rust: Latent neural scene representations from unposed imagery. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17297–17306, 2023. 15

    2023

  58. [58]

    Structure-from-motion revisited

    Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. InConference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  59. [59]

    Maximilian Seitzer, Sjoerd van Steenkiste, Thomas Kipf, Klaus Greff, and Mehdi S. M. Sajjadi. DyST: Towards dynamic neural scene representations on real-world videos. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  60. [60]

    FastVGGT: Training-Free Acceleration of Visual Geometry Transformer

    You Shen, Zhipeng Zhang, Yansong Qu, and Liujuan Cao. Fastvggt: Training-free acceleration of visual geometry transformer. arXiv preprint arXiv:2509.02560, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  61. [61]

    Oriane Siméoni, Huy V . V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julie...

    2025

  62. [62]

    Flowcam: Training generalizable 3d radiance fields without camera poses via pixel-aligned scene flow.arXiv preprint arXiv:2306.00180, 2023

    Cameron Smith, Yilun Du, Ayush Tewari, and Vincent Sitzmann. Flowcam: Training generalizable 3d radiance fields without camera poses via pixel-aligned scene flow.arXiv preprint arXiv:2306.00180, 2023

    work page arXiv 2023

  63. [63]

    RoFormer: Enhanced Transformer with Rotary Position Embedding

    Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: enhanced transformer with rotary position embedding. corr abs/2104.09864 (2021).arXiv preprint arXiv:2104.09864, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  64. [64]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Harts...

    2023

  65. [65]

    The double sphere camera model

    Vladyslav Usenko, Nikolaus Demmel, and Daniel Cremers. The double sphere camera model. In2018 International Conference on 3D Vision (3DV), pages 552–560. IEEE, 2018

    2018

  66. [66]

    Scaling laws in patchification: An image is worth 50,176 tokens and more.arXiv preprint arXiv:2502.03738, 2025

    Feng Wang, Yaodong Yu, Guoyizhe Wei, Wei Shao, Yuyin Zhou, Alan Yuille, and Cihang Xie. Scaling laws in patchification: An image is worth 50,176 tokens and more.arXiv preprint arXiv:2502.03738, 2025

    work page arXiv 2025

  67. [67]

    RayZer: A Self-supervised Large View Synthesis Model

    Haoru Wang. Open-Rayzer: a open-source Self-Reimplemented Version of the paper "RayZer: A Self-supervised Large View Synthesis Model" — github.com.https://github.com/ou524u/Open-Rayzer, 2025

    2025

  68. [68]

    The Less You Depend, The More You Learn: Synthesizing Novel Views from Sparse, Unposed Images with Minimal 3D Knowledge

    Haoru Wang, Kai Ye, Yangyan Li, Wenzheng Chen, and Baoquan Chen. The less you depend, the more you learn: Synthesizing novel views from sparse, unposed images without any 3d knowledge.arXiv preprint arXiv:2506.09885, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  69. [69]

    Vggsfm: Visual geometry grounded deep structure from motion

    Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. 2023

    2023

  70. [70]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025

    2025

  71. [71]

    Spatialvid: A large-scale video dataset with spatial annotations

    Jiahao Wang, Yufeng Yuan, Rujie Zheng, Youtian Lin, Jian Gao, Lin-Zhuo Chen, Yajie Bao, Yi Zhang, Chang Zeng, Yanxi Zhou, et al. Spatialvid: A large-scale video dataset with spatial annotations.arXiv preprint arXiv:2509.09676, 2025

    work page arXiv 2025

  72. [72]

    Continuous 3d perception model with persistent state

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 10510–10522, 2025

    2025

  73. [73]

    Recollection from pensieve: Novel view synthesis via learning from uncalibrated videos.arXiv preprint arXiv:2505.13440, 2025

    Ruoyu Wang, Yi Ma, and Shenghua Gao. Recollection from pensieve: Novel view synthesis via learning from uncalibrated videos.arXiv preprint arXiv:2505.13440, 2025

    work page arXiv 2025

  74. [74]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697–20709, 2024

    2024

  75. [75]

    Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He.π 3: Scalable permutation-equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025. 16

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  76. [76]

    Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004

    2004

  77. [77]

    Novel view synthesis with diffusion models,

    Daniel Watson, William Chan, Ricardo Martin-Brualla, Jonathan Ho, Andrea Tagliasacchi, and Mohammad Norouzi. Novel view synthesis with diffusion models.arXiv preprint arXiv:2210.04628, 2022

    work page arXiv 2022

  78. [78]

    Controlling space and time with diffusion models.arXiv preprint arXiv:2407.07860, 2024

    Daniel Watson, Saurabh Saxena, Lala Li, Andrea Tagliasacchi, and David J Fleet. Controlling space and time with diffusion models.arXiv preprint arXiv:2407.07860, 2024

    work page arXiv 2024

  79. [79]

    Reconfusion: 3d reconstruction with diffusion priors

    Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P Srinivasan, Dor Verbin, Jonathan T Barron, Ben Poole, et al. Reconfusion: 3d reconstruction with diffusion priors. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21551–21561, 2024

    2024

  80. [80]

    Cat4d: Create anything in 4d with multi-view video diffusion models

    Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T Barron, and Aleksander Holynski. Cat4d: Create anything in 4d with multi-view video diffusion models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 26057–26068, 2025

    2025

Showing first 80 references.