Pith. sign in

REVIEW 3 major objections 5 minor 79 references

The paper claims that splitting single-image view synthesis into panorama generation followed by keyframe-anchored video interpolation keeps views globally consistent along long or looped camera paths.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Single-image novel view synthesis is decomposed into panorama outpainting plus keyframe-conditioned video diffusion, producing loop-consistent scene tours.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A genuinely different two-stage pipeline for single-image 360° exploration, with plausible loop-consistency wins in the qualitative results, but the quantitative superiority claim is undercut by mismatched training protocols and non-standard FVD. the 3 major comments →

arxiv 2509.00843 v1 pith:WDXEUDMI submitted 2025-08-31 cs.CV cs.AI

Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion

Xueyang Kang , Zhengkang Xiang , Zezheng Zhang , Kourosh Khoshelham This is my paper
classification cs.CV cs.AI
keywords novel view synthesispanorama diffusionvideo diffusioncamera controlloop closurePlücker raymapkeyframe interpolationdiffusion transformer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to solve a hard version of novel view synthesis: generating views along long, user-defined camera paths — including paths that loop back to their starting point — from a single input photograph. Its proposal is to decompose the problem into two stages. A panorama diffusion model first extends the input view into a complete 360° scene; then perspective keyframes cut from that panorama, plus depth-warped walk-in versions of them, anchor a video diffusion model that generates all the in-between frames. The reason to care: long-range and loop-closure consistency is exactly what current single-image methods lose, and the paper reports that its decomposition keeps both scene and view alignment stable as the path grows, while enabling flexible camera control. If the claim holds, exploring a scene from one snapshot — turning, walking in, and returning — becomes a tractable generation task.

Core claim

On the paper's own terms, the discovery is a factorization. The joint distribution over an entire trajectory of novel views — an ill-posed object when only one input image exists — is decomposed into a keyframe distribution followed by a view-interpolation distribution. The keyframes come from a globally consistent 360° panorama produced by a Diffusion Transformer outpainting model with CLIP conditioning and a 90°-rotation cycle-consistency loss; additional walk-in keyframes are created by depth-based warping. A video diffusion model then interpolates all frames between anchors, conditioned on Plücker raymaps that encode the camera poses and on a spatial weighting that ties each frame to the

What carries the argument

The load-bearing object is the panoramic scene prior — a 360° equirectangular image produced by a Diffusion Transformer outpainting model whose sampling enforces a 90°-rotation cycle-consistency loss so the panorama stays coherent across its seam. The panorama converts a long-range generation problem into a local interpolation problem: perspective keyframes are cut from it by equirectangular projection, with extra walk-in keyframes produced by depth-based warping, and a pre-trained video diffusion model generates all frames between anchors, receiving the camera trajectory through Plücker raymap embeddings. The spatial noise-diffusion step computes a normalized weight for each interpolated fr

Load-bearing premise

Stage 1's 360° panorama is accurate and seamless enough that keyframes cut and warped from it stay correctly aligned with the real scene — if the panorama hallucinates or misaligns anything in regions the input photo never showed, Stage 2 inherits and amplifies those errors.

What would settle it

Run the full pipeline on a loop that ends at the input camera pose and compare the closing frame with the actual input view using the paper's own mTSED criterion (epipolar error < 2.5 px, at least 10 matched features); if the closing frame consistently falls below its reported match rates on scenes with heavy occlusion or extreme lighting, the loop-consistency claim fails. A sharper Stage-1-only test: extract two overlapping perspective keyframes from antipodal regions of the generated panorama — regions the input never saw — and verify they agree in the shared area; visible disagreement there

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Any trajectory that can be written as a sequence of camera poses — rotation, walk-in, or a mix — can be rendered from one input image, because the second stage only interpolates between panorama-derived anchors.
  • Loop closure stops being a long-range memory problem: since anchors span the full 360° panorama, a path that returns to its start is anchored at both ends, which is why the paper reports stable FVD and mTSED as frame count grows.
  • Scene hallucination is amortized: the global scene is fixed once in the panorama, so many different trajectories through the same room can be generated from that single prior without re-hallucinating the environment.
  • The design scales in principle by concatenation — outpainting a new panorama at the current frontier and generating the next video segment — which the authors state as a route to an indefinitely navigable environment.
  • On both evaluated datasets the method reports the best combined perceptual, temporal, and geometric-consistency scores, and its per-100-frame generation time is competitive with the fastest baseline it is compared against.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A generated panorama is a reusable asset: the paper trains only the interpolation conditioning, so a single Stage-1 panorama could anchor many trajectories; the authors do not quantify this reuse, but it follows directly from decoupling the scene prior from the path.
  • Stage-1 reliability is the predictable failure mode: regions of the panorama the input never observed are where hallucination lives, and a confidence map over the panorama would predict where Stage-2 drift will occur — a diagnostic the paper does not provide but that its two-stage design makes easy to build.
  • The pose-proximity weighting in the spatial diffusion step is effectively attention over anchors; it could be repurposed for goal-conditioned navigation by treating future poses as anchors, without retraining the video backbone.
  • The walk-in keyframes depend on monocular depth; swapping in metric depth for a scene should measurably change alignment quality, offering a direct way to test how much of the consistency gain comes from warping accuracy versus from the video model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a two-stage generative pipeline for single-image novel view synthesis. Stage 1 outpainting a 360-degree panorama from the input via a Diffusion Transformer with CLIP conditioning and cycle consistency; Stage 2 samples perspective and walk-in keyframes from the panorama, encodes camera trajectories as Plücker raymaps, and uses a video diffusion model with a proposed spatial noise weighting to generate long, potentially looped view sequences. Experiments are reported on Matterport3D and RealEstate10K against ViewCrafter, VistaDream, and PhotoNVS, with quantitative metrics including mTSED and FVD.

Significance. The two-stage decomposition is a plausible and interesting way to impart a global scene prior to long-range view synthesis. If validated, the work would be a useful contribution, especially for loop-closure trajectories where local video diffusion tends to drift. The paper includes ablations for panorama components and video-diffusion conditioning, and it makes the implementation publicly available, which aids reproducibility. I do not see the factorized distribution in Eq. (1) as circular; it is a modeling choice. However, the headline quantitative claims are not yet established because the proposed video-diffusion stage is fine-tuned on the same datasets used for evaluation while the baselines are not, and the FVD metric is non-standard and reported without error bars.

major comments (3)
  1. [§4.2 vs. §4.1] The method is fine-tuned on 5,000 RealEstate10K walk-through clips and 11,000 Matterport3D episodes (§4.2), while the evaluation in §4.1 uses test episodes from those same datasets. The baselines (ViewCrafter, VistaDream, PhotoNVS) are used as released pre-trained models with no reported fine-tuning on these datasets. Consequently, the large margins in Table 2 (e.g., mTSED 0.726 vs. 0.549 on Matterport3D, FVD 113.6 vs. 198.3 on RealEstate10K) are confounded by domain adaptation and additional training data/compute rather than being attributable to the proposed architecture. Please either fine-tune all baselines under a matched protocol, or evaluate all methods zero-shot on held-out scene distributions, and explicitly report train/test split separation.
  2. [§4.1 (FVD)] FVD is computed by 'stacking normalized ResNet50 feature embeddings' instead of the standard I3D features. This non-standard variant is not directly comparable to FVD numbers in prior work. More importantly, no error bars, confidence intervals, or significance tests are reported for any metric in Table 2. The reported differences, particularly in FVD, could be within run-to-run noise. Please report standard I3D-based FVD and provide variance estimates (e.g., across multiple generation runs or bootstrap resampling) for the central metrics.
  3. [§3.1–§3.2 and Table 5] The two-stage design relies on the generated panorama being an accurate geometric and photometric prior. The manuscript does not quantify how Stage-1 errors propagate into Stage-2 consistency, which is a load-bearing assumption. The ablation in Table 5 varies the number/type of keyframes, but it does not compare conditioning on the generated panorama versus the ground-truth panorama, nor does it corrupt the panorama to measure robustness. Adding an oracle experiment (ground-truth panorama conditioning) and a degraded-panorama experiment would directly test the central assumption and separate the benefit of the panorama prior from the benefit of more conditioning views.
minor comments (5)
  1. [§4.2] There are two nearly identical paragraphs describing the DiT architecture, and the second one has missing values: 'configured as , with input images resized to .' and 'rotation interval ... is set to .' Please merge and fill in the numbers.
  2. [§3.1, Eqs. (2)–(5)] The notation around forward and reverse processes is confusing: Eq. (2) writes p_theta(z_{t-1}|z_{t-2}) for 'known' regions and Eq. (3) writes q(z_t|z_{t-1}) for 'unknown' regions, while the text calls p_theta 'forward diffusion' and q 'reverse denoising.' This is the opposite of the standard convention. Please clarify the notation and align it with the DDPM update in Eq. (5).
  3. [Figures 5 and 6] The caption notes that the first generated frame in each row does not directly correspond to the input of each method. This makes the qualitative comparison hard to interpret. Please annotate the actual input used by each baseline and ensure the starting frames are aligned across methods.
  4. [Appendix 8.2] The user study was 'conducted during the rebuttal phase.' This is unusual for a published paper and the protocol lacks details such as participant recruitment, whether the evaluators were blind to method identity, and any statistical significance test. Please either remove the user study or report it with a standard protocol.
  5. [§4.3] The text says WonderJourney results 'are not included in the paper,' yet Figure 24 in the appendix shows WonderJourney sample results. Please reconcile this inconsistency.

Circularity Check

0 steps flagged

No significant circularity: the two-stage factorization is a modeling choice and the reported gains are supported by component-wise ablations, not by equations that reduce to their inputs.

full rationale

The derivation chain does not reduce to its inputs. Eq. (1) defines the target conditional distribution p_theta(x_i | x0, R_i, T_i, K_i). The paper then explicitly factorizes this into a panorama generation stage p(X0 | x0) (Sec. 3.1) followed by keyframe extraction/warping (Sec. 3.2) and video diffusion conditioned on those keyframes and raymaps (Sec. 3.3-3.4). This is an architectural decomposition, not an identity: the panorama is generated, not taken from the target, and the video model is trained with its own weights. Keyframe extraction is deterministic equirectangular projection (Eq. 6) and warping (Eq. 8), not a fit to the evaluation metrics. Ablations in Tables 3-5 isolate the contributions of CLIP conditioning, cycle loss, raymap/spatial weighting, and the number of panorama/walk-in keyframes, so the central claims have independent content. The self-citations [20] and [52] appear in related-work and an appendix remark about the cycle loss being usable in a simulator; neither is load-bearing for the method or the comparisons. The skeptic's concern about fine-tuning on the evaluation datasets is a controlled-comparison/protocol issue, not a circular-derivation issue, and is therefore outside the circularity score.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The method rests on standard diffusion and projection machinery, plus several hand-set parameters and the strong assumption that the Stage-1 panorama is a reliable global scene prior. No new physical entities are introduced.

free parameters (6)
  • Position temperature tau_T = 4.7 m (optimal on RealEstate10K subset)
    Controls Gaussian weighting of positional proximity in Eq. (9). Tuned via grid search (grid size 0.1) on a RealEstate10K subset, reported in Appendix Figure 20.
  • Orientation temperature tau_q = 1.68 rad (optimal on RealEstate10K subset)
    Controls Gaussian weighting of orientation proximity in Eq. (10). Tuned jointly with tau_T on the same subset.
  • Walk-in ratio = 0.8
    Ratio of walking distance to scene depth; set by hand in Sec 4.2 to determine the forward-motion offset keyframes.
  • Keyframe angular step = 30 to 60 degrees
    Hand-set interval for selecting neighboring perspective views from the panorama (Sec 3.2).
  • Overlap ratio of perspective views = one-third or no overlap
    Hand-set in Sec 4.2 to control the overlap between extracted keyframes.
  • Cycle consistency rotation step = 90 degrees
    Fixed rotation interval for the panorama cycle loss during sampling (Sec 4.2).
axioms (5)
  • domain assumption The scene is static and the camera intrinsics K_i are constant across the sequence.
    Stated in Sec 3 and in the conclusion: 'we currently model static scenes'. The method does not handle dynamic objects or changing intrinsics.
  • domain assumption The panorama generated in Stage 1 is a faithful and globally consistent representation of the scene.
    Core design assumption: all Stage 2 keyframes are extracted from this panorama, so any hallucination or seam error propagates. Invoked throughout Sec 3.1 and 3.2.
  • domain assumption Monocular depth estimates from Depth Anything V2 are accurate enough for keyframe warping.
    Used for RealEstate10K keyframe warping (Sec 4.2). Warping errors directly affect the inpainting masks and spatial conditioning.
  • ad hoc to paper The joint distribution in Eq. (1) can be factorized into a keyframe distribution and an interpolation factor.
    The paper states the decomposition but does not derive it; it is a modeling assumption that enables the two-stage pipeline.
  • standard math Standard DDPM/SVD diffusion math (noise scheduling, reverse denoising, quaternion interpolation) is correct and applicable.
    Relies on existing diffusion formulations (Eq. 5, Algorithm 1, SLERP in the appendix) without proving them.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion." pith.science (2026). https://pith.science/paper/WDXEUDMI

@misc{pith2026250900843,
  author       = {Pith},
  title        = {Pith review of: Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WDXEUDMI}},
  note         = {Machine review of arXiv:2509.00843}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Novel view synthesis (NVS) from a single image is highly ill-posed due to large unobserved regions, especially for views that deviate significantly from the input. While existing methods focus on consistency between the source and generated views, they often fail to maintain coherence and correct view alignment across long-range or looped trajectories. We propose a model that addresses this by decomposing single-view NVS into a 360-degree scene extrapolation followed by novel view interpolation. This design ensures long-term view and scene consistency by conditioning on keyframes extracted and warped from a generated panoramic representation. In the first stage, a panorama diffusion model learns the scene prior from the input perspective image. Perspective keyframes are then sampled and warped from the panorama and used as anchor frames in a pre-trained video diffusion model, which generates novel views through a proposed spatial noise diffusion process. Compared to prior work, our method produces globally consistent novel views -- even in loop closure scenarios -- while enabling flexible camera control. Experiments on diverse scene datasets demonstrate that our approach outperforms existing methods in generating coherent views along user-defined trajectories. Our implementation is available at https://github.com/YiGuYT/LookBeyond.

Figures

Figures reproduced from arXiv: 2509.00843 by Kourosh Khoshelham, Xueyang Kang, Zezheng Zhang, Zhengkang Xiang.

Figure 1
Figure 1. Figure 1: Two-Stage Scene View Diffusion with Camera Control. Stage 1 expands a single view into a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: First, the input image x0 is mapped onto a 360◦ panorama mask and outpainted by a diffusion model to generate a panorama image X0. This panorama is then decomposed into several perspec￾tive keyframes x𝑘 , (𝑘 = 1, . . . , 𝑁∗ ). A subset of these keyframes are selected and paired based on their relative positions in the panorama, to serve as source-target pairs for video diffusion-based novel view interpolat… view at source ↗
Figure 2
Figure 2. Figure 2: The process comprises two stages. Stage I uses a Diffusion Transformer (DiT) to outpaint a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) Input view. (b) Linearly warped view from source [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison of panorama generation. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Generated indoor frames along our predefined tra [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of FVD and mTSED frame pair match percentage scores across different numbers of gen￾erated frames for our model and baseline methods on RealEstate10K. In terms of efficiency as measured by the average video genera￾tion time for 100 frames, PhotoNVS takes 13,800s ≫ ViewCrafter at 1187.4s > Vistadream at 683.8s > Ours at 654.9s (Panorama gen￾eration ≈ 34s + video diffusion ≈ 620.9s) [PITH_FULL_IM… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of outdoor frames with baselines on [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Ablation of panorama inference sampling. Red in [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Spatial diffusion ablation study for novel video view [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Video consistency evaluation: a) FVD and mTSED [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Cycle consistency loss is enforced during the inference sampling stage by partitioning the panorama into four [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Panorama from or to perspective view image. [PITH_FULL_IMAGE:figures/full_fig_p013_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Custom trajectory generation in panoramic scenes. Using depth maps from (a) panorama view and (b) depth [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Our panorama diffusion transformer block con [PITH_FULL_IMAGE:figures/full_fig_p014_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Outpainting diffusion process at each iteration on panorama image by fusion of known region through forward [PITH_FULL_IMAGE:figures/full_fig_p015_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: The video diffusion model 𝑓𝜃 is frozen, while the raymap conditioning encoder ℎ𝜃 is learnable. For keyframe-anchored inpainting, each input frame x𝑡 is masked by the Hadamard product x𝑡 ⊙ m𝑡 to guide generation in occluded regions. The corresponding raymap embeddings {r0, . . . , r𝑁 }, derived from Plücker-embedded camera rays via intrinsic K and essential matrix E, are fused across time and channels into… view at source ↗
Figure 17
Figure 17. Figure 17: Perspective keyframe extracted from the panorama view image through neighboring keyframes with various overlapping ratios of 4, 6, and 8 keyframes. Our efficient Fréchet Video Distance (FVD) [46] implementation processes each image by first passing it through an encoder for feature embedding, followed by fusing temporal information along the stacked sequence dimension. To compute the Fréchet Video Distanc… view at source ↗
Figure 18
Figure 18. Figure 18: Keyframe extracted from the panorama view im [PITH_FULL_IMAGE:figures/full_fig_p017_18.png] view at source ↗
Figure 20
Figure 20. Figure 20: The plot of mTSEd as a function of temperature [PITH_FULL_IMAGE:figures/full_fig_p017_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: User study results showing method preference [PITH_FULL_IMAGE:figures/full_fig_p018_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Additional qualitative results of our model, showing generated views (odd rows) along a looped trajectory. The [PITH_FULL_IMAGE:figures/full_fig_p019_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Comparison with ViewCrafter. The first row shows our generated panorama; the second row presents our synthesized [PITH_FULL_IMAGE:figures/full_fig_p020_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: WonderJouney sample results on a single image input. [PITH_FULL_IMAGE:figures/full_fig_p020_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Comparison of baseline models and our approach on the RealEstate10K [ [PITH_FULL_IMAGE:figures/full_fig_p021_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Comparison of baseline models and our approach on the RealEstate10K [ [PITH_FULL_IMAGE:figures/full_fig_p022_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Comparison of baseline models and our approach on the RealEstate10K [ [PITH_FULL_IMAGE:figures/full_fig_p023_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: Comparison of baseline models and our approach on the Matterport3D [ [PITH_FULL_IMAGE:figures/full_fig_p024_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Comparison of baseline models and our approach on the Matterport3D [ [PITH_FULL_IMAGE:figures/full_fig_p025_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Comparison of baseline models and our approach on the Matterport3D [ [PITH_FULL_IMAGE:figures/full_fig_p026_30.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 45 canonical work pages · 3 internal anchors

  1. [1]

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al . 2024. VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control. arXiv preprint arXiv:2407.12781 (2024)

    Pith/arXiv arXiv 2024

  2. [2]

    Wenbo Bao, Wei-Sheng Lai, Chao Ma, Xiaoyun Zhang, Zhiyong Gao, and Ming- Hsuan Yang. 2019. Depth-aware video frame interpolation. In Proceedings of the Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion MM ’25, October 27–31, 2025, Dublin, Ireland IEEE/CVF conference on computer vision and pattern recognition . 3703–3712

    work page 2019

  3. [3]

    Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, et al. 2024. Lumiere: A space-time diffusion model for video generation. arXiv preprint arXiv:2401.12945 (2024)

    Pith/arXiv arXiv 2024

  4. [4]

    Lucy Chai, Richard Tucker, Zhengqi Li, Phillip Isola, and Noah Snavely. 2023. Persistent nature: A generative model of unbounded 3D worlds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 20863–20874

    work page 2023

  5. [5]

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3D: Learning from RGB-D Data in Indoor Environments. International Conference on 3D Vision (3DV) (2017)

    work page 2017

  6. [6]

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. 2024. VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models. arXiv:2401.09047 [cs.CV]

    Pith/arXiv arXiv 2024

  7. [7]

    Zilong Chen, Yikai Wang, Feng Wang, Zhengyi Wang, and Huaping Liu. 2024. V3d: Video diffusion models are effective 3d generators.arXiv preprint arXiv:2403.06738 (2024)

    Pith/arXiv arXiv 2024

  8. [8]

    Duolikun Danier, Fan Zhang, and David Bull. 2023. LDMVFI: Video Frame Interpolation with Latent Diffusion Models. arXiv preprint arXiv:2303.09508 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  9. [9]

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. 2023. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7346–7356

    work page 2023

  10. [10]

    Mengyang Feng, Jinlin Liu, Miaomiao Cui, and Xuansong Xie. 2023. Diffusion360: Seamless 360 degree panoramic image generation based on diffusion models. arXiv preprint arXiv:2311.13141 (2023)

    Pith/arXiv arXiv 2023

  11. [11]

    Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel. 2024. Scenescape: Text-driven consistent scene generation. Advances in Neural Information Process- ing Systems 36 (2024)

    work page 2024

  12. [12]

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin- Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. 2024. CAT3D: Create Anything in 3D with Multi-View Diffusion Models. arXiv preprint arXiv:2405.10314 (2024)

    Pith/arXiv arXiv 2024

  13. [13]

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. 2025. Cameractrl: Enabling camera control for text-to-video generation. (2025)

    work page 2025

  14. [14]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778

    2016

  15. [15]

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017)

    2017

  16. [16]

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. 2022. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 (2022)

    Pith/arXiv arXiv 2022

  17. [17]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. 2022. Video diffusion models. Advances in Neu- ral Information Processing Systems 35 (2022), 8633–8646

    work page 2022

  18. [18]

    Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou

  19. [19]

    Xiaoliang Ju, Zhaoyang Huang, Yijin Li, Guofeng Zhang, Yu Qiao, and Hong- sheng Li. 2024. DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4526–4535

    work page 2024

  20. [20]

    Xueyang Kang, Zhengkang Xiang, Zezheng Zhang, and Kourosh Khoshelham

    work page

  21. [21]

    Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Hen- schel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. 2023. Text2video-zero: Text-to-image diffusion models are zero-shot video genera- tors. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 15954–15964

    work page 2023

  22. [22]

    Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)

    Pith/arXiv arXiv 2013

  23. [23]

    Wetzstein

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hongsheng Li, Leonidas Guibas, and Gordon. Wetzstein. 2024. Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control. In arXiv

    work page 2024

  24. [24]

    Jeong-gi Kwak, Erqun Dong, Yuhe Jin, Hanseok Ko, Shweta Mahajan, and Kwang Moo Yi. 2024. Vivid-1-to-3: Novel view synthesis with video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6775–6785

    work page 2024

  25. [25]

    Ming-Feng Li, Yueh-Feng Ku, Hong-Xuan Yen, Yu-Lun Liu Chi Liu, Albert Y. C. Chen, Cheng-Hao Kuo, and Min Sun. 2024. GenRC: 3D Indoor Scene Generation from Sparse Image Collections. In ECCV

    work page 2024

  26. [26]

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. 2023. Magic3D: High-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition . 300–309

    work page 2023

  27. [27]

    Aoming Liu, Zhong Li, Zhang Chen, Nannan Li, Yi Xu, and Bryan A Plummer

    work page

  28. [28]

    Yu-Lun Liu, Yi-Tung Liao, Yen-Yu Lin, and Yung-Yu Chuang. 2019. Deep video frame interpolation using cyclic frame generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 8794–8802

    work page 2019

  29. [29]

    Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu

    Zhen Liu, Yao Feng, Michael J. Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. 2023. MeshDiffusion: Score-based Generative 3D Mesh Modeling. In International Conference on Learning Representations . https://openreview.net/ forum?id=0cpM2ApF9p6

    work page 2023

  30. [30]

    Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Natalia Neverova, Andrea Vedaldi, Oran Gafni, and Filippos Kokkinos. 2024. Im-3d: Iterative multiview diffusion and reconstruction for high-quality 3d generation. arXiv preprint arXiv:2402.08682 (2024)

    Pith/arXiv arXiv 2024

  31. [31]

    Quan Meng, Lei Li, Matthias Nießner, and Angela Dai. 2024. LT3SD: Latent Trees for 3D Scene Diffusion. arXiv preprint arXiv:2409.08215 (2024)

    Pith/arXiv arXiv 2024

  32. [32]

    Norman Müller, Katja Schwarz, Barbara Rössle, Lorenzo Porzi, Samuel Rota Bulò, Matthias Nießner, and Peter Kontschieder. 2024. MultiDiff: Consistent Novel View Synthesis from a Single Image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10258–10268

    work page 2024

  33. [33]

    Simon Niklaus, Long Mai, and Feng Liu. 2017. Video frame interpolation via adaptive separable convolution. InProceedings of the IEEE international conference on computer vision. 261–270

    work page 2017

  34. [34]

    William Peebles and Saining Xie. 2023. Scalable diffusion models with transform- ers. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 4195–4205

    work page 2023

  35. [35]

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. 2022. Dreamfusion: Text-to-3D using 2d diffusion. arXiv preprint arXiv:2209.14988 (2022)

    Pith/arXiv arXiv 2022

  36. [36]

    Xuanchi Ren and Xiaolong Wang. 2022. Look outside the room: Synthesizing a consistent long-term 3D scene video from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 3563–3573

    work page 2022

  37. [37]

    Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. 2019. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF international conference on computer vision . 9339–9347

    work page 2019

  38. [38]

    Hoigi Seo, Hayeon Kim, Gwanghyun Kim, and Se Young Chun. 2023. Ditto- nerf: Diffusion-based iterative text to omni-directional 3D model. arXiv preprint arXiv:2304.02827 (2023)

    Pith/arXiv arXiv 2023

  39. [39]

    Junyoung Seo, Kazumi Fukuda, Takashi Shibuya, Takuya Narihira, Naoki Mu- rata, Shoukang Hu, Chieh-Hsin Lai, Seungryong Kim, and Yuki Mitsufuji. 2024. GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping. Advances in Neural Information Processing Systems 37 (2024)

    work page 2024

  40. [40]

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. 2023. Zero123++: a single image to consistent multi-view diffusion base model. arXiv preprint arXiv:2310.15110 (2023)

    Pith/arXiv arXiv 2023

  41. [41]

    Zhihao Shi, Xiangyu Xu, Xiaohong Liu, Jun Chen, and Ming-Hsuan Yang. 2022. Video frame interpolation transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 17482–17491

    work page 2022

  42. [42]

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. 2022. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 (2022)

    Pith/arXiv arXiv 2022

  43. [43]

    Gabriela Ben Melech Stan, Diana Wofk, Scottie Fox, Alex Redden, Will Saxton, Jean Yu, Estelle Aflalo, Shao-Yen Tseng, Fabio Nonato, Matthias Muller, et al. 2023. Ldm3D: Latent diffusion model for 3D. arXiv preprint arXiv:2305.10853 (2023)

    Pith/arXiv arXiv 2023

  44. [44]

    Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias Nießner. 2024. Diffuscene: Denoising diffusion models for generative indoor scene synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    work page 2024

  45. [45]

    Hung-Yu Tseng, Qinbo Li, Changil Kim, Suhib Alsisan, Jia-Bin Huang, and Jo- hannes Kopf. 2023. Consistent view synthesis with pose-guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. 16773–16783

    work page 2023

  46. [46]

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. 2018. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717 (2018)

    Pith/arXiv arXiv 2018

  47. [47]

    Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. 2024. Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion. In European Conference on Computer Vision . Springer, 439–457. MM ’25, October 27–31, 2025, Dublin, Ireland Xue...

    work page 2024

  48. [48]

    Haiping Wang, Yuan Liu, Ziwei Liu, Zhen Dong, Wenping Wang, and Bisheng Yang. 2024. VistaDream: Sampling multiview consistent images for single-view scene reconstruction. arXiv preprint arXiv:2410.16892 (2024)

    Pith/arXiv arXiv 2024

  49. [49]

    Hai Wang, Xiaoyu Xiang, Yuchen Fan, and Jing-Hao Xue. 2024. Customizing 360-degree panoramas through text-to-image diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 4933–4943

    work page 2024

  50. [50]

    Xiaojuan Wang, Boyang Zhou, Brian Curless, Ira Kemelmacher-Shlizerman, Alek- sander Holynski, and Steven M Seitz. 2025. Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation. (2025)

    work page 2025

  51. [51]

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing 13, 4 (2004), 600–612

    2004

  52. [52]

    Lik Hang Kenny Wong, Xueyang Kang, Kaixin Bai, and Jianwei Zhang. 2025. A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI. arXiv preprint arXiv:2505.01458 (2025)

    Pith/arXiv arXiv 2025

  53. [53]

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. 2023. Tune-a- video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 7623– 7633

    work page 2023

  54. [54]

    Kailu Wu, Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu, Yueqi Duan, and Kaisheng Ma. 2024. Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image. arXiv:2405.20343 [cs.CV]

    Pith/arXiv arXiv 2024

  55. [55]

    Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. 2023. PanoDiffusion: 360- degree Panorama Outpainting via Diffusion. In The Twelfth International Confer- ence on Learning Representations

    work page 2023

  56. [56]

    Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. 2023. PanoDiffusion: Depth- aided 360-degree Indoor RGB Panorama Outpainting via Latent Diffusion Model. arXiv:2307.03177 [cs.CV]

    arXiv 2023

  57. [57]

    Xiaoling Xia, Cui Xu, and Bing Nan. 2017. Inception-v3 for flower classification. In 2017 2nd international conference on image, vision and computing (ICIVC) . IEEE, 783–787

    work page 2017

  58. [58]

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. 2024. InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models. arXiv preprint arXiv:2404.07191 (2024)

    Pith/arXiv arXiv 2024

  59. [59]

    Jiale Xu, Jia Zheng, Yanyu Xu, Rui Tang, and Shenghua Gao. 2021. Layout- Guided Novel View Synthesis From a Single Indoor Panorama. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    work page 2021

  60. [60]

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. 2024. Depth Anything V2. arXiv:2406.09414 (2024)

    Pith/arXiv arXiv 2024

  61. [61]

    Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. 2024. Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion. arXiv preprint arXiv:2402.03162 (2024)

    Pith/arXiv arXiv 2024

  62. [62]

    Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Yixuan Li, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. 2024. LayerPano3D: Layered 3D Panorama for Hyper- Immersive Scene Generation. arXiv:2408.13252 [cs.CV] https://arxiv.org/abs/ 2408.13252

    Pith/arXiv arXiv 2024

  63. [63]

    Xiuyu Yang, Yunze Man, Jun-Kun Chen, and Yu-Xiong Wang. 2024. SceneCraft: Layout-Guided 3D Scene Generation. arXiv preprint arXiv:2410.09049 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  64. [64]

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al . 2024. CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer. arXiv preprint arXiv:2408.06072 (2024)

    Pith/arXiv arXiv 2024

  65. [65]

    Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, et al

    work page

  66. [66]

    Jason J Yu, Fereshteh Forghani, Konstantinos G Derpanis, and Marcus A Brubaker

    work page

  67. [67]

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. 2024. ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis. arXiv preprint arXiv:2409.02048 (2024)

    Pith/arXiv arXiv 2024

  68. [68]

    Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xiaoshui Huang, Dinh Phung, Wanli Ouyang, and Jianfei Cai. 2024. Taming Stable Diffusion for Text to 360 Panorama Image Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 6347–6357

    work page 2024

  69. [69]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wonderjourney: Going from anywhere to everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 6658–6667

    work page

  70. [70]

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang

  71. [71]

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. 2023. Controlvideo: Training-free controllable text-to-video genera- tion. arXiv preprint arXiv:2305.13077 (2023)

    Pith/arXiv arXiv 2023

  72. [72]

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely

    work page

  73. [74]

    Jason Y Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani. 2024. Cameras as Rays: Pose Estimation via Ray Diffusion. In International Conference on Learning Representations (ICLR)

    work page 2024

  74. [79]

    Patchify & Linear Layer

    Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv:1805.09817 (2018). Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion MM ’25, October 27–31, 2025, Dublin, Ireland Appendix for Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion Supplementary explanations and e...

    Pith/arXiv arXiv 2018

  75. [2018]

    In Proceedings of the IEEE conference on computer vision and pattern recognition

    The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition . 586–595

  76. [2022]

    In European Conference on Computer Vision

    Real-time intermediate flow estimation for video frame interpolation. In European Conference on Computer Vision . Springer, 624–642

    work page

  77. [2023]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Long-term photometric consistent novel view synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7094–7104

    work page

  78. [2024]

    PanoFree: Tuning-Free Holistic Multi-view Image Generation with Cross-view Self-Guidance

    PanoFree: Tuning-Free Holistic Multi-view Image Generation with Cross- view Self-Guidance. arXiv preprint arXiv:2408.02157 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  79. [2025]

    In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy

    Multi-view Geometry-Aware Diffusion Transformer for Indoor Novel View Synthesis. In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy

    work page 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.