T-SVG: Text-Driven Stereoscopic Video Generation

Qiao Jin; Tao Mei; Wu Liu; Xiaodong Chen; Yongdong Zhang

REVIEW 4 major objections 7 minor 1 cited by

T-SVG: Text-Driven Stereoscopic Video Generation

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a simple point-cloud re-rendering pipeline turns text prompts into stereoscopic video with no training.

desk verdict Sensible modular pipeline, but Eq. (3) likely inverts the depth convention and the evaluation compares the system only to itself; the main claim is not yet supported. read the letter →

arxiv 2412.09323 v2 pith:FV36J6OT submitted 2024-12-12 cs.CV

Qiao Jin , Xiaodong Chen , Wu Liu , Tao Mei , Yongdong Zhang This is my paper · ORCID

classification cs.CV

keywords stereoscopicvideogenerationtext-to-videozero-shotdepthestimationpointcloudrenderinginpaintingbinocularparallaxstereopsis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a usable stereoscopic video, a left-eye and right-eye pair with depth-inducing parallax, can be produced from a plain text prompt, with no stereo camera rig and no model training. The proposed system chains three off-the-shelf capabilities: a text-to-video generator creates an ordinary video, a depth estimator gives every frame a depth map, and each RGB-D frame is lifted into a 3D point cloud. The cloud is rotated and shifted by a fixed eye-baseline transform to render two perspectives, and a video inpainter fills the black gaps left by re-projection. The claim is that this geometric pipeline, borrowing already-mature models, yields a natural stereoscopic effect at a small fraction of the usual production cost.

What carries the argument

The load-bearing object is the per-frame 3D point cloud lifted from RGB-D data by the equations $X = D(x,y)(x-c_x)/f_x$, $Y = D(x,y)(y-c_y)/f_y$, and $Z = D(x,y)$. Parallax is generated not by a learned stereo network but by a single rigid transform $M = R(\theta)T$ that rotates the point cloud about the Y-axis and translates it along the X-axis before re-projection, simulating the horizontal separation of two eyes. The third piece is the video inpainter, which consumes the dropout masks from the re-projected views and synthesizes the missing pixels; it is what converts sparse, hole-ridden renders into viewable frames. The entire construction is modular: the geometric transform is fixed, while the video, depth, and inpainting models are interchangeable plugins.

What would settle it

On a dataset with ground-truth depth and stereo pairs, apply T-SVG to monocular frames and compare the rendered left and right disparities with the ground-truth stereo disparities across representative scenes; if the fixed-baseline transform produces disparities that are systematically off by a large factor, or if a large fraction of viewers cannot fuse the left-right pair on a stereo display, the central claim fails.

Watch

Extended reading notes

Core claim

T-SVG's central claim is that stereo parallax needs no learned stereo-specific model: given any text-to-video model, any monocular depth estimator, and any video inpainter, the system can synthesize a convincing stereo pair. For each frame, depth values are treated as metric distances, and each pixel is lifted into a 3D point by the camera intrinsics. The whole cloud is then transformed by a fixed rigid matrix, a rotation about the Y-axis combined with a translation along the X-axis meant to mimic the eye baseline, before being re-projected into left and right views. Re-projection creates pixel dropout wherever no point lands, and the inpainter masks and fills those regions. On the quantitative comparison, the full pipeline outperforms the no-inpainting baseline on PSNR, SSIM, LPIPS, and FVD, which the paper takes as evidence that the final videos are more faithful and more temporally consistent.

Load-bearing premise

The load-bearing premise is that the depth map produced by a monocular estimator is metric depth, so a single fixed eye-separation rotation and translation yields natural, comfortable parallax for every scene.

Editorial extensions

If this is right

New text-to-video or depth models can be dropped into T-SVG without retraining, so stereo quality improves automatically as those upstream models improve.
Producers without 3D expertise or stereo rigs can generate immersive footage from a single prompt, making stereoscopic content creation accessible to non-specialists.
The inpainting step is what separates watchable output from raw geometry: the leave-blank baseline scores 11.411 PSNR and 723.521 FVD, while the full system scores 12.793 and 398.563.
Because the stereo transform is per-frame and independent of model choice, the same pipeline also works when the input video comes from an image prompt or an existing video, not only from text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not test how the fixed eye-baseline transform behaves across scenes with very different depth scales; normalizing the baseline by median depth would be a natural, untested improvement.
The reported metrics measure pixel fidelity and perceptual similarity, not stereo fusion or viewing comfort, so a user study with an actual stereo display would be needed to confirm the natural stereoscopic effect for human viewers.
The same point-cloud rendering could generate more than two viewpoints, opening a direct path to multi-view or light-field video for glasses-free displays, a direction the paper leaves implicit.
Because the inpainter hallucinates the content behind the dropout mask, stereo correctness at object boundaries depends on the inpainter's guesses rather than on the geometric transform itself.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: T-SVG is a clean, training-free composition of existing blocks — text-to-video, monocular depth, DIBR-style point cloud re-rendering, and video inpainting. If it works as advertised, it is a useful lower-cost route to stereo video. The paper is honest about being modular and about the 'leave blank' ablation, but the evidence section does not establish that the output is natural stereo.

What is new: the specific pipeline. Individually the pieces are known; as a system that accepts text and outputs side-by-side stereo, I don't know a directly competing paper. That counts for something. The paper also gives timing and parameter counts, and it is careful to describe pixel dropout and the inpainting stage. The writing is straightforward.

Soft spots, in order of importance. First, the geometry. Equation (3) sets Z = D(x,y), the raw output of the depth estimator. DepthAnything and NVDS are relative-depth models; in the MiDaS convention the output is inverse depth (disparity), where larger values are closer. If the code feeds that raw map to Eq. (3), near objects get large Z and the point cloud is depth-inverted. A viewpoint shift then gives small parallax for near objects, i.e., a pseudoscopic pair, not a natural stereo pair. Even if DepthAnything happens to be ordered correctly, the absolute scale is arbitrary, so the fixed θ and t_x give uncontrolled disparity. The paper never states the depth convention, never normalizes, and never validates disparity. This is not a minor calibration detail; it determines whether the central claim is true.

Second, the evaluation supports almost nothing. The only comparison is against the same pipeline with inpainting removed. On KITTI, but with no description of how the stereo video is used, what the reference is, or why PSNR/SSIM/LPIPS/FVD are computed against what. The metrics are reported without the comparison target. No user study, no ground-truth stereo pair, no comparison to 2D-to-3D conversion baselines. The abstract says 'natural stereoscopic effect,' and the numbers only say inpainting is better than black holes.

Third, the paper does not cite the DIBR and depth-image-based rendering literature, even though the geometry is textbook DIBR. That is a real gap, but it is fixable and it does not invalidate the contribution.

My take: the system is plausible and the writing is coherent, but the load-bearing claim needs either a fixed normalization step plus validation on real stereo pairs, or a drastically reduced claim. As written, I would not accept the 'natural stereoscopic effect' claim. I would still send it to review — the idea deserves referee time — but with the expectation of major revision. For a reading group, the depth-convention issue is a good teaching moment.

Recommendation: engage with it as a workshop-level or major-revision paper; do not desk reject, but do not trust the quantitative section until it is redone.

Referee Report

4 major / 7 minor

Summary. The paper presents T-SVG, a training-free, model-agnostic pipeline that converts a text prompt into a stereoscopic video: Open Sora generates a reference video, DepthAnything or NVDS estimates per-frame depth, each RGB-D frame is back-projected into a 3D point cloud (Eqs. (1)-(3)), the point cloud is transformed by a rotation-plus-translation matrix (Eqs. (4)-(5)) and rendered from two perspectives to mimic the left and right eyes, ProPainter inpaints the resulting 'pixel dropout' holes, and the two views are concatenated into a stereo video. The authors claim that this achieves a 'natural stereoscopic effect' and constitutes 'a significant advancement in stereoscopic content creation.' The quantitative evaluation (Table II) compares the full system to the same pipeline without inpainting ('leave blank') on the KITTI dataset using PSNR, SSIM, LPIPS, and FVD. The manuscript is clearly organized, but the central claims rest on an unexamined assumption about the depth estimator's output convention and on an evaluation that does not measure stereoscopic correctness.

Significance. The proposed composition of off-the-shelf components is appealing: the system is zero-shot, requires no training, and the modular design genuinely allows each component to be swapped without retraining. The inpainting stage demonstrably addresses the pixel-dropout artifact it is designed for, and the architecture description is easy to follow. If the depth-convention and evaluation issues identified below are resolved, T-SVG could serve as a useful, cheap baseline for text-driven stereo content creation and as a reference point for future work. At present, however, the headline claim of a 'natural stereoscopic effect' lacks supporting evidence: there is no comparison against ground-truth stereo data, no measure of disparity correctness or binocular consistency, no human viewing study, and the only quantitative baseline is an internally degraded version of the same system. The concept is a modest but legitimate contribution; its significance depends on the additional validation requested below.

major comments (4)

[III-A, Eq. (3)] The pipeline sets Z = D(x,y) directly from the raw output of DepthAnything [27] or NVDS [28], but these monocular estimators are trained with scale-and-shift-invariant losses; in the MiDaS convention that DepthAnything inherits, the output is inverse depth (disparity), with larger values meaning closer points. Substituting that raw value into Eq. (3) assigns large Z to near objects, so the viewpoint shift of Eq. (4) produces small parallax for near objects and large parallax for far objects - a depth-reversed, pseudoscopic pair, the opposite of a natural stereoscopic effect. Even if the ordering convention were the opposite, the absolute scale of D is arbitrary, so a fixed theta and t_x give uncontrolled, scene-dependent disparity. The manuscript does not disclose the output convention, does not normalize or invert the depth, and provides no disparity validation; this is load-bearing for the paper's central claim. A concrete remedy is to state the models' convention, invert/normalize D accordingly, and verify predicted disparities against KITTI ground truth.
[IV-B, Table II] The only quantitative comparison is against a 'leave blank' ablation of the same pipeline without inpainting. Because inpainting exists precisely to fill the black dropout pixels, it trivially improves PSNR/SSIM/LPIPS against a reference, so these numbers show that inpainting repairs dropout, not that the stereo geometry is correct. None of the four metrics is a stereoscopic quality measure: there is no disparity error relative to ground truth, no left-right consistency check, no evaluation of perceived depth, fusion, or viewing comfort, and no check of temporal stability of the parallax across frames, all of which are essential for the claimed 'natural stereoscopic effect.' The evaluation needs an external anchor, e.g., synthesizing a right view from KITTI left frames and comparing it to the real KITTI right frames, or comparing predicted disparity to KITTI LiDAR ground truth, together with a subjective viewing test.
[IV-B] The KITTI evaluation setup is not described. The paper does not state which sequences were used, how many frames were evaluated, whether the inputs were generated from text prompts or were real KITTI frames, what served as the reference for PSNR/SSIM/LPIPS, or what distribution was used as the real-video reference for FVD. The stated test resolution (480x540, 120 frames) is inconsistent with KITTI's native resolution (1242x375) and no resizing or cropping procedure is given. Consequently, the numbers in Table II cannot be reproduced or interpreted.
[III-B and IV-A] The parallax parameters are unspecified. The values of theta and t_x in Eq. (4) and of f_x, f_y in Eqs. (1)-(2) are never given, and Algorithm 1's GetExtrinsics(P_pcd, vparams) is not defined: the text describes a single matrix M 'to match a single eye's perspective,' yet a single transform cannot produce two views, so the parameterization of the second eye (e.g., opposite signs of theta and t_x) is missing. These parameters control the magnitude and naturalness of the parallax, so the experiments are not reproducible without them.

minor comments (7)

[Eq. (4)] The displayed rotation matrix R(theta) has a '1' in its (1,4) entry instead of 0, so the written R(theta) is not a valid rotation matrix and the product R(theta)*T is displayed incorrectly; the equation should be corrected.
[Table II] The column header 'PNSR' is a typo for 'PSNR'; also, the reported values (11.4-12.8 dB PSNR, SSIM below 0.5) are modest and the paper gives no interpretation of their absolute magnitude.
[Fig. 1 caption and Algorithm 1] Fig. 1 and Fig. 2 indicate that the inputs can be text, images, or videos, but Algorithm 1 and Section III describe a text-only input; the role of the optional reference video should be integrated into the algorithm description.
[Index Terms and Section IV-A] The index term 'Text-driven Stereoscopy Video Generation' should read 'Stereoscopic Video Generation'; Section IV.A also contains 'This allows border use of T-SVG,' which appears to mean 'broader use.'
[Section II] The sentence 'This section reviews the development of text-to-video generation methods [9]-[13]' cites [9]-[13], which are the authors' prior works on point cloud action recognition, motion capture, whole-body motion generation, video temporal grounding, and human video generation, none of which are text-to-video generation methods; appropriate text-to-video references should be substituted.
[Section IV-A] Table I lists two depth estimators (DepthAnything and NVDS) but the experiments do not compare them; the paper should state which estimator was used for the results in Table II or report both.
[Section IV-B] The claim that the approach is 'the first attempt in this field' is unsupported; the manuscript should discuss prior depth-based view synthesis and 2D-to-3D video conversion literature before asserting priority.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: T-SVG is a constructive pipeline whose stereoscopic output is defined by its depth-based rendering equations, not derived from the claim it is supposed to support.

full rationale

T-SVG does not derive a prediction from first principles; it constructs stereo video by a chain of external models (video generator, depth estimator, inpainter) and fixed geometric transformations. Equations (1)-(3) convert depth values into 3D points, and Equations (4)-(5) apply a rotation and translation to generate two viewpoints. This is a definition of a rendering procedure, not a reduction of the conclusion to the premise; the output would be whatever the procedure produces. The depth estimator and inpainter are cited external tools, so the pipeline is self-contained against external components. The quantitative evaluation in Table II compares T-SVG to a 'leave blank' version of the same pipeline without inpainting; this is an internal ablation and provides only weak evidence for the global 'natural stereoscopic effect' claim, but it is not circular because the two compared conditions differ by a real component and the metrics are computed on the generated frames. The concern that DepthAnything/NVDS output affine-invariant or inverse depth is a correctness and robustness issue (the pipeline may produce pseudoscopic or miscalibrated parallax), not a circularity issue, because it does not make the derivation equivalent to its inputs. Several related-work references are self-citations, but they are not load-bearing: none justifies the central geometric claim or forbids alternative designs. Hence no circular step meeting the required evidence threshold is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard rendering assumptions and hand-set parameters. The most fragile item is the treatment of monocular depth as metric depth for parallax generation, which is an implicit free parameter and a load-bearing domain assumption.

free parameters (3)

Parallax rotation angle theta and translation t_x = not disclosed
Used in Eq. (4) to control the stereo baseline; no values or scene-adaptive strategy are given. The perceived depth depends directly on these hand-set parameters.
Focal lengths f_x and f_y = approximated to match average human eye
Set in Section III-A for the pinhole projection; the exact values are not reported, and they affect the 3D reconstruction and parallax magnitude.
Depth scale and shift = implicit, uncalibrated
Depth from monocular estimators such as DepthAnything is affine-invariant; using D(x,y) directly as Z in Eq. (3) treats it as metric depth, so the absolute scale is arbitrary and scene-dependent, affecting the stereo baseline.

assumptions (4)

standard math Pinhole camera model with known intrinsics
Assumed in Eqs. (1)-(3) to unproject RGB-D frames to 3D point clouds.
domain assumption Monocular depth estimates are suitable metric depths for parallax generation
The system sets Z = D(x,y) and applies a fixed baseline; this assumes the depth map corresponds to real distances, which is generally false for affine-invariant estimators.
domain assumption Video inpainting can plausibly fill disocclusion holes without artifacts
The method relies on ProPainter to fill pixel dropout regions; no analysis of inpainting failures or their effect on stereo quality is provided.
domain assumption Rotation and translation of a point cloud approximates human binocular vision
Section III-B mimics eye separation with a rigid transform, ignoring vergence, accommodation, and per-scene depth adjustments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of T-SVG: Text-Driven Stereoscopic Video Generation." pith.science (2026). https://pith.science/paper/FV36J6OT

@misc{pith2026241209323,
  author       = {Pith},
  title        = {Pith review of: T-SVG: Text-Driven Stereoscopic Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FV36J6OT}},
  note         = {Machine review of arXiv:2412.09323}
}

read the original abstract

The advent of stereoscopic videos has opened new horizons in multimedia, particularly in extended reality (XR) and virtual reality (VR) applications, where immersive content captivates audiences across various platforms. Despite its growing popularity, producing stereoscopic videos remains challenging due to the technical complexities involved in generating stereo parallax. This refers to the positional differences of objects viewed from two distinct perspectives and is crucial for creating depth perception. This complex process poses significant challenges for creators aiming to deliver convincing and engaging presentations. To address these challenges, this paper introduces the Text-driven Stereoscopic Video Generation (T-SVG) system. This innovative, model-agnostic, zero-shot approach streamlines video generation by using text prompts to create reference videos. These videos are transformed into 3D point cloud sequences, which are rendered from two perspectives with subtle parallax differences, achieving a natural stereoscopic effect. T-SVG represents a significant advancement in stereoscopic content creation by integrating state-of-the-art, training-free techniques in text-to-video generation, depth estimation, and video inpainting. Its flexible architecture ensures high efficiency and user-friendliness, allowing seamless updates with newer models without retraining. By simplifying the production pipeline, T-SVG makes stereoscopic video generation accessible to a broader audience, demonstrating its potential to revolutionize the field.

Figures

Figures reproduced from arXiv: 2412.09323 by the authors.

**Figure 2.** An overview structure of T-SVG. 1) First, we generate the video-depth pair according to the input text or video. 2) Then we generate 3D point clouds [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Details of Point Cloud Modeling and Perspective Rotation. The figure intuitively illustrates the processing steps and results of a single frame image [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Illustration of pixel dropout and video inpainting. The first column [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Restereo: Diffusion stereo video generation and restoration
cs.CV 2025-06 conditional novelty 6.0 of 10

A diffusion model fine-tuned on synthetically degraded stereo videos simultaneously generates a consistent stereo pair and restores low-resolution or compressed input, outperforming prior stereo generators on low-qual...

Reference graph

Works this paper leans on

31 extracted references · 18 canonical work pages · cited by 1 Pith paper

[27]

Depth anything: Unleashing the power of large-scale unlabeled data,

L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” CoRR, vol. abs/2401.10891, 2024

arXiv 2024
[28]

Neural video depth stabilizer,

Y . Wang, M. Shi, J. Li, Z. Huang, Z. Cao, J. Zhang, K. Xian, and G. Lin, “Neural video depth stabilizer,” in ICCV. IEEE, 2023, pp. 9432–9442

work page 2023
[1]

S. M. LaValle, Virtual reality. Cambridge university press, 2023

work page 2023
[2]

Matsuyama, S

T. Matsuyama, S. Nobuhara, T. Takai, and T. Tung, 3D Video and Its Applications . Springer, 2012. [Online]. Available: https: //doi.org/10.1007/978-1-4471-4120-4

work page doi:10.1007/978-1-4471-4120-4 2012
[3]

Make-a-video: Text-to-video generation without text-video data,

U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, D. Parikh, S. Gupta, and Y . Taigman, “Make-a-video: Text-to-video generation without text-video data,” in The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023. [Online]. Available: htt...

work page 2023
[4]

Stable video diffusion: Scaling latent video diffusion models to large datasets,

A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y . Levi, Z. English, V . V oleti, A. Letts, V . Jampani, and R. Rombach, “Stable video diffusion: Scaling latent video diffusion models to large datasets,” CoRR, vol. abs/2311.15127, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2311.15127

arXiv doi:10.48550/arxiv.2311.15127 2023
[5]

Videocrafter2: Overcoming data limitations for high-quality video diffusion models,

H. Chen, Y . Zhang, X. Cun, M. Xia, X. Wang, C. Weng, and Y . Shan, “Videocrafter2: Overcoming data limitations for high-quality video diffusion models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024 . IEEE, 2024, pp. 7310–7320. [Online]. Available: https://doi.org/10.1109/CVPR52733.2024.00698

arXiv 2024
[6]

Video generation models as world simulators,

T. Brooks, B. Peebles, C. Holmes, W. DePue, Y . Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luhman, C. Ng, R. Wang, and A. Ramesh, “Video generation models as world simulators,” 2024. [Online]. Available: https: //openai.com/research/video-generation-models-as-world-simulators

2024

Show all 31 references

[7]

3d video communications: Challenges and opportunities,

G. Su, Y . Lai, A. Kwasinski, and H. Wang, “3d video communications: Challenges and opportunities,” Int. J. Commun. Syst. , vol. 24, no. 10, pp. 1261–1281, 2011. [Online]. Available: https://doi.org/10.1002/dac.1190

2011 doi
[8]

From 2d- to stereo- to multi-view video,

S. Knorr, A. Smolic, and T. Sikora, “From 2d- to stereo- to multi-view video,” in 2007 3DTV Conference , 2007, pp. 1–4

2007
[9]

Maple: Masked pseudo-labeling autoencoder for semi-supervised point cloud action recognition,

X. Chen, W. Liu, X. Liu, Y . Zhang, J. Han, and T. Mei, “Maple: Masked pseudo-labeling autoencoder for semi-supervised point cloud action recognition,” in Proceedings of the 30th ACM International Conference on Multimedia , 2022, pp. 708–718

2022
[10]

Motion capture from inertial and vision sensors,

X. Chen, W. Liu, Q. Bao, X. Liu, Q. Yang, R. Dai, and T. Mei, “Motion capture from inertial and vision sensors,” arXiv preprint arXiv:2407.16341, 2024

2024 arXiv
[11]

M-adaptor: Text-driven whole-body human motion generation,

A. Li, X. Chen, B. Liang, Q. Bao, and W. Liu, “M-adaptor: Text-driven whole-body human motion generation,” in Proceedings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 2604–2613

2025
[12]

Animateanywhere: Context-controllable human video generation with id-consistent one-shot learning,

H. Liu, X. Chen, X. Liu, X. Gu, and W. Liu, “Animateanywhere: Context-controllable human video generation with id-consistent one-shot learning,” in Proceedings of the 5th International Workshop on Human- centric Multimedia Analysis , 2024, pp. 41–43

2024
[13]

Chatvtg: Video temporal grounding via chat with video dialogue large language models,

M. Qu, X. Chen, W. Liu, A. Li, and Y . Zhao, “Chatvtg: Video temporal grounding via chat with video dialogue large language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1847–1856

2024
[14]

Imagen video: High definition video generation with diffusion models,

J. Ho, W. Chan, C. Saharia, J. Whang, R. Gao, A. A. Gritsenko, D. P. Kingma, B. Poole, M. Norouzi, D. J. Fleet, and T. Salimans, “Imagen video: High definition video generation with diffusion models,” CoRR, vol. abs/2210.02303, 2022. [Online]. Available: https://doi.org/10.485...

doi arXiv
[15]

Video diffusion models,

J. Ho, T. Salimans, A. A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,” in Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - Dece...

2022
[16]

Audio-visual cross-modal generation with multimodal variational generative model,

Z. Xu, T. Wang, D. Liu, D. Hu, H. Zeng, and J. Cao, “Audio-visual cross-modal generation with multimodal variational generative model,” in IEEE International Symposium on Circuits and Systems, ISCAS 2024, Singapore, May 19-22, 2024 . IEEE, 2024, pp. 1–5. [Online]. Available: h...

2024
[17]

Latte: Latent diffusion transformer for video generation,

X. Ma, Y . Wang, G. Jia, X. Chen, Z. Liu, Y . Li, C. Chen, and Y . Qiao, “Latte: Latent diffusion transformer for video generation,” CoRR, vol. abs/2401.03048, 2024. [Online]. Available: https://doi.org/ 10.48550/arXiv.2401.03048

2024 doi arXiv
[18]

Generative adversarial networks,

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y . Bengio, “Generative adversarial networks,” CoRR, vol. abs/1406.2661, 2014. [Online]. Available: http://arxiv.org/abs/1406.2661

2014 arXiv
[19]

Denoising diffusion probabilistic models,

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , H. Larochelle, M. Ranzato, R. Hadse...

2020
[20]

Denoising diffusion implicit models,

J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. [Online]. Available: https://openreview.net/forum?id=St1giarCHLP

2021
[21]

Auto-encoding variational bayes,

D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Y . Bengio and Y . LeCun, Eds., 2014. [Online]. Available: http://arxiv.o...

2014 arXiv
[22]

Neural discrete representation learning,

A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural discrete representation learning,” in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, I. Guyon, U. von Luxbu...

2017
[23]

High-resolution image synthesis with latent diffusion models,

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 . IEEE, 2022, pp. 10 674–10 685. [O...

2022
[24]

Scaling laws for neural language models,

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” CoRR, vol. abs/2001.08361, 2020. [Online]. Available: https://arxiv.org/abs/2001.08361

2001 arXiv
[25]

The analysis of stereopsis,

G. F. Poggio and T. Poggio, “The analysis of stereopsis,” Annual Review of Neuroscience, vol. 7, no. 1, pp. 379–412, Mar. 1984

1984
[26]

Open-sora: Democratizing efficient video production for all,

hpcaitech, “Open-sora: Democratizing efficient video production for all,” GitHub, 2024, accessed: 2024/05/28. [Online]. Available: https://github.com/hpcaitech/Open-Sora

2024
[29]

Propainter: Improving propagation and transformer for video inpainting,

S. Zhou, C. Li, K. C. K. Chan, and C. C. Loy, “Propainter: Improving propagation and transformer for video inpainting,” in ICCV. IEEE, 2023, pp. 10 443–10 452

2023
[30]

Object scene flow,

M. Menze, C. Heipke, and A. Geiger, “Object scene flow,” ISPRS Journal of Photogrammetry and Remote Sensing (JPRS) , 2018

2018
[2020]

Available: https://proceedings.neurips.cc/paper/2020/ hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html

[Online]. Available: https://proceedings.neurips.cc/paper/2020/ hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html

2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [27]

Depth anything: Unleashing the power of large-scale unlabeled data,

L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” CoRR, vol. abs/2401.10891, 2024

arXiv 2024

[2] [28]

Neural video depth stabilizer,

Y . Wang, M. Shi, J. Li, Z. Huang, Z. Cao, J. Zhang, K. Xian, and G. Lin, “Neural video depth stabilizer,” in ICCV. IEEE, 2023, pp. 9432–9442

work page 2023

[3] [1]

S. M. LaValle, Virtual reality. Cambridge university press, 2023

work page 2023

[4] [2]

Matsuyama, S

T. Matsuyama, S. Nobuhara, T. Takai, and T. Tung, 3D Video and Its Applications . Springer, 2012. [Online]. Available: https: //doi.org/10.1007/978-1-4471-4120-4

work page doi:10.1007/978-1-4471-4120-4 2012

[5] [3]

Make-a-video: Text-to-video generation without text-video data,

U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, D. Parikh, S. Gupta, and Y . Taigman, “Make-a-video: Text-to-video generation without text-video data,” in The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023. [Online]. Available: htt...

work page 2023

[6] [4]

Stable video diffusion: Scaling latent video diffusion models to large datasets,

A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y . Levi, Z. English, V . V oleti, A. Letts, V . Jampani, and R. Rombach, “Stable video diffusion: Scaling latent video diffusion models to large datasets,” CoRR, vol. abs/2311.15127, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2311.15127

arXiv doi:10.48550/arxiv.2311.15127 2023

[7] [5]

Videocrafter2: Overcoming data limitations for high-quality video diffusion models,

H. Chen, Y . Zhang, X. Cun, M. Xia, X. Wang, C. Weng, and Y . Shan, “Videocrafter2: Overcoming data limitations for high-quality video diffusion models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024 . IEEE, 2024, pp. 7310–7320. [Online]. Available: https://doi.org/10.1109/CVPR52733.2024.00698

arXiv 2024

[8] [6]

Video generation models as world simulators,

T. Brooks, B. Peebles, C. Holmes, W. DePue, Y . Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luhman, C. Ng, R. Wang, and A. Ramesh, “Video generation models as world simulators,” 2024. [Online]. Available: https: //openai.com/research/video-generation-models-as-world-simulators

2024