Pith. sign in

REVIEW 12 cited by

Depth Any Video with Scalable Synthetic Data

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.10815 v2 pith:OUK4W4KC submitted 2024-10-14 cs.CV cs.AI

Honghui Yang , Di Huang , Wei Yin , Chunhua Shen , Haifeng Liu , Xiaofei He , Binbin Lin , Wanli Ouyang
show 1 more author
Tong He
This is my paper
classification cs.CVcs.AI
keywords depthvideodatamodelmodelsscalableacrossframes
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse virtual environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates-even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency. The code and model weights are open-sourced.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Unified Video Dense Prediction from Disjoint Data

    cs.CV 2026-07 conditional novelty 7.0 of 10

    A single video backbone predicts eight dense scene tasks from separate single-task datasets via latent distillation from diffusion-based specialists, with no co-annotated data or pseudo-labels.

  2. Forget, Anticipate and Adapt: Test Time Training for Long Videos

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    FFN enables efficient TTT for long videos by operating on three frames and using a surprise-based adaptive window, shown on a new dataset of up to 3-hour videos for segmentation and classification tasks.

  3. VDPP: Video Depth Post-Processing for Speed and Scalability

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    VDPP is an RGB-free video depth post-processor that achieves over 43 FPS on Jetson Orin Nano by refining geometry at low resolution rather than reconstructing full scenes.

  4. Video Generation Models are General-Purpose Vision Learners

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A video-diffusion backbone fine-tuned as a single-step multi-task perceiver matches or beats specialists on depth, normals, pose and segmentation, with high data efficiency and sim-to-real transfer.

  5. Forget, Anticipate and Adapt: Test Time Training for Long Videos

    cs.CV 2026-06 conditional novelty 6.0 of 10

    FFN performs efficient test-time training on multi-hour videos by forgetting the exiting frame, anticipating the next, and adapting only when a surprise metric exceeds a dynamic threshold.

  6. Forget, Anticipate and Adapt: Test Time Training for Long Videos

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    FFN performs TTT on multi-hour videos by restricting updates to three frames and using a surprise metric for adaptive window sizing, plus a new EpicTours dataset.

  7. Towards Consistent Video Geometry Estimation

    cs.CV 2026-05 conditional novelty 6.0 of 10

    One transformer, trained with random-sized temporal attention chunks, unifies offline, streaming, and long-video depth, normal, and point-map estimation and reports new best numbers on five public benchmarks.

  8. UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    UniVidX unifies diverse video generation tasks into one conditional diffusion model using stochastic condition masking, decoupled gated LoRAs, and cross-modal self-attention.

  9. Less is More: Data-Efficient Adaptation for Controllable Text-to-Video Generation

    cs.CV 2025-11 unverdicted novelty 6.0 of 10

    Fine-tuning text-to-video models on sparse low-quality synthetic data for physical camera controls outperforms fine-tuning on photorealistic data.

  10. Towards Consistent Video Geometry Estimation

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    ViGeo is a feed-forward transformer for video geometry that introduces dynamic chunking attention and a completion-based data refinement framework to achieve SOTA on depth, normals, and point map estimation.

  11. TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration

    cs.CV 2026-01 unverdicted novelty 5.0 of 10

    TPGDiff introduces hierarchical triple-prior guidance in a diffusion network, placing degradation priors throughout, structural priors in shallow layers, and semantic priors in deep layers for improved all-in-one imag...

  12. Reconstructing 4D Spatial Intelligence: A Survey

    cs.CV 2025-07 accept novelty 4.0 of 10

    A review that classifies 4D scene reconstruction methods into five progressive levels: low-level cues, scene components, dynamic scenes, interactions, and physics.

Pith tools