Pith. sign in

REVIEW 10 cited by

Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2212.11565 v2 pith:QX2M2SGY submitted 2022-12-22 cs.CV

Jay Zhangjie Wu , Yixiao Ge , Xintao Wang , Weixian Lei , Yuchao Gu , Yufei Shi , Wynne Hsu , Ying Shan
show 2 more authors
Xiaohu Qie Mike Zheng Shou
This is my paper
classification cs.CV
keywords modelsgenerationone-shottuningdiffusionemploygenerateimage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

To replicate the success of text-to-image (T2I) generation, recent works employ large-scale video datasets to train a text-to-video (T2V) generator. Despite their promising results, such paradigm is computationally expensive. In this work, we propose a new T2V generation setting$\unicode{x2014}$One-Shot Video Tuning, where only one text-video pair is presented. Our model is built on state-of-the-art T2I diffusion models pre-trained on massive image data. We make two key observations: 1) T2I models can generate still images that represent verb terms; 2) extending T2I models to generate multiple images concurrently exhibits surprisingly good content consistency. To further learn continuous motion, we introduce Tune-A-Video, which involves a tailored spatio-temporal attention mechanism and an efficient one-shot tuning strategy. At inference, we employ DDIM inversion to provide structure guidance for sampling. Extensive qualitative and numerical experiments demonstrate the remarkable ability of our method across various applications.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. V2V-Bench: A Comprehensive Benchmark for Video-to-Video Generation Evaluation

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    V2V-Bench is a new 11-dimension benchmark for video-to-video generation that achieves 0.905 Spearman correlation with human judgments on six V2V-specific dimensions.

  2. Functionalization via Structure Completion and Motion Rectification

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Object functionalization is cast as neural graph completion over a functional graph of parts, contacts, and motions, followed by geometry realization that also rectifies erroneous motions, demonstrated on furniture wi...

  3. Physics-Aware Video Instance Removal Benchmark

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    The PVIR benchmark tests video object removal on physical consistency using 95 annotated videos and shows that existing methods struggle with complex interactions like lingering shadows.

  4. Learning Interactive Real-World Simulators

    cs.AI 2023-10 conditional novelty 7.0 of 10

    UniSim learns a universal real-world simulator from orchestrated diverse datasets, enabling zero-shot deployment of policies trained purely in simulation.

  5. OSVE: One Step Video Editing with One Step Diffusion Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    OSVE performs text-guided video editing with one-step diffusion models by training a single-pass inversion encoder and unifying frame latents for cross-frame attention, achieving quality comparable to multi-step metho...

  6. PixArt-$\alpha$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

    cs.CV 2023-09 accept novelty 6.0 of 10

    PixArt-α matches commercial text-to-image quality with a diffusion transformer trained in 675 A100 GPU days through decomposed training stages, cross-attention text injection, and vision-language model dense captions.

  7. TokenFlow: Consistent Diffusion Features for Consistent Video Editing

    cs.CV 2023-07 conditional novelty 6.0 of 10

    TokenFlow produces consistent text-driven video edits by propagating diffusion features according to inter-frame correspondences extracted from the source video.

  8. InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

    cs.CV 2023-07 unverdicted novelty 6.0 of 10

    InternVid supplies 7M videos and LLM captions to train ViCLIP, which reaches leading zero-shot action recognition and competitive retrieval performance.

  9. SOWing Information: Cultivating Contextual Coherence with MLLMs in Image Generation

    cs.CV 2024-11 unverdicted novelty 5.0 of 10

    SOW uses MLLMs and attention to selectively control unidirectional diffusion for pixel-level fidelity and contextual coherence in text-vision-to-image tasks.

  10. Communicative Agents for Slideshow Storytelling Video Generation based on LLMs

    cs.AI 2025-09 conditional novelty 4.0 of 10

    VGTeam uses communicating LLM agents plus commercial APIs to turn a text prompt into a slideshow video for about $0.10 per clip, with a self-reported 75.7% quality rate.

Pith tools