Pith. sign in

REVIEW 11 cited by

MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.05014 v2 pith:OKP64JOX submitted 2024-04-07 cs.CV

Shenghai Yuan , Jinfa Huang , Yujun Shi , Yongqi Xu , Ruijie Zhu , Bin Lin , Xinhua Cheng , Li Yuan
show 1 more author
Jiebo Luo
This is my paper · ORCID
classification cs.CV
keywords metamorphicvideosgenerationtime-lapsevideoknowledgemagictimephysical
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Recent advances in Text-to-Video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions. A largely overlooked problem in T2V is that existing models have not adequately encoded physical knowledge of the real world, thus generated videos tend to have limited motion and poor variations. In this paper, we propose \textbf{MagicTime}, a metamorphic time-lapse video generation model, which learns real-world physics knowledge from time-lapse videos and implements metamorphic generation. First, we design a MagicAdapter scheme to decouple spatial and temporal training, encode more physical knowledge from metamorphic videos, and transform pre-trained T2V models to generate metamorphic videos. Second, we introduce a Dynamic Frames Extraction strategy to adapt to metamorphic time-lapse videos, which have a wider variation range and cover dramatic object metamorphic processes, thus embodying more physical knowledge than general videos. Finally, we introduce a Magic Text-Encoder to improve the understanding of metamorphic video prompts. Furthermore, we create a time-lapse video-text dataset called \textbf{ChronoMagic}, specifically curated to unlock the metamorphic video generation ability. Extensive experiments demonstrate the superiority and effectiveness of MagicTime for generating high-quality and dynamic metamorphic videos, suggesting time-lapse video generation is a promising path toward building metamorphic simulators of the physical world. Code: https://github.com/PKU-YuanGroup/MagicTime

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. E-4DGS: High-Fidelity Dynamic Reconstruction from the Multi-view Event Cameras

    cs.CV 2025-08 conditional novelty 6.0 of 10

    E-4DGS is a deformable 3D Gaussian Splatting method that reconstructs dynamic scenes directly from multi-view event camera streams, outperforming event-to-image baseline approaches.

  2. ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos

    cs.CV 2025-05 conditional novelty 6.0 of 10

    ProphetDWM is a one-stage diffusion world model that jointly predicts future driving video and low-level actions from a current frame and a short action sequence.

  3. Ingredients: Blending Custom Photos with Video Diffusion Transformers

    cs.CV 2025-01 conditional novelty 6.0 of 10

    Ingredients adds a mask-supervised identity router to a video diffusion transformer, enabling multi-person videos from a few reference photos without per-identity fine-tuning.

  4. GenMAC: Compositional Text-to-Video Generation with Multi-Agent Collaboration

    cs.CV 2024-12 conditional novelty 6.0 of 10

    An iterative design-generate-redesign pipeline with four specialized LLM agents and self-routing correction improves compositional text-to-video generation on T2V-CompBench, with the largest gains in object numeracy.

  5. DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A two-stage diffusion pipeline that enriches 2D pose guidance with generated depth and normal maps to achieve state-of-the-art human image animation.

  6. WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Model

    cs.CV 2024-11 conditional novelty 6.0 of 10

    WF-VAE uses multi-level Haar wavelets to route low-frequency video content around a smaller backbone, cutting compute and memory, and adds a lossless Causal Cache for block-wise inference.

  7. Identity-Preserving Text-to-Video Generation by Frequency Decomposition

    cs.CV 2024-11 conditional novelty 6.0 of 10

    ConsisID generates identity-preserving videos by injecting low-frequency facial features into shallow layers and high-frequency identity features into attention blocks of a DiT video model.

  8. Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric

    cs.CV 2024-11 conditional novelty 6.0 of 10

    Human-AGVQA is a 6,000-video benchmark with human quality scores and body-part distortion labels for AI-generated human activity videos, and GHVQ is a trained metric that outperforms prior quality metrics on it.

  9. ViBe: A Text-to-Video Benchmark for Evaluating Hallucination in Large Multimodal Models

    cs.CV 2024-11 conditional novelty 6.0 of 10

    The paper builds and human-labels a 3,782-video dataset spanning five hallucination categories in text-to-video outputs, and shows that standard classifiers reach only about 35% accuracy on the resulting classification task.

  10. AnimateAnything: Consistent and Controllable Animation for Video Generation

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A two-stage video generation system that converts camera, drag, and reference-video controls into unified optical flows, and adds a frequency-domain stabilizer to reduce flicker.

  11. Identity-Preserving Text-to-Video Generation via Training-Free Prompt, Image, and Guidance Enhancement

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A training-free prompt, image, and guidance enhancement framework improves face consistency and video quality for identity-preserving text-to-video generation, winning the ACM Multimedia 2025 IPVG challenge.

Pith tools