Pith. sign in

REVIEW 22 cited by

Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2307.06940 v1 pith:WQ5V7BDS submitted 2023-07-13 cs.CV

Yingqing He , Menghan Xia , Haoxin Chen , Xiaodong Cun , Yuan Gong , Jinbo Xing , Yong Zhang , Xintao Wang
show 3 more authors
Chao Weng Ying Shan Qifeng Chen
This is my paper · ORCID
classification cs.CV
keywords videomotionstructurestorytellingvideosapproachclipsdesired
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Generating videos for visual storytelling can be a tedious and complex process that typically requires either live-action filming or graphics animation rendering. To bypass these challenges, our key idea is to utilize the abundance of existing video clips and synthesize a coherent storytelling video by customizing their appearances. We achieve this by developing a framework comprised of two functional modules: (i) Motion Structure Retrieval, which provides video candidates with desired scene or motion context described by query texts, and (ii) Structure-Guided Text-to-Video Synthesis, which generates plot-aligned videos under the guidance of motion structure and text prompts. For the first module, we leverage an off-the-shelf video retrieval system and extract video depths as motion structure. For the second module, we propose a controllable video generation model that offers flexible controls over structure and characters. The videos are synthesized by following the structural guidance and appearance instruction. To ensure visual consistency across clips, we propose an effective concept personalization approach, which allows the specification of the desired character identities through text prompts. Extensive experiments demonstrate that our approach exhibits significant advantages over various existing baselines.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 22 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ShotPlan: Cinematic Video Generation with Learnable Planning Token

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Learnable planning tokens with fractional positional timestamps let one diffusion pass generate multi-shot video with frame-accurate cuts and timed camera motion.

  2. DualDiff: Dual-branch Diffusion Model for Autonomous Driving with Semantic Fusion

    cs.CV 2025-05 conditional novelty 6.0 of 10

    DualDiff introduces occupancy ray sampling, semantic fusion attention, and a foreground-aware loss to achieve state-of-the-art FID of 10.99 on nuScenes driving scene generation.

  3. MV-Crafter: An Intelligent System for Music-guided Video Generation

    cs.HC 2025-04 conditional novelty 6.0 of 10

    MV-Crafter generates beat-synchronized music videos from music and a text theme by combining LLM-based scripting, diffusion video generation, and a dynamic beat-matching warping algorithm.

  4. IP-Prompter: Training-Free Theme-Specific Image Generation via Dynamic Visual Prompting

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A training-free pipeline that uses dynamic visual prompts in an inpainting model to generate theme-consistent images without any model fine-tuning.

  5. VideoAuteur: Towards Long Narrative Video Generation

    cs.CV 2025-01 conditional novelty 6.0 of 10

    VideoAuteur builds a cooking narrative dataset and an autoregressive pipeline that generates coherent long-form cooking videos by predicting actions, captions, and CLIP-based visual embeddings step by step.

  6. VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models

    cs.CV 2024-12 conditional novelty 6.0 of 10

    VideoMaker shows that a video diffusion model can itself extract and inject reference-subject features, using reference-frame concatenation and self-attention, achieving state-of-the-art zero-shot customized video generation.

  7. Large Motion Video Autoencoding with Cross-modal Video VAE

    cs.CV 2024-12 conditional novelty 6.0 of 10

    A two-stage video autoencoder with temporal-aware spatial compression and a separate motion compressor reports state-of-the-art reconstruction quality on WebVid, Inter4K, and a large-motion test set.

  8. LeviTor: 3D Trajectory Oriented Image-to-Video Synthesis

    cs.CV 2024-12 conditional novelty 6.0 of 10

    LeviTor controls 3D object trajectories in generated videos by feeding K-means clustered mask points with estimated depth into a video diffusion model.

  9. VideoDPO: Omni-Preference Alignment for Video Diffusion Generation

    cs.CV 2024-12 reject novelty 6.0 of 10

    VideoDPO shows that DPO-style training on automatically selected best and worst video pairs improves overall VBench scores on three open text-to-video models, with some sub-metrics degrading and weak gains on external...

  10. VSD2M: A Large-scale Vision-language Sticker Dataset for Multi-frame Animated Sticker Generation

    cs.HC 2024-12 conditional novelty 6.0 of 10

    VSD2M, a 2.09 million sample bilingual sticker dataset with animated GIFs, plus a Spatial Temporal Interaction layer, improves animated sticker generation over standard video diffusion baselines.

  11. Mind the Time: Temporally-Controlled Multi-Event Video Generation

    cs.CV 2024-12 conditional novelty 6.0 of 10

    MinT generates multi-event videos where each event's timing is controlled by the user, using a fine-tuned video diffusion transformer with a time-aware rotary position embedding.

  12. PhysMotion: Physics-Grounded Dynamics From a Single Image

    cs.CV 2024-11 conditional novelty 6.0 of 10

    PhysMotion generates physically plausible videos from a single image by simulating 3D object motion with a material point method, then enhancing the rendering with a diffusion model.

  13. PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A training strategy that uses reward feedback on generated videos to put a specific face into text-to-video outputs while preserving motion and prompt following.

  14. Captain Cinema: Towards Short Movie Generation

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A two-stage text-to-movie system that plans keyframes for the story and then synthesizes video between them, using a compressed memory bank to keep long narratives consistent.

  15. PresentAgent: Multimodal Agent for Presentation Video Generation

    cs.CV 2025-07 reject novelty 5.0 of 10

    PresentAgent chains LLM segmentation, slide rendering, TTS, and ffmpeg to turn documents into narrated presentation videos, but the human-level claim rests on five documents and an unvalidated VLM judge.

  16. Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Sparse-vDiT replaces dense attention with fixed per-head sparse patterns chosen offline, achieving 1.58-1.85x end-to-end speedups on CogVideoX1.5, HunyuanVideo, and Wan2.1 with minimal quality loss.

  17. FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance

    cs.CV 2025-05 conditional novelty 5.0 of 10

    FinePhys adds a learned PhysNet that re-estimates 3D skeletons through an Euler-Lagrange parameterization and uses the fused skeletons as 2D heatmap guidance for a latent diffusion model, reporting improved generation...

  18. PhysAnimator: Physics-Guided Generative Cartoon Animation

    cs.GR 2025-01 conditional novelty 5.0 of 10

    PhysAnimator combines 2D deformable-body physics simulation with a sketch-guided video diffusion model to animate static anime illustrations with controllable, physically plausible motion.

  19. ModelGrow: Continual Text-to-Video Pre-training with Model Expansion and Language Understanding Enhancement

    cs.CV 2024-12 conditional novelty 5.0 of 10

    Continual pre-training of a text-to-video model with block duplication plus an LLM-conditioned cross-attention improves benchmark scores, but aggregate gains hide several per-dimension regressions.

  20. RMD: A Simple Baseline for More General Human Motion Generation via Training-free Retrieval-Augmented Motion Diffuse

    cs.CV 2024-12 conditional novelty 5.0 of 10

    RMD improves text-to-motion generation by decomposing prompts with an LLM, retrieving and recombining body-part motions, and refining them with a pretrained diffusion model, all without additional training.

  21. Leveraging Pre-Trained Visual Models for AI-Generated Video Detection

    cs.CV 2025-07 conditional novelty 4.0 of 10

    Pre-trained SigLIP/VideoMAE features with a linear probe or nearest-neighbor distance separate real videos from text-to-video model outputs, reaching about 90% average F1 on the new VID-AID benchmark, but much lower o...

  22. ManipDreamer: Boosting Robotic Manipulation World Model with Action Tree and Visual Guidance

    cs.RO 2025-04 reject novelty 3.0 of 10

    ManipDreamer conditions a robot-manipulation video diffusion model on action-tree instruction embeddings and multi-modal visual guidance, reporting modest gains over RoboDreamer that are undercut by evaluation inconsi...

Pith tools