Introduces KathaTrace protocol and KathaBench-25K benchmark to quantify Semantic Trajectory Gap (STG) as the loss of transition meaning in visualized narratives, reporting STG of 23.5 +/- 1.3 across generators.
hub
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman
17 Pith papers cite this work. Polarity classification is still indexing.
17
Pith papers citing it
abstract
The advent of AI-Generated Content (AIGC) has spurred research into automated video generation to streamline conventional processes. However, automating storytelling video production, particularly for customized narratives, remains challenging due to the complexity of maintaining subject consistency across shots. While existing approaches like Mora and AesopAgent integrate multiple agents for Story-to-Video (S2V) generation, they fall short in preserving protagonist consistency and supporting Customized Storytelling Video Generation (CSVG). To address these limitations, we propose StoryAgent, a multi-agent framework designed for CSVG. StoryAgent decomposes CSVG into distinct subtasks assigned to specialized agents, mirroring the professional production process. Notably, our framework includes agents for story design, storyboard generation, video creation, agent coordination, and result evaluation. Leveraging the strengths of different models, StoryAgent enhances control over the generation process, significantly improving character consistency. Specifically, we introduce a customized Image-to-Video (I2V) method, LoRA-BE, to enhance intra-shot temporal consistency, while a novel storyboard generation pipeline is proposed to maintain subject consistency across shots. Extensive experiments demonstrate the effectiveness of our approach in synthesizing highly consistent storytelling videos, outperforming state-of-the-art methods. Our contributions include the introduction of StoryAgent, a versatile framework for video generation tasks, and novel techniques for preserving protagonist consistency.
hub tools
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
EM-Vid introduces an entity-centric latent patch memory bank with sparse token conditioning and budgeted updates for training-free consistent multi-shot video generation.
CausalCine enables real-time causal autoregressive multi-shot video generation via multi-shot training, content-aware memory routing for coherence, and distillation to few-step inference.
GenHSI is a training-free three-stage pipeline that turns a scene image, character image, and complex HSI prompt into long videos with plausible chained interactions by generating atomic actions, 3D keyframes via 2D inpainting plus optimization, and then feeding them to pre-trained video diffusion.
Geometry-indexed depth–pose retrieval plus schema SFT and GRPO planning improves faithfulness, consistency, and controllability of plot-to-short-drama video generation over multi-agent and text-only baselines.
A training-free framework that reorders shot generation and maintains per-entity visual memory improves cross-shot character, object, and scene consistency over narrative-order memory baselines.
CCDM uses attribute-decoupled LoRA with relevance-guided aggregation and controllable regional context synthesis to support incremental concept customization in diffusion models while mitigating catastrophic forgetting and concept neglect.
DrawVideo is a sketch-guided framework that decomposes long videos into controllable shots using keyframe sketches, appearance prompts, and motion prompts, supported by a new SketchLongVideo dataset.
Parallel, role-specialized prompt agents improve cultural relevance in text-to-video generation, with a new cross-cultural benchmark showing the largest gains for location cues.
CineAGI is a multi-agent LLM framework that generates multi-scene movies with improved character consistency, narrative coherence, and audio-visual alignment.
Tool-validated LLM agents author formal event-graph story specifications that a deterministic game engine executes into multi-actor videos with perfect annotations, reaching 80% seeded end-to-end success versus 0% for a staged LLM pipeline.
ProPhy adds explicit physics-aware conditioning via semantic and refinement experts plus VLM knowledge transfer to produce more physically coherent dynamic videos than prior methods.
Rolling Forcing generates multi-minute videos in real time by jointly denoising frames at increasing noise levels, anchoring attention to early frames, and using windowed distillation to limit error accumulation.
An open-source causal video world model sustains hour-long, 720p/60fps interactive generation without visual drift, paired with a VLM-based director-pilot agentic harness for rich, open-ended interaction.
ViMax coordinates screenwriting, shot-planning, character-styling, video-generation, and VLM-judge agents with hierarchical RAG planning and graph-based visual dependencies to generate coherent long-form multi-shot videos.
A hierarchical multi-agent framework converts a single sentence into a short drama using debate-based scripting, 3D-grounded first frames for spatial consistency, and multi-stage reviewer loops.
A self-reasoning agentic framework constructs a Product Narrative Framework, generates constraint-aware unified grid collages, and refines outputs via failure attribution to improve narrative coherence and aesthetics over direct prompting.
citing papers explorer
-
KathaTrace: Diagnosing Semantic Trajectory Collapse in Generated Visual Narratives
Introduces KathaTrace protocol and KathaBench-25K benchmark to quantify Semantic Trajectory Gap (STG) as the loss of transition meaning in visualized narratives, reporting STG of 23.5 +/- 1.3 across generators.
-
EM-Vid: Training-Free Entity-Centric Memory for Efficient and Consistent Multi-Shot Video Generation
EM-Vid introduces an entity-centric latent patch memory bank with sparse token conditioning and budgeted updates for training-free consistent multi-shot video generation.
-
CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives
CausalCine enables real-time causal autoregressive multi-shot video generation via multi-shot training, content-aware memory routing for coherence, and distillation to few-step inference.
-
GenHSI: Controllable Generation of Human-Scene Interaction Videos
GenHSI is a training-free three-stage pipeline that turns a scene image, character image, and complex HSI prompt into long videos with plausible chained interactions by generating atomic actions, 3D keyframes via 2D inpainting plus optimization, and then feeding them to pre-trained video diffusion.
-
DramaDirector: Geometry-Guided Short Drama Generation
Geometry-indexed depth–pose retrieval plus schema SFT and GRPO planning improves faithfulness, consistency, and controllability of plot-to-short-drama video generation over multi-agent and text-only baselines.
-
GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Scheduling
A training-free framework that reorders shot generation and maintains per-entity visual memory improves cross-shot character, object, and scene consistency over narrative-order memory baselines.
-
Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization
CCDM uses attribute-decoupled LoRA with relevance-guided aggregation and controllable regional context synthesis to support incremental concept customization in diffusion models while mitigating catastrophic forgetting and concept neglect.
-
DrawVideo: Generating Long Video from Storyboard Keyframe Sketches
DrawVideo is a sketch-guided framework that decomposes long videos into controllable shots using keyframe sketches, appearance prompts, and motion prompts, supported by a new SketchLongVideo dataset.
-
When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generation
Parallel, role-specialized prompt agents improve cultural relevance in text-to-video generation, with a new cross-cultural benchmark showing the largest gains for location cues.
-
CineAGI: Character-Consistent Movie Creation through LLM-Orchestrated Multi-Modal Generation and Cross-Scene Integration
CineAGI is a multi-agent LLM framework that generates multi-scene movies with improved character consistency, narrative coherence, and audio-visual alignment.
-
Authoring for Living Worlds: Tool-Constrained LLM Agents for Executable Multi-Actor Scenarios
Tool-validated LLM agents author formal event-graph story specifications that a deterministic game engine executes into multi-actor videos with perfect annotations, reaching 80% seeded end-to-end success versus 0% for a staged LLM pipeline.
-
ProPhy: Progressive Physical Alignment for Dynamic World Simulation
ProPhy adds explicit physics-aware conditioning via semantic and refinement experts plus VLM knowledge transfer to produce more physically coherent dynamic videos than prior methods.
-
Rolling Forcing: Autoregressive Long Video Diffusion in Real Time
Rolling Forcing generates multi-minute videos in real time by jointly denoising frames at increasing noise levels, anchoring attention to early frames, and using windowed distillation to limit error accumulation.
-
Infinite Worlds with Versatile Interactions
An open-source causal video world model sustains hour-long, 720p/60fps interactive generation without visual drift, paired with a VLM-based director-pilot agentic harness for rich, open-ended interaction.
-
ViMax: Agentic Video Generation
ViMax coordinates screenwriting, shot-planning, character-styling, video-generation, and VLM-judge agents with hierarchical RAG planning and graph-based visual dependencies to generate coherent long-form multi-shot videos.
-
One Sentence, One Drama: Personalized Short-Form Drama Generation via Multi-Agent Systems
A hierarchical multi-agent framework converts a single sentence into a short drama using debate-based scripting, 3D-grounded first frames for spatial consistency, and multi-stage reviewer loops.
-
Self-Reasoning Agentic Framework for Narrative Product Grid-Collage Generation
A self-reasoning agentic framework constructs a Product Narrative Framework, generates constraint-aware unified grid collages, and refines outputs via failure attribution to improve narrative coherence and aesthetics over direct prompting.