REVIEW 10 cited by
TaleCrafter: Interactive Story Visualization with Multiple Characters
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Accurate Story visualization requires several necessary elements, such as identity consistency across frames, the alignment between plain text and visual content, and a reasonable layout of objects in images. Most previous works endeavor to meet these requirements by fitting a text-to-image (T2I) model on a set of videos in the same style and with the same characters, e.g., the FlintstonesSV dataset. However, the learned T2I models typically struggle to adapt to new characters, scenes, and styles, and often lack the flexibility to revise the layout of the synthesized images. This paper proposes a system for generic interactive story visualization, capable of handling multiple novel characters and supporting the editing of layout and local structure. It is developed by leveraging the prior knowledge of large language and T2I models, trained on massive corpora. The system comprises four interconnected components: story-to-prompt generation (S2P), text-to-layout generation (T2L), controllable text-to-image generation (C-T2I), and image-to-video animation (I2V). First, the S2P module converts concise story information into detailed prompts required for subsequent stages. Next, T2L generates diverse and reasonable layouts based on the prompts, offering users the ability to adjust and refine the layout to their preference. The core component, C-T2I, enables the creation of images guided by layouts, sketches, and actor-specific identifiers to maintain consistency and detail across visualizations. Finally, I2V enriches the visualization process by animating the generated images. Extensive experiments and a user study are conducted to validate the effectiveness and flexibility of interactive editing of the proposed system.
Forward citations
Cited by 10 Pith papers
-
ShotPlan: Cinematic Video Generation with Learnable Planning Token
Learnable planning tokens with fractional positional timestamps let one diffusion pass generate multi-shot video with frame-accurate cuts and timed camera motion.
-
Plot'n Polish: Zero-shot Story Visualization and Disentangled Editing with Text-to-Image Diffusion Models
A zero-shot method combines grid priors, depth-conditioned ControlNet, and latent blending to generate and consistently edit story visualizations across multiple frames.
-
FairyGen: Storied Cartoon Video from a Single Child-Drawn Character
A pipeline that generates story-driven cartoon videos from one child-drawn character by separating foreground style, background synthesis, and motion learning.
-
Audit & Repair: An Agentic Framework for Consistent Story Visualization in Text-to-Image Diffusion Models
A multi-agent system that audits story images with a vision-language model and repairs inconsistencies with targeted diffusion edits improves multi-panel consistency over existing story visualization methods.
-
AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generation
AnimeShooter provides hierarchical story and shot annotations plus reference images for 148K one-minute animation stories, and AnimeShooterGen trained on it shows improved cross-shot consistency.
-
MV-Crafter: An Intelligent System for Music-guided Video Generation
MV-Crafter generates beat-synchronized music videos from music and a text theme by combining LLM-based scripting, diffusion video generation, and a dynamic beat-matching warping algorithm.
-
Manga Generation via Layout-controllable Diffusion
The authors build the Manga109Story dataset and a layout-controllable diffusion model that generates a full multi-panel manga page from a text story, with mixed quantitative evidence.
-
Mind the Time: Temporally-Controlled Multi-Event Video Generation
MinT generates multi-event videos where each event's timing is controlled by the user, using a fine-tuned video diffusion transformer with a time-aware rotary position embedding.
-
StorySync: Training-Free Subject Consistency in Text-to-Image Generation via Region Harmonization
A training-free inference-time pipeline uses masked cross-image attention sharing and region harmonization to keep subjects consistent across generated story images.
-
Do We Need to Design Specific Diffusion Models for Different Tasks? Try ONE-PIC
ONE-PIC unifies diffusion fine-tuning tasks by arranging source and target images into a single visual context and training SDXL with LoRA to predict only the masked target region.
Discussion (0). Continue with ORCID to comment.