Pith. sign in

REVIEW 4 major objections 4 minor 66 references

MagicAnime: A Hierarchically Annotated, Multimodal and Multitasking Dataset with Benchmarks for Cartoon Animation Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 400k-clip cartoon dataset with hierarchical multimodal annotations and benchmarks aims to improve AI-driven animation generation.

desk verdict A potentially useful cartoon dataset, but the finetuning evidence is one row thick and the table has a suspicious duplicate. read the letter →

arxiv 2507.20368 v1 pith:K4YVMN2T submitted 2025-07-27 cs.CV cs.MM

Shuolin Xu , Bingyuan Wang , Zeyu Cai , Fangteng Fu , Yue Ma , Tongyi Lee , Hongchuan Yu , Zeyu Wang This is my paper
classification cs.CVcs.MM
keywords cartoonanimationgenerationmultimodaldatasethierarchicalannotationvideobenchmarkaudio-drivenfacialpose-drivenimage-to-videotext-to-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces MagicAnime, a dataset of more than 400,000 cartoon video clips with hierarchical multimodal annotations, plus a benchmark suite called MagicAnime-Bench covering audio-driven facial animation, face reenactment, image-to-video generation, pose-driven animation, and text-to-video generation. The central claim is that this resource helps close the gap between real-world video data and stylized cartoon animation, where existing models often produce color mismatches, lost facial features, and jitter. The authors report that fine-tuning a representative portrait-animation model on their audio subset improves quantitative fidelity metrics, and they present this as evidence that the dataset supports higher-quality cartoon animation generation. If the claim holds, the dataset provides a shared training and evaluation ground for multimodal cartoon video generation.

What carries the argument

The load-bearing mechanism is a hierarchical filtering and annotation pipeline that uses human-pose and facial-keypoint detection as both a filter and an annotator. Full-body 133-point keypoints are detected first; clips with no detectable pose are discarded, and the same keypoints become the pose-driven annotation. Face clips are then cropped from the 68 facial landmarks, side profiles are removed, and the strictest audio subset additionally requires temporally aligned clean speech. Manual curation selects benchmark cases with controlled style proportions and difficulty levels. This pipeline lets one raw video collection yield four task-specific subsets with linked annotations.

What would settle it

Compute the pose-detection retention rate separately for humanoid and non-humanoid cartoon characters by labeling a random sample of source clips; if non-humanoid characters are dropped at a much higher rate, the dataset is biased toward human-like designs. Additionally, retrain the finetuned AniPortrait on a training split that excludes the films used for the benchmark and compare scores, to see whether the reported gains persist across films.

Watch

Extended reading notes

Core claim

The paper claims that a hierarchically annotated corpus built from 137 animated films—roughly 60 percent Japanese, 30 percent American, and 10 percent Chinese, matching global production shares—supports four generation tasks simultaneously. The dataset is organized into nested subsets: 400k clips for image-to-video, 50k clips with 133-point whole-body keypoints and motion-focused text for pose-driven and text-to-video, 12k face clips with 68-point keypoints for face reenactment, and 2.9k audio-aligned face clips for audio-driven facial animation. Benchmarks are manually curated with controlled style proportions and difficulty levels tied to motion complexity and degree of anthropomorphism. Quantitative results show that fine-tuning AniPortrait on the audio subset raises PSNR from 15.780 to 16.251 and lowers LPIPS from 0.349 to 0.323, which the paper takes as evidence that the dataset improves cartoon animation quality.

Load-bearing premise

The pipeline assumes that human-trained pose and face keypoint detectors reliably find characters in stylized cartoon frames, so clips that fail detection—often the most exaggerated or non-human characters—are silently dropped from the dataset.

Editorial extensions

If this is right

  • Fine-tuning on the MagicAnime audio-portrait subset improves quantitative fidelity metrics over the un-finetuned AniPortrait baseline.
  • Existing video generation models show a wide range of reliability on cartoon benchmarks, with valid sample ratios from 20 percent for EchoMimic audio to 100 percent for LivePortrait and FADM face reenactment.
  • The face-reenactment and audio-driven benchmarks share the same 100 manually selected clips, making results across the two tasks directly comparable.
  • The dataset style proportions reflect the global animation market, allowing cross-style performance comparisons among Japanese, American, and Chinese animation.
  • The 50k pose-and-text subset doubles as a text-to-video set, so motion-focused text descriptions can be evaluated independently of pose input.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If pose detectors silently fail on non-humanoid or exaggerated characters, the face and pose subsets likely underrepresent the very designs the paper identifies as hardest, meaning the measured gains may be concentrated on human-like characters.
  • Because benchmark clips are manually selected from the same 137 films that supply training clips, the reported fine-tuning gains may partly reflect in-distribution familiarity; a held-out film split would test true generalization.
  • The 400k image-to-video subset currently lacks text annotations, so the claim of multimodal support rests mostly on the smaller 50k subset; adding text to the full set would test whether multimodal advantages scale.
  • The same hierarchical keypoint pipeline could transfer to other stylized visual domains, such as comic art or game art, by replacing the backbone detectors with style-robust alternatives.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces MagicAnime, a large-scale cartoon animation dataset comprising 400k image-to-video clips, 50k pose-driven clips with keypoints and text, 12.08k face-reenactment clips, and 2.9k audio-driven facial animation clips, together with a benchmark suite (MagicAnime-Bench) covering several tasks. The authors describe a hierarchical annotation pipeline that uses pose and face keypoint detection to filter and label clips, and they report quantitative benchmark results for a set of existing video generation models on audio-driven facial animation, face reenactment, image-to-video generation, and frame interpolation. The paper concludes that finetuning on MagicAnime significantly improves animation quality, particularly for portrait audio-to-video generation.

Significance. If the dataset is released and the annotations verified, MagicAnime would fill a clear gap in cartoon animation resources, providing multimodal annotations (keypoints, audio, text) that existing datasets such as Sakuga-42M and AniSora lack. The hierarchical filtering-and-annotation pipeline is a reasonable design and the benchmark suite is potentially useful to the community. However, the central effectiveness claim is currently supported by a single controlled finetuning comparison on one model and one task, and several inconsistencies between the abstract and the experimental tables prevent the paper from substantiating its stated contributions.

major comments (4)
  1. [IV-A, Table II] The abstract and introduction state that the benchmarks cover four tasks including pose-driven character animation, but Table II and the task list in Section IV-A include only audio-driven facial animation, face reenactment, image-to-video generation, and frame interpolation. No pose-driven animation experiment appears anywhere in the paper, despite the pose-driven subset being one of the three core dataset components in Section III-B. The claimed comprehensive validation over four tasks is therefore incomplete.
  2. [IV-C, Table II] In the Face reenactment block, the before- and after-finetuning rows for AniPortrait are numerically identical (PSNR 24.648, SSIM 0.920, LPIPS 0.067, L1-loss 0.023, VSR 54%). This is the only controlled finetuning comparison reported for that task, and the identical numbers make the finetuning-improvement claim in Section IV-C and the conclusion unsupported. Setting this aside, the only remaining finetuning evidence is the audio-driven AniPortrait gain of 0.47 dB PSNR (15.780 to 16.251), reported without error bars, multiple seeds, or significance testing; this is too thin to support the statement that finetuning "significant improves animation quality."
  3. [III-A, IV-A] The benchmark clips are manually selected from the same 137 films that supply the training subsets, and the paper never documents a train/test separation at the source-film level. Consequently, the reported finetuning gains and baseline scores may reflect in-distribution performance on material used during finetuning rather than generalization to new cartoon content. The authors should either enforce disjoint source-film lists between the training subsets and the benchmark and report the exact split, or explicitly discuss the implications of the overlap for the measured improvements.
  4. [III-C1, III-B] The first filtering step applies large-scale human pose keypoint detection and retains only clips where a pose is detected, yet Section III-B acknowledges that face perception models fail on anthropomorphic characters that deviate significantly from human facial structures. This creates a systematic selection bias toward human-like characters and undermines the claim of supporting diverse, exaggerated cartoon content. The paper should quantify how many clips are rejected by the pose detector across the three animation styles and should verify that the benchmark's stated 12% non-humanoid samples actually pass the pose-based filter; otherwise the non-humanoid benchmark claim is not credible.
minor comments (4)
  1. [III-B] In the final sentence of Section III-B, "complexityy" is a typo and should read "complexity."
  2. [III-C2] The sentence beginning "I have designed a hierarchical data processing framework" should use the plural "we have designed" to match the rest of the paper.
  3. [Table II and IV-B] The metric label "L1-loss" is used without formal definition; Section IV-B refers to "Least Absolute Deviations" but the standard term is mean absolute error or L1 distance. Please define the metric precisely and state how it is computed.
  4. [V] The disclaimer limits dataset access to research institutions and through signed agreements. This is acceptable, but for reproducibility the authors should commit to releasing the benchmark test set and the exact training/benchmark split at least, so that reported numbers can be independently verified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: MagicAnime is an empirical dataset/benchmark paper; finetuning gains are measured, not fitted, and self-citations are not load-bearing.

full rationale

MagicAnime reports dataset construction, annotation statistics, and benchmark evaluations rather than a derived prediction. The finetuning claim rests on Table II, where AniPortrait shows a small measured PSNR gain on audio-driven facial animation (15.780 to 16.251) and an identical before/after row for face reenactment; the identical row weakens the headline claim but is a reporting or experimental issue, not circularity. Table I states that each subset 'includes data from the respective benchmark,' and benchmarks are manually selected from the same 137-film source pool as the training subsets, so in-distribution evaluation is a legitimate generalization concern; however, the paper finetunes on 2,900 Train samples while the corresponding benchmark has 100 cases, and no equation or fitted parameter forces the reported improvement to equal the training input by construction. The dataset filtering pipeline does use pose and face keypoint detectors as selection criteria, but this is a data-curation choice, not a self-referential derivation: the paper also explicitly concedes in Section III-B that face perception models 'fail to process such anthropomorphic characters.' Self-citations to the authors' prior work ([23], [26], [27], [46]) appear only as baselines or related work and are not load-bearing for the dataset's construction or for any stated theorem. No circular step meets the required bar of quoting a specific reduction of the claimed result to its own inputs.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new entities are postulated. The dataset and benchmarks repackage existing animated films and annotations; the "difficulty levels" in Figure 4 are a categorization, not an invented mechanism. The two free parameters are hand-chosen design thresholds, and the three axioms are domain assumptions about market share, detector validity, and audio alignment that the central claim depends on.

free parameters (2)
  • Style distribution (Japanese/American/Chinese) = 60% / 30% / <10%
    Hand-chosen proportions applied to every subset and benchmark, justified by an uncited market-share claim rather than derived from task performance.
  • Minimum resolution threshold = 60% of clips at 1080P or higher
    Hand-chosen quality target stated in Section III-A; no analysis shows why 60 percent or 1080P is the right operating point.
assumptions (3)
  • domain assumption Global animation production is roughly 60% Japanese, 30% American, and under 10% Chinese, so dataset subsets should follow this split.
    Section III-A asserts this from "industry research" without citation; the 60/30/10 split of every subset and benchmark depends on it.
  • domain assumption Human-trained pose and face keypoint detectors (Xpose) give valid detections on cartoon characters, so clips can be selected and filtered on detected poses.
    Section III-C2 uses keypoint detection as the primary automatic filter. Section III-B concedes detectors fail on anthropomorphic characters, partially contradicting the assumption.
  • domain assumption The source films' audio tracks are temporally aligned with character lip motion and can be separated from noise and other speakers.
    Section III-C2 imposes three strict audio criteria on the audio-driven subset. Section IV-D later explains weak audio effects by noting anime mouth shapes are only mechanically regular, qualifying this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MagicAnime: A Hierarchically Annotated, Multimodal and Multitasking Dataset with Benchmarks for Cartoon Animation Generation." pith.science (2026). https://pith.science/paper/K4YVMN2T

@misc{pith2026250720368,
  author       = {Pith},
  title        = {Pith review of: MagicAnime: A Hierarchically Annotated, Multimodal and Multitasking Dataset with Benchmarks for Cartoon Animation Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K4YVMN2T}},
  note         = {Machine review of arXiv:2507.20368}
}
read the original abstract

Generating high-quality cartoon animations multimodal control is challenging due to the complexity of non-human characters, stylistically diverse motions and fine-grained emotions. There is a huge domain gap between real-world videos and cartoon animation, as cartoon animation is usually abstract and has exaggerated motion. Meanwhile, public multimodal cartoon data are extremely scarce due to the difficulty of large-scale automatic annotation processes compared with real-life scenarios. To bridge this gap, We propose the MagicAnime dataset, a large-scale, hierarchically annotated, and multimodal dataset designed to support multiple video generation tasks, along with the benchmarks it includes. Containing 400k video clips for image-to-video generation, 50k pairs of video clips and keypoints for whole-body annotation, 12k pairs of video clips for video-to-video face animation, and 2.9k pairs of video and audio clips for audio-driven face animation. Meanwhile, we also build a set of multi-modal cartoon animation benchmarks, called MagicAnime-Bench, to support the comparisons of different methods in the tasks above. Comprehensive experiments on four tasks, including video-driven face animation, audio-driven face animation, image-to-video animation, and pose-driven character animation, validate its effectiveness in supporting high-fidelity, fine-grained, and controllable generation.

Figures

Figures reproduced from arXiv: 2507.20368 by the authors.

Figure 1
Figure 1. Various failure cases on cartoon animation generation tasks: (a) Image2Video on these methods [2], [11], [21] we evaluated the qualitative performance [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Dataset overview: (a) Dataset modalities. Our dataset consists of different modalities including whole body and face keypoint frame-by-frame, video, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Dataset construction pipeline and overviews. The overall framework and screening process of our dataset is hierarchical The key innovation of the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Examples of different levels of difficulty in our cartoon animation [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Comparison between ground truth (GT) videos, results generated by [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 48 canonical work pages

  1. [1]

    Keling Video Model

    Kuaishou, “Keling Video Model”, 2024. https://kling.kuaishou.com/

    work page 2024

  2. [2]

    DynamiCrafter: Animating Open-Domain Images with Video Diffusion Priors

    Xing, Jinbo et al., “DynamiCrafter: Animating Open-Domain Images with Video Diffusion Priors”, in Proceedings of the European Conference on Computer Vision (ECCV) , pages 399–417, 2024

    work page 2024

  3. [3]

    Sora: Creating Video from Text

    OpenAI, “Sora: Creating Video from Text”, 2024. https://openai.com/ sora/

    work page 2024

  4. [4]

    HunyuanVideo: A Systematic Framework for Large Video Generative Models

    Kong, Weijie et al., “HunyuanVideo: A Systematic Framework for Large Video Generative Models”, arXiv preprint arXiv:2412.03603 , 2024

    arXiv 2024

  5. [5]

    W AN: Open and Advanced Large-Scale Video Generative Models

    Wang, Ang et al., “W AN: Open and Advanced Large-Scale Video Generative Models”, arXiv preprint arXiv:2503.20314 , 2025

    arXiv 2025

  6. [6]

    SadTalker: Learning Realistic 3D Motion Coeffi- cients for Stylized Audio-Driven Single Image Talking Face Animation

    Zhang, Wenxuan et al., “SadTalker: Learning Realistic 3D Motion Coeffi- cients for Stylized Audio-Driven Single Image Talking Face Animation”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8652–8661, 2023

    work page 2023

  7. [7]

    AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation

    Wei, Huawei et al., “AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation”, arXiv preprint arXiv:2403.17694 , 2024

    arXiv 2024

  8. [8]

    Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation

    Xu, Mingwang et al., “Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation”, arXiv preprint arXiv:2406.08801 , 2024

    arXiv 2024

Show all 66 references
  1. [9]

    Hallo2: Long-Duration and High-Resolution Audio- Driven Portrait Image Animation

    Cui, Jiahao et al., “Hallo2: Long-Duration and High-Resolution Audio- Driven Portrait Image Animation”, arXiv preprint arXiv:2410.07718 , 2024

    2024 arXiv

  2. [10]

    Moore-AnimateAnyone

    Moore Threads, “Moore-AnimateAnyone”, 2024. https://github.com/ MooreThreads/Moore-AnimateAnyone

    2024

  3. [11]

    ToonCrafter: Generative Cartoon Interpolation

    Xing, Jinbo et al., “ToonCrafter: Generative Cartoon Interpolation”, ACM Transactions on Graphics (TOG), vol. 43, no. 6, pages 1–11, 2024

    2024

  4. [12]

    AniDoc: Animation Creation Made Easier

    Meng, Yihao et al., “AniDoc: Animation Creation Made Easier”, arXiv preprint arXiv:2412.14173, 2024

    2024 arXiv

  5. [13]

    MangaNinja: Line Art Colorization with Precise Reference Following

    Liu, Zhiheng et al., “MangaNinja: Line Art Colorization with Precise Reference Following”, arXiv preprint arXiv:2501.08332 , 2025

    2025 arXiv

  6. [14]

    MikuDance: Animating Character Art with Mixed Motion Dynamics

    Zhang, Jiaxu et al., “MikuDance: Animating Character Art with Mixed Motion Dynamics”, arXiv preprint arXiv:2411.08656 , 2024

    2024 arXiv

  7. [15]

    Cartoon Image Processing: A Survey

    Zhao, Yang et al., “Cartoon Image Processing: A Survey”, International Journal of Computer Vision , vol. 130, no. 11, pages 2733–2769, 2022

    2022

  8. [16]

    LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control

    Guo, Jianzhu et al., “LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control”, arXiv preprint arXiv:2407.03168 , 2024

    2024 arXiv

  9. [17]

    OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation

    Li, Hui et al., “OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation”, arXiv preprint arXiv:2412.00115, 2024

    2024 arXiv

  10. [18]

    VBench: Comprehensive Benchmark Suite for Video Generative Models

    Huang, Ziqi et al., “VBench: Comprehensive Benchmark Suite for Video Generative Models”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 21807–21818, 2024

    2024

  11. [19]

    Sakuga-42M Dataset: Scaling Up Cartoon Re- search

    Pan, Zhenglin et al., “Sakuga-42M Dataset: Scaling Up Cartoon Re- search”, arXiv preprint arXiv:2405.07425 , 2024

    2024 arXiv

  12. [20]

    Exploring the Frontiers of Animation Video Generation in the Sora Era: Method, Dataset and Benchmark

    Jiang, Yudong et al., “Exploring the Frontiers of Animation Video Generation in the Sora Era: Method, Dataset and Benchmark”, arXiv preprint arXiv:2412.10255, 2024

    2024 arXiv

  13. [21]

    Generating Videos with Scene Dynamics

    V ondrick, Carl et al., “Generating Videos with Scene Dynamics”, Advances in Neural Information Processing Systems (NeurIPS) , vol. 29, 2016

    2016

  14. [22]

    EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions

    Chen, Zhiyuan et al., “EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions”, arXiv preprint arXiv:2407.08136, 2024

    2024 arXiv

  15. [23]

    Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation

    Ma, Yue et al., “Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation”, in Proceedings of the SIGGRAPH Asia Conference, pages 1–12, 2024

    2024

  16. [24]

    VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution

    Xie, Liangbin et al., “VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 657–666, 2022

    2022

  17. [25]

    CelebV-HQ: A Large-Scale Video Facial Attributes Dataset

    Zhu, Hao et al., “CelebV-HQ: A Large-Scale Video Facial Attributes Dataset”, in Proceedings of the European Conference on Computer Vision (ECCV), pages 650–667, 2022

    2022

  18. [26]

    Controllable Video Generation: A Survey

    Ma, Yue et al., “Controllable Video Generation: A Survey”, arXiv preprint arXiv:2507.16869, 2025

    2025

  19. [27]

    Diffusion-based visual art creation: A survey and new perspectives

    Wang, Bingyuan et al., “Diffusion-based visual art creation: A survey and new perspectives”, ACM Computing Surveys , vol. 57, no. 10, pages 1–37, 2025

    2025

  20. [28]

    High-Resolution Image Synthesis with Latent Diffusion Models

    Rombach, Robin et al., “High-Resolution Image Synthesis with Latent Diffusion Models”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10684–10695, 2022

    2022

  21. [29]

    U-Net: Convolutional Networks for Biomed- ical Image Segmentation

    Ronneberger, Olaf et al., “U-Net: Convolutional Networks for Biomed- ical Image Segmentation”, in Medical Image Computing and Computer- Assisted Intervention (MICCAI) , pages 234–241, 2015

    2015

  22. [30]

    Denoising Diffusion Implicit Models

    Song, Jiaming et al., “Denoising Diffusion Implicit Models”, arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  23. [31]

    Video Diffusion Models

    Ho, Jonathan et al., “Video Diffusion Models”, Advances in Neural Information Processing Systems (NeurIPS) , vol. 35, pages 8633–8646, 2022

    2022

  24. [32]

    AnimateDiff: Animate Your Personalized Text- to-Image Diffusion Models without Specific Tuning

    Guo, Yuwei et al., “AnimateDiff: Animate Your Personalized Text- to-Image Diffusion Models without Specific Tuning”, in International Conference on Learning Representations (ICLR) , 2024

    2024

  25. [33]

    Scalable Diffusion Models with Transformers

    Peebles, William and Xie, Saining, “Scalable Diffusion Models with Transformers”, in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 4195–4205, 2023

    2023

  26. [34]

    PixArt- α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

    Chen, Junsong et al., “PixArt- α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis”, in International Conference on Learning Representations (ICLR) , 2024

    2024

  27. [35]

    CogVideo: Large-Scale Pretraining for Text- to-Video Generation via Transformers

    Hong, Wenyi et al., “CogVideo: Large-Scale Pretraining for Text- to-Video Generation via Transformers”, in The Eleventh International Conference on Learning Representations (ICLR) , 2023

    2023

  28. [36]

    Dit4edit: Diffusion transformer for image editing

    Feng, Kunyu et al., “Dit4edit: Diffusion transformer for image editing”, in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 39, no. 3, pages 2969–2977, 2025

    2025

  29. [37]

    Scaling Rectified Flow Transformers for High- Resolution Image Synthesis

    Esser, Patrick et al., “Scaling Rectified Flow Transformers for High- Resolution Image Synthesis”, in Forty-First International Conference on Machine Learning (ICML) , 2024

    2024

  30. [38]

    Auto-Encoding Variational Bayes

    Kingma, Diederik P. and Welling, Max, “Auto-Encoding Variational Bayes”, in International Conference on Learning Representations (ICLR), 2014

    2014

  31. [39]

    Neural Discrete Representation Learning

    Van Den Oord, Aaron et al., “Neural Discrete Representation Learning”, Advances in Neural Information Processing Systems (NeurIPS) , vol. 30, 2017

    2017

  32. [40]

    Taming Transformers for High-Resolution Image Synthesis

    Esser, Patrick et al., “Taming Transformers for High-Resolution Image Synthesis”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 12873–12883, 2021

    2021

  33. [41]

    Ltx-video: Realtime video latent diffusion

    HaCohen, Yoav et al., “Ltx-video: Realtime video latent diffusion”, arXiv preprint arXiv:2501.00103 , 2024

    2024 arXiv

  34. [42]

    Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

    Raffel, Colin et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer”, Journal of Machine Learning Re- search, vol. 21, no. 140, pages 1–67, 2020

    2020

  35. [43]

    Learning Transferable Visual Models from Natural Language Supervision

    Radford, Alec et al., “Learning Transferable Visual Models from Natural Language Supervision”, in International Conference on Machine Learn- ing (ICML), pages 8748–8763, 2021

    2021

  36. [44]

    Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets

    Blattmann, Andreas et al., “Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets”, arXiv preprint arXiv:2311.15127 , 2023

    2023 arXiv

  37. [45]

    Magiccolor: Multi-instance sketch colorization

    Zhang, Yinhan et al., “Magiccolor: Multi-instance sketch colorization”, arXiv preprint arXiv:2503.16948 , 2025

    2025 arXiv

  38. [46]

    MagicScroll: Enhancing immersive storytelling with controllable scroll image generation

    Wang, Bingyuan et al., “MagicScroll: Enhancing immersive storytelling with controllable scroll image generation”, in 2025 IEEE Conference Virtual Reality and 3D User Interfaces (VR) , pages 431–441, 2025

    2025

  39. [47]

    From expanded cinema to extended reality: How AI can expand and extend cinematic experiences

    Song, Junrong et al., “From expanded cinema to extended reality: How AI can expand and extend cinematic experiences”, in Proceedings of the 16th International Symposium on Visual Information Communication and Interaction, pages 1–5, 2023

    2023

  40. [48]

    Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation

    Hu, Li, “Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8153–8163, 2024

    2024

  41. [49]

    Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance

    Zhu, Shenhao et al., “Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance”, in European Conference on Computer Vision (ECCV) , pages 145–162, 2024

    2024

  42. [50]

    HumanVid: Demystifying Training Data for Camera-Controllable Human Image Animation

    Wang, Zhenzhi et al., “HumanVid: Demystifying Training Data for Camera-Controllable Human Image Animation”, in Conference on Neural Information Processing Systems (NeurIPS) , 2024

    2024

  43. [51]

    Adaptive Super Resolution for One-Shot Talking- Head Generation

    Song, Luchuan et al., “Adaptive Super Resolution for One-Shot Talking- Head Generation”, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 4115–4119, 2024. 10

    2024

  44. [52]

    Talking Face Video Generation with Editable Expression

    Song, Luchuan et al., “Talking Face Video Generation with Editable Expression”, in International Conference on Image and Graphics (ICIG) , pages 753–764, 2021

    2021

  45. [53]

    Emo-Avatar: Efficient Monocular Video Style Avatar through Texture Rendering

    Liu, Pinxin et al., “Emo-Avatar: Efficient Monocular Video Style Avatar through Texture Rendering”, arXiv preprint arXiv:2402.00827 , 2024

    2024 arXiv

  46. [54]

    TextToon: Real-Time Text toonify Head Avatar from Single Video

    Song, Luchuan et al., “TextToon: Real-Time Text toonify Head Avatar from Single Video”, in SIGGRAPH Asia Conference Papers, pages 1–11, 2024

    2024

  47. [55]

    Enhanced Deep Animation Video Interpolation

    Shen, Wang et al., “Enhanced Deep Animation Video Interpolation”, in IEEE International Conference on Image Processing (ICIP), pages 31–35, 2022

    2022

  48. [56]

    Deep Animation Video Interpolation in the Wild

    Siyao, Li et al., “Deep Animation Video Interpolation in the Wild”, in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6587–6595, 2021

    2021

  49. [57]

    AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos

    Wu, Yanze et al., “AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos”, Advances in Neural Information Pro- cessing Systems (NeurIPS) , vol. 35, pages 11241–11252, 2022

    2022

  50. [58]

    Panda-70M: Captioning 70M Videos with Mul- tiple Cross-Modality Teachers

    Chen, Tsai-Shien et al., “Panda-70M: Captioning 70M Videos with Mul- tiple Cross-Modality Teachers”, in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 13320–13331, 2024

    2024

  51. [59]

    InternVid: A Large-Scale Video-Text Dataset for Multimodal Understanding and Generation

    Wang, Yi et al., “InternVid: A Large-Scale Video-Text Dataset for Multimodal Understanding and Generation”, in International Conference on Learning Representations (ICLR) , 2024

    2024

  52. [60]

    PySceneDetect,

    Castellano, Brandon, “PySceneDetect,” GitHub Repository, 2022. Avail- able at: https://github.com/Breakthrough/PySceneDetect. Accessed: 2024- 06-05

    2022

  53. [61]

    Dinov2: Learning Robust Visual Features Without Supervision,

    Oquab, Maxime et al., “Dinov2: Learning Robust Visual Features Without Supervision,” arXiv preprint arXiv:2304.07193 , 2023

    2023 arXiv

  54. [62]

    Face Animation with an Attribute-Guided Diffusion Model

    Zeng, Bohan et al., “Face Animation with an Attribute-Guided Diffusion Model”, in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 628–637, 2023

    2023

  55. [63]

    Image Quality Metrics: PSNR vs. SSIM

    Hore, Alain and Ziou, Djemel, “Image Quality Metrics: PSNR vs. SSIM”, in International Conference on Pattern Recognition (ICPR), pages 2366–2369, 2010

    2010

  56. [64]

    Image Quality Assessment: From Error Visibility to Structural Similarity

    Wang, Zhou et al., “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Transactions on Image Processing , vol. 13, no. 4, pages 600–612, 2004

    2004

  57. [65]

    The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

    Zhang, Richard et al., “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric”, in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 586–595, 2018

    2018

  58. [66]

    Regression Shrinkage and Selection via the Lasso

    Tibshirani, Robert, “Regression Shrinkage and Selection via the Lasso”, Journal of the Royal Statistical Society Series B: Statistical Methodology , vol. 58, no. 1, pages 267–288, 1996

    1996

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.