Pith. sign in

REVIEW 3 major objections 2 minor 99 references

Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generation

T0 review · 3 major / 2 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Layout conditions make AI story characters stay consistent.

desk verdict A promising infrastructure paper—new task, dataset, benchmark—but the headline SOTA claim is unverifiable from the abstract and the self-built benchmark raises a train/test contamination question. read the letter →

arxiv 2508.08949 v1 pith:LSAQOVPW submitted 2025-08-12 cs.CV

Ao Ma , Jiasong Feng , Ke Cao , Jing Wang , Yun Wang , Quanwei Zhang , Zhanjie Zhang This is my paper · ORCID
classification cs.CV
keywords storygenerationlayoutconditioningDiffusionTransformerssubjectconsistencycartoonvideodatasetlayout-to-togglestorytellingbenchmarktext-to-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that explicit layout conditions—where and how the subject appears, what it wears, its expression and posture—are the missing control signal for consistent AI story generation. It argues that layout conditions let a model coordinate fine-grained interactions between frames, fixing the subject-consistency failures of training-free and text-only methods. To support this, it builds Lay2Story-1M, over a million high-resolution cartoon images with layout annotations derived from about 11,300 hours of video, and Lay2Story-Bench, 3,000 prompts for comparison. On that benchmark, its Lay2Story framework, built on Diffusion Transformers, reports the best consistency, semantic correlation, and aesthetic quality among compared methods.

What carries the argument

The load-bearing object is the layout-conditioned Diffusion Transformer: a diffusion model whose transformer backbone receives layout tokens encoding each subject's position and detailed attributes, so the denoising process can coordinate cross-frame interactions. Around it, the paper places Lay2Story-1M, the dataset of layout-annotated cartoon frames that supplies the training signal, and Lay2Story-Bench, the 3,000-prompt benchmark that supplies the comparison. The layout tokens are what carry the argument: they are the mechanism claimed to turn weak text-level consistency into fine-grained, controllable consistency.

What would settle it

Run the same methods on an independent story-generation benchmark whose prompts carry no layout annotations, and have blind human raters score consistency and storytelling; if the gap between Lay2Story and prior methods disappears, the claim that layout conditions drive the improvement is falsified. A second check: generate stories about subjects and art styles absent from Lay2Story-1M's cartoon videos; a large drop in consistency would indicate the result does not generalize beyond the training distribution.

Watch

Extended reading notes

Core claim

The paper's central claim is that layout-to-toggle storytelling is a well-posed task and that layout conditions, not just text prompts, are what allow a generative model to keep a subject consistent across frames while letting the user control position, appearance, clothing, expression, and posture. Building on this, the authors construct a DiT-based framework, Lay2Story, that takes layout conditions as extra input, and they introduce a large dataset and benchmark to train and measure it. Their experiments claim that this approach outperforms previous state-of-the-art methods on consistency, semantic correlation, and aesthetic quality.

Load-bearing premise

The central performance claim depends on Lay2Story-Bench being a fair and representative test of story generation quality; if the benchmark is tilted toward layout-conditioned outputs, the reported state-of-the-art advantage may not hold in everyday storytelling use.

Editorial extensions

If this is right

  • Users could toggle a subject's position or appearance in a story prompt and get a frame sequence that keeps the subject recognizable, enabling iterative storyboarding.
  • Layout-conditioned training could reduce the need for inference-time tricks to maintain identity, since consistency is built into the generation process itself.
  • Lay2Story-1M gives the research community a large, layout-annotated cartoon resource, lowering the data barrier for layout-conditioned story generation.
  • Lay2Story-Bench offers a 3,000-prompt common testbed for comparing future story-generation methods on consistency, semantic correlation, and aesthetics.
  • If layout conditions are the key to subject consistency, the same conditioning strategy could be transferred to other generative backbones and video domains, not just cartoon images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If layout tokens are the true consistency mechanism, similar conditioning could be added to other generative backbones, such as autoregressive or masked image models, with the same benefit—an extension the paper does not test.
  • Because the benchmark and dataset are built by the same team from cartoon video, the claimed advantage may be strongest within the cartoon domain; testing on live-action or stylized non-cartoon footage would show how far the finding generalizes.
  • The 'togglable' framing suggests an interactive editing workflow: a user could change one layout attribute, such as the subject's position, and re-run generation to get a consistent alternate story; the paper describes this capability but does not user-test it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The abstract-only submission proposes a new task, Layout-Togglable Storytelling, in which layout conditions (position, appearance, clothing, expression, posture) are used to control a diffusion-transformer-based story generation model. The authors introduce a 1M-image dataset (Lay2Story-1M) derived from roughly 11,300 hours of cartoon videos, a 3,000-prompt benchmark (Lay2Story-Bench) built from the same source, and a framework (Lay2Story) based on Diffusion Transformers. The central claim is that Lay2Story outperforms previous state-of-the-art methods on this benchmark in subject consistency, semantic correlation, and aesthetic quality. No quantitative results, baseline names, metrics, ablations, or methodological details are provided in the submitted material.

Significance. If the claims are correct, the work would contribute a novel task formulation, a large-scale dataset with layout annotations, and a strong baseline method, which could be useful for the story-generation research community. The idea of using layout conditions to guide inter-frame interaction is plausible and interesting. However, the lack of any experimental evidence in the submitted manuscript makes it impossible to verify the central performance claim. The paper cannot currently be judged as a significant advance because the only evidence offered is the abstract's assertion of superiority, and the benchmark is self-constructed from the same data distribution as the training set, raising standard generalization concerns.

major comments (3)
  1. [Abstract] The abstract claims that the proposed method 'outperforms the previous state-of-the-art techniques' in consistency, semantic correlation, and aesthetic quality, but it reports no quantitative results, no error bars, no comparison baselines, and no evaluation protocol. The submitted manuscript contains only the abstract; the full text with method description, experiments, and tables is absent. As a result, the central claim is unverifiable from the provided material and the paper is not reproducible.
  2. [Abstract (Lay2Story-1M / Lay2Story-Bench)] Both the training data and the evaluation benchmark are derived from the same source, approximately 11,300 hours of cartoon videos. The abstract does not state whether the benchmark prompts and their corresponding subjects are disjoint from the training videos or characters. This creates a concrete train/test contamination risk: high subject-consistency scores on Lay2Story-Bench could reflect memorization of characters and scenes seen during training rather than generalization to new storytelling contexts. The authors should provide evidence of a subject-disjoint split, unseen character evaluation, or an explicit overlap-control protocol.
  3. [Abstract (task and metrics)] The paper introduces the 'Layout-Togglable Storytelling' task and claims best results in 'consistency, semantic correlation, and aesthetic quality,' but does not define how these qualities are measured, what layout annotations are used, how the benchmark prompts are generated, or which baseline methods are compared. Without definitions of the metrics and the exact evaluation setting, the claim cannot be reproduced or falsified, and the reader cannot assess whether the proposed method is genuinely better or merely tuned to the self-created benchmark.
minor comments (2)
  1. [Abstract] The name 'Lay2Story' is used for the method, the dataset, and the benchmark (Lay2Story-1M, Lay2Story-Bench, and the framework Lay2Story), which is confusing; distinct names or explicit disambiguation would improve clarity.
  2. [Abstract] The phrase 'previous state-of-the-art (SOTA) techniques' mentions no specific methods or citations; naming at least representative baselines would make the comparison claim more concrete.

Circularity Check

0 steps flagged · score 0.0 of 10

No definitional circularity found in the abstract; self-built benchmark is a validation concern, not a circular derivation.

full rationale

The abstract's central claim is an empirical performance comparison: the proposed Lay2Story method outperforms previous SOTA on Lay2Story-Bench in consistency, semantic correlation, and aesthetic quality. The paper introduces a new task, a dataset, a benchmark, and a method, but no equation or definition in the abstract makes the claimed result true by construction. The benchmark is indeed built from the same Lay2Story-1M data used for training, which raises a legitimate train/test contamination risk and limits external generalizability; however, that is a benchmark-design concern, not a circularity. The rules require exhibiting a specific reduction, such as an output metric being identical to an input fit or a prediction being the definition of a fitted parameter. No such reduction is visible in the abstract. The method is not said to be derived from the benchmark metric, the benchmark is not described as scoring the method by its own training objective, and no self-citation is invoked to justify the central claim. Accordingly, the appropriate finding is no significant circularity, and the risk highlighted by the reader is better classified as an evaluation-validity concern than as a circularity defect.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

Abstract-only review; no specific free parameters, hyperparameters, or invented entities can be identified. The listed axioms are the unverified background assumptions that support the central claim, all taken from statements in the abstract.

assumptions (3)
  • domain assumption Layout conditions provide sufficient guidance to enforce inter-frame interaction and subject consistency.
    The abstract states this as a demonstrated finding, but no proof or empirical comparison is provided in the abstract. It is the central modeling premise.
  • domain assumption The Lay2Story-1M dataset is of sufficient quality and diversity to train a controllable story generation model.
    The abstract claims high-quality 720p+ images from cartoon videos, but gives no details on filtering, annotation, or validation procedures.
  • domain assumption Diffusion Transformers are an appropriate base architecture for layout-conditioned generation.
    The abstract assumes the DiT architecture can be extended to this task without providing architectural justification or ablation support.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generation." pith.science (2026). https://pith.science/paper/LSAQOVPW

@misc{pith2026250808949,
  author       = {Pith},
  title        = {Pith review of: Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LSAQOVPW}},
  note         = {Machine review of arXiv:2508.08949}
}
read the original abstract

Storytelling tasks involving generating consistent subjects have gained significant attention recently. However, existing methods, whether training-free or training-based, continue to face challenges in maintaining subject consistency due to the lack of fine-grained guidance and inter-frame interaction. Additionally, the scarcity of high-quality data in this field makes it difficult to precisely control storytelling tasks, including the subject's position, appearance, clothing, expression, and posture, thereby hindering further advancements. In this paper, we demonstrate that layout conditions, such as the subject's position and detailed attributes, effectively facilitate fine-grained interactions between frames. This not only strengthens the consistency of the generated frame sequence but also allows for precise control over the subject's position, appearance, and other key details. Building on this, we introduce an advanced storytelling task: Layout-Togglable Storytelling, which enables precise subject control by incorporating layout conditions. To address the lack of high-quality datasets with layout annotations for this task, we develop Lay2Story-1M, which contains over 1 million 720p and higher-resolution images, processed from approximately 11,300 hours of cartoon videos. Building on Lay2Story-1M, we create Lay2Story-Bench, a benchmark with 3,000 prompts designed to evaluate the performance of different methods on this task. Furthermore, we propose Lay2Story, a robust framework based on the Diffusion Transformers (DiTs) architecture for Layout-Togglable Storytelling tasks. Through both qualitative and quantitative experiments, we find that our method outperforms the previous state-of-the-art (SOTA) techniques, achieving the best results in terms of consistency, semantic correlation, and aesthetic quality.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

99 extracted references · 31 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    The k-means algorithm: A comprehensive survey and performance evaluation

    Mohiuddin Ahmed, Raihan Seraj, and Syed Mohammed Shamsul Islam. The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, 9 0 (8): 0 1295, 2020

    2020

  3. [3]

    BlackForestlabs AI. Flux. https://github.com/black-forest-labs/flux, 2024. Accessed: March 6, 2025

    2024

  4. [4]

    Automatic story generation: A survey of approaches

    Arwa I Alhussain and Aqil M Azmi. Automatic story generation: A survey of approaches. ACM Computing Surveys (CSUR), 54 0 (5): 0 1--38, 2021

    2021

  5. [5]

    Using my artistic style? you must obtain my authorization

    Xiuli Bi, Haowei Liu, Weisheng Li, Bo Liu, and Bin Xiao. Using my artistic style? you must obtain my authorization. In European Conference on Computer Vision, pages 305--321. Springer, 2024

    2024

  6. [6]

    Customttt: Motion and appearance customized video generation via test-time training

    Xiuli Bi, Jian Lu, Bo Liu, Xiaodong Cun, Yong Zhang, Weisheng Li, and Bin Xiao. Customttt: Motion and appearance customized video generation via test-time training. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1871--1879, 2025

    2025

  7. [7]

    Relactrl: Relevance-guided efficient control for diffusion transformers

    Ke Cao, Jing Wang, Ao Ma, Jiasong Feng, Zhanjie Zhang, Xuanhua He, Shanyuan Liu, Bo Cheng, Dawei Leng, Yuhui Yin, et al. Relactrl: Relevance-guided efficient control for diffusion transformers. arXiv preprint arXiv:2502.14377, 2025

    arXiv 2025

  8. [8]

    Gamegen-x: Interactive open-world game video generation

    Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open-world game video generation. arXiv preprint arXiv:2411.00769, 2024

    arXiv 2024

Show all 99 references
  1. [9]

    Pixart- : Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart- : Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023

    arXiv

  2. [10]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In Proceedings of the IEEE/CVF Confere...

    2024

  3. [11]

    Ctr-driven advertising image generation with multimodal large language models

    Xingye Chen, Wei Feng, Zhenbang Du, Weizhen Wang, Yanyin Chen, Haohan Wang, Linkai Liu, Yaoyu Li, Jinyuan Zhao, Yu Li, et al. Ctr-driven advertising image generation with multimodal large language models. In Proceedings of the ACM on Web Conference 2025, pages 2262--2275, 2025

    2025

  4. [12]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on mach...

    2024

  5. [13]

    Fancyvideo: Towards dynamic and consistent video generation via cross-frame textual guidance

    Jiasong Feng, Ao Ma, Jing Wang, Bo Cheng, Xiaodan Liang, Dawei Leng, and Yuhui Yin. Fancyvideo: Towards dynamic and consistent video generation via cross-frame textual guidance. arXiv preprint arXiv:2408.08189, 2024

    2024 arXiv

  6. [14]

    Dreamsim: Learning new dimensions of human visual similarity using synthetic data

    Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. arXiv preprint arXiv:2306.09344, 2023

    2023 arXiv

  7. [15]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

    2022 arXiv

  8. [16]

    Check locate rectify: A training-free layout calibration system for text-to-image generation

    Biao Gong, Siteng Huang, Yutong Feng, Shiwei Zhang, Yuyuan Li, and Yu Liu. Check locate rectify: A training-free layout calibration system for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6624--6634, 2024

    2024

  9. [17]

    Variational autoencoder: An unsupervised model for encoding and decoding fmri activity in visual cortex

    Kuan Han, Haiguang Wen, Junxing Shi, Kun-Han Lu, Yizhen Zhang, Di Fu, and Zhongming Liu. Variational autoencoder: An unsupervised model for encoding and decoding fmri activity in visual cortex. NeuroImage, 198: 0 125--136, 2019

    2019

  10. [18]

    Anystory: Towards unified single and multiple subject personalization in text-to-image generation

    Junjie He, Yuxiang Tuo, Binghui Chen, Chongyang Zhong, Yifeng Geng, and Liefeng Bo. Anystory: Towards unified single and multiple subject personalization in text-to-image generation. arXiv preprint arXiv:2501.09503, 2025 a

    2025 arXiv

  11. [19]

    Freeedit: Mask-free reference-based image editing with multi-modal instruction

    Runze He, Kai Ma, Linjiang Huang, Shaofei Huang, Jialin Gao, Xiaoming Wei, Jiao Dai, Jizhong Han, and Si Liu. Freeedit: Mask-free reference-based image editing with multi-modal instruction. arXiv preprint arXiv:2409.18071, 2024

    2024 arXiv

  12. [20]

    Plangen: Towards unified layout planning and image generation in auto-regressive vision language models

    Runze He, Bo Cheng, Yuhang Ma, Qingxiang Jia, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Liebucha Wu, Dawei Leng, and Yuhui Yin. Plangen: Towards unified layout planning and image generation in auto-regressive vision language models. arXiv preprint arXiv:2503.10127, 2025 b

    2025 arXiv

  13. [21]

    Context-aware layout to image generation with enhanced object appearance

    Sen He, Wentong Liao, Michael Ying Yang, Yongxin Yang, Yi-Zhe Song, Bodo Rosenhahn, and Tao Xiang. Context-aware layout to image generation with enhanced object appearance. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15049--15058, 2021

    2021

  14. [22]

    Style aligned image generation via shared attention

    Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4775--4785, 2024

    2024

  15. [23]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718, 2021

    2021 arXiv

  16. [24]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  17. [25]

    Interactdiffusion: Interaction control in text-to-image diffusion models

    Jiun Tian Hoe, Xudong Jiang, Chee Seng Chan, Yap-Peng Tan, and Weipeng Hu. Interactdiffusion: Interaction control in text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6180--6189, 2024

    2024

  18. [26]

    Learning disentangled identifiers for action-customized text-to-image generation

    Siteng Huang, Biao Gong, Yutong Feng, Xi Chen, Yuqian Fu, Yu Liu, and Donglin Wang. Learning disentangled identifiers for action-customized text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7797--7806, 2024 a

    2024

  19. [27]

    Reversion: Diffusion-based relation inversion from images

    Ziqi Huang, Tianxing Wu, Yuming Jiang, Kelvin CK Chan, and Ziwei Liu. Reversion: Diffusion-based relation inversion from images. In SIGGRAPH Asia 2024 Conference Papers, pages 1--11, 2024 b

    2024

  20. [28]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  21. [29]

    Res-tuning: A flexible and efficient tuning paradigm via unbinding tuner from backbone

    Zeyinzi Jiang, Chaojie Mao, Ziyuan Huang, Ao Ma, Yiliang Lv, Yujun Shen, Deli Zhao, and Jingren Zhou. Res-tuning: A flexible and efficient tuning paradigm via unbinding tuner from backbone. Advances in Neural Information Processing Systems, 36: 0 42689--42716, 2023

    2023

  22. [30]

    Miradata: A large-scale video dataset with long durations and structured captions

    Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. Miradata: A large-scale video dataset with long durations and structured captions. Advances in Neural Information Processing Systems, 37: 0 48955--48970, 2025

    2025

  23. [31]

    Story generation with crowdsourced plot graphs

    Boyang Li, Stephen Lee-Urban, George Johnston, and Mark Riedl. Story generation with crowdsourced plot graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 598--604, 2013

    2013

  24. [32]

    Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing

    Dongxu Li, Junnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. Advances in Neural Information Processing Systems, 36: 0 30146--30166, 2023 a

    2023

  25. [33]

    Planning and rendering: Towards product poster generation with diffusion models

    Zhaochen Li, Fengheng Li, Wei Feng, Honghe Zhu, Yaoyu Li, Zheng Zhang, Jingjing Lv, Junjie Shen, Zhangang Lin, Jingping Shao, et al. Planning and rendering: Towards product poster generation with diffusion models. arXiv preprint arXiv:2312.08822, 2023 b

    2023 arXiv

  26. [34]

    Photomaker: Customizing realistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming-Ming Cheng, and Ying Shan. Photomaker: Customizing realistic human photos via stacked id embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8640--8650, 2024

    2024

  27. [35]

    Ragar: Retrieval augment personalized image generation guided by recommendation

    Run Ling, Wenji Wang, Yuting Liu, Guibing Guo, Linying Jiang, and Xingwei Wang. Ragar: Retrieval augment personalized image generation guided by recommendation. arXiv preprint arXiv:2505.01657, 2025

    2025 arXiv

  28. [36]

    Intelligent grimm-open-ended visual storytelling via latent diffusion models

    Chang Liu, Haoning Wu, Yujie Zhong, Xiaoyun Zhang, Yanfeng Wang, and Weidi Xie. Intelligent grimm-open-ended visual storytelling via latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6190--6200, 2024 a

    2024

  29. [37]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pages 38--55. Springe...

    2024

  30. [38]

    Bridge diffusion model: Bridge chinese text-to-image diffusion model with english communities

    Shanyuan Liu, Bo Cheng, Yuhang Ma, Liebucha Wu, Ao Ma, Xiaoyu Wu, Dawei Leng, and Yuhui Yin. Bridge diffusion model: Bridge chinese text-to-image diffusion model with english communities. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 5541--5549, 2025 a

    2025

  31. [39]

    One-prompt-one-story: Free-lunch consistent text-to-image generation using a single prompt

    Tao Liu, Kai Wang, Senmao Li, Joost van de Weijer, Fahad Shahbaz Khan, Shiqi Yang, Yaxing Wang, Jian Yang, and Ming-Ming Cheng. One-prompt-one-story: Free-lunch consistent text-to-image generation using a single prompt. arXiv preprint arXiv:2501.13554, 2025 b

    2025 arXiv

  32. [40]

    Recent advances in ood detection: Problems and approaches

    Shuo Lu, Yingsheng Wang, Lijun Sheng, Aihua Zheng, Lingxiao He, and Jian Liang. Recent advances in ood detection: Problems and approaches. arXiv preprint arXiv:2409.11884, 2024

    2024 arXiv

  33. [41]

    Uni-layout: Integrating human feedback in unified layout generation and evaluation

    Shuo Lu, Yanyin Chen, Wei Feng, Jiahao Fan, Fengheng Li, Zheng Zhang, Jingjing Lv, Junjie Shen, Ching Law, and Jian Liang. Uni-layout: Integrating human feedback in unified layout generation and evaluation. arXiv preprint arXiv:2508.02374, 2025

    2025 arXiv

  34. [42]

    Unified multi-modal latent diffusion for joint subject and text conditional image generation

    Yiyang Ma, Huan Yang, Wenjing Wang, Jianlong Fu, and Jiaying Liu. Unified multi-modal latent diffusion for joint subject and text conditional image generation. arXiv preprint arXiv:2303.09319, 2023

    2023 arXiv

  35. [43]

    Hico: Hierarchical controllable diffusion model for layout-to-image generation

    Yuhang Ma, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Dawei Leng, and Yuhui Yin. Hico: Hierarchical controllable diffusion model for layout-to-image generation. Advances in Neural Information Processing Systems, 37: 0 128886--128910, 2024 a

    2024

  36. [44]

    Storynizor: Consistent story generation via inter-frame synchronized and shuffled id injection

    Yuhang Ma, Wenting Xu, Chaoyi Zhao, Keqiang Sun, Qinfeng Jin, Zeng Zhao, Changjie Fan, and Zhipeng Hu. Storynizor: Consistent story generation via inter-frame synchronized and shuffled id injection. arXiv preprint arXiv:2409.19624, 2024 b

    2024 arXiv

  37. [45]

    Story-adapter: A training-free iterative framework for long story visualization

    Jiawei Mao, Xiaoke Huang, Yunfei Xie, Yuanqi Chang, Mude Hui, Bingjie Xu, and Yuyin Zhou. Story-adapter: A training-free iterative framework for long story visualization. arXiv preprint arXiv:2410.06244, 2024

    2024

  38. [46]

    Lego: Learning to disentangle and invert personalized concepts beyond object appearance in text-to-image diffusion models

    Saman Motamed, Danda Pani Paudel, and Luc Van Gool. Lego: Learning to disentangle and invert personalized concepts beyond object appearance in text-to-image diffusion models. arXiv preprint arXiv:2311.13833, 2023

    2023 arXiv

  39. [47]

    Openvid-1m: A large-scale high-quality dataset for text-to-video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to-video generation. arXiv preprint arXiv:2407.02371, 2024

    2024 arXiv

  40. [48]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195--4205, 2023

    2023

  41. [49]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M \"u ller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  42. [50]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  43. [51]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 0 (140): 0 1--67, 2020

    2020

  44. [52]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1 0 (2): 0 3, 2022

    2022 arXiv

  45. [53]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695, 2022

    2022

  46. [54]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention--MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part...

    2015

  47. [55]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22500...

    2023

  48. [56]

    Carvekit: Automated high-quality background removal framework

    Nikita Selin. Carvekit: Automated high-quality background removal framework. https:// github.com/OPHoperHPO/image-background-remove-tool, 2023. Accessed: March 6, 2025

    2023

  49. [57]

    Eventvad: Training-free event-aware video anomaly detection

    Yihua Shao, Haojin He, Sijie Li, Siyu Chen, Xinwei Long, Fanhu Zeng, Yuxuan Fan, Muyang Zhang, Ziyang Yan, Ao Ma, et al. Eventvad: Training-free event-aware video anomaly detection. arXiv preprint arXiv:2504.13092, 2025 a

    2025 arXiv

  50. [58]

    Tr-dq: Time-rotation diffusion quantization

    Yihua Shao, Deyang Lin, Fanhu Zeng, Minxi Yan, Muyang Zhang, Siyu Chen, Yuxuan Fan, Ziyang Yan, Haozhe Wang, Jingcai Guo, et al. Tr-dq: Time-rotation diffusion quantization. arXiv preprint arXiv:2503.06564, 2025 b

    2025 arXiv

  51. [59]

    In-context meta lora generation

    Yihua Shao, Minxi Yan, Yang Liu, Siyu Chen, Wenjie Chen, Xinwei Long, Ziyang Yan, Lei Li, Chenyu Zhang, Nicu Sebe, et al. In-context meta lora generation. arXiv preprint arXiv:2501.17635, 2025 c

    2025 arXiv

  52. [60]

    Storybooth: Training-free multi-subject consistency for improved visual storytelling

    Jaskirat Singh, Junshen K Chen, Jonas K Kohler, and Michael F Cohen. Storybooth: Training-free multi-subject consistency for improved visual storytelling. In The Thirteenth International Conference on Learning Representations

  53. [61]

    Styledrop: Text-to-image synthesis of any style

    Kihyuk Sohn, Lu Jiang, Jarred Barber, Kimin Lee, Nataniel Ruiz, Dilip Krishnan, Huiwen Chang, Yuanzhen Li, Irfan Essa, Michael Rubinstein, et al. Styledrop: Text-to-image synthesis of any style. Advances in Neural Information Processing Systems, 36: 0 66860--66889, 2023

    2023

  54. [62]

    Instantx flux.1-dev ip-adapter page, 2024

    InstantX Team. Instantx flux.1-dev ip-adapter page, 2024

    2024

  55. [63]

    Training-free consistent text-to-image generation

    Yoad Tewel, Omri Kaduri, Rinon Gal, Yoni Kasten, Lior Wolf, Gal Chechik, and Yuval Atzmon. Training-free consistent text-to-image generation. ACM Transactions on Graphics (TOG), 43 0 (4): 0 1--18, 2024

    2024

  56. [64]

    Converting video formats with ffmpeg

    Suramya Tomar. Converting video formats with ffmpeg. Linux journal, 2006 0 (146): 0 10, 2006

    2006

  57. [65]

    Face0: Instantaneously conditioning a text-to-image model on a face

    Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan. Face0: Instantaneously conditioning a text-to-image model on a face. In SIGGRAPH Asia 2023 Conference Papers, pages 1--10, 2023

    2023

  58. [66]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  59. [67]

    Is this loss informative? faster text-to-image customization by tracking objective dynamics

    Anton Voronov, Mikhail Khoroshikh, Artem Babenko, and Max Ryabinin. Is this loss informative? faster text-to-image customization by tracking objective dynamics. Advances in Neural Information Processing Systems, 36: 0 37491--37510, 2023

    2023

  60. [68]

    p+: Extended textual conditioning in text-to-image generation

    Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. p+: Extended textual conditioning in text-to-image generation. arXiv preprint arXiv:2303.09522, 2023

    2023 arXiv

  61. [69]

    Qihoo-t2x: An efficiency-focused diffusion transformer via proxy tokens for text-to-any-task

    Jing Wang, Ao Ma, Jiasong Feng, Dawei Leng, Yuhui Yin, and Xiaodan Liang. Qihoo-t2x: An efficiency-focused diffusion transformer via proxy tokens for text-to-any-task. arXiv e-prints, pages arXiv--2409, 2024

    2024

  62. [70]

    Wisa: World simulator assistant for physics-aware text-to-video generation

    Jing Wang, Ao Ma, Ke Cao, Jun Zheng, Zhanjie Zhang, Jiasong Feng, Shanyuan Liu, Yuhang Ma, Bo Cheng, Dawei Leng, et al. Wisa: World simulator assistant for physics-aware text-to-video generation. arXiv preprint arXiv:2503.08153, 2025 a

    2025 arXiv

  63. [71]

    Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation

    Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation. 2023 a

    2023

  64. [72]

    Spnet: Learning stereo matching with slanted plane aggregation

    Yun Wang, Longguang Wang, Hanyun Wang, and Yulan Guo. Spnet: Learning stereo matching with slanted plane aggregation. IEEE Robotics and Automation Letters, 7 0 (3): 0 6258--6265, 2022

    2022

  65. [73]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023 b

    2023 arXiv

  66. [74]

    Adstereo: Efficient stereo matching with adaptive downsampling and disparity alignment

    Yun Wang, Kunhong Li, Longguang Wang, Junjie Hu, Dapeng Oliver Wu, and Yulan Guo. Adstereo: Efficient stereo matching with adaptive downsampling and disparity alignment. IEEE Transactions on Image Processing, 2025 b

    2025

  67. [75]

    Learning robust stereo matching in the wild with selective mixture-of-experts

    Yun Wang, Longguang Wang, Chenghao Zhang, Yongjian Zhang, Zhanjie Zhang, Ao Ma, Chenyou Fan, Tin Lun Lam, and Junjie Hu. Learning robust stereo matching in the wild with selective mixture-of-experts. arXiv preprint arXiv:2507.04631, 2025 c

    2025 arXiv

  68. [76]

    Dualnet: Robust self-supervised stereo matching with pseudo-label supervision

    Yun Wang, Jiahao Zheng, Chenghao Zhang, Zhanjie Zhang, Kunhong Li, Yongjian Zhang, and Junjie Hu. Dualnet: Robust self-supervised stereo matching with pseudo-label supervision. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8178--8186, 2025 d

    2025

  69. [77]

    Styleadapter: A unified stylized image generation model

    Zhouxia Wang, Xintao Wang, Liangbin Xie, Zhongang Qi, Ying Shan, Wenping Wang, and Ping Luo. Styleadapter: A unified stylized image generation model. arXiv preprint arXiv:2309.01770, 2023 c

    2023 arXiv

  70. [78]

    Dropoutgs: Dropping out gaussians for better sparse-view rendering

    Yexing Xu, Longguang Wang, Minglin Chen, Sheng Ao, Li Li, and Yulan Guo. Dropoutgs: Dropping out gaussians for better sparse-view rendering. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 701--710, 2025

    2025

  71. [79]

    Freestyle layout-to-image synthesis

    Han Xue, Zhiwu Huang, Qianru Sun, Li Song, and Wenjun Zhang. Freestyle layout-to-image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14256--14266, 2023

    2023

  72. [80]

    Facestudio: Put your face everywhere in seconds

    Yuxuan Yan, Chi Zhang, Rui Wang, Yichao Zhou, Gege Zhang, Pei Cheng, Gang Yu, and Bin Fu. Facestudio: Put your face everywhere in seconds. arXiv preprint arXiv:2312.02663, 2023

    2023 arXiv

  73. [81]

    Seed-story: Multimodal long story generation with large language model

    Shuai Yang, Yuying Ge, Yang Li, Yukang Chen, Yixiao Ge, Ying Shan, and Yingcong Chen. Seed-story: Multimodal long story generation with large language model. arXiv preprint arXiv:2407.08683, 2024

    2024 arXiv

  74. [82]

    Reco: Region-controlled text-to-image generation

    Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, et al. Reco: Region-controlled text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14246--14...

    2023

  75. [83]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  76. [84]

    Controlnet-xs: Rethinking the control of text-to-image diffusion models as feedback-control systems

    Denis Zavadski, Johann-Friedrich Feiden, and Carsten Rother. Controlnet-xs: Rethinking the control of text-to-image diffusion models as feedback-control systems. In European Conference on Computer Vision, pages 343--362. Springer, 2024

    2024

  77. [85]

    Creatilayout: Siamese multimodal diffusion transformer for creative layout-to-image generation

    Hui Zhang, Dexiang Hong, Tingwei Gao, Yitong Wang, Jie Shao, Xinglong Wu, Zuxuan Wu, and Yu-Gang Jiang. Creatilayout: Siamese multimodal diffusion transformer for creative layout-to-image generation. arXiv preprint arXiv:2412.03859, 2024 a

    2024 arXiv

  78. [86]

    A survey on personalized content synthesis with diffusion models

    Xulu Zhang, Xiao-Yong Wei, Wengyu Zhang, Jinlin Wu, Zhaoxiang Zhang, Zhen Lei, and Qing Li. A survey on personalized content synthesis with diffusion models. arXiv preprint arXiv:2405.05538, 2024 b

    2024

  79. [87]

    Generative active learning for image synthesis personalization

    Xulu Zhang, Wengyu Zhang, Xiaoyong Wei, Jinlin Wu, Zhaoxiang Zhang, Zhen Lei, and Qing Li. Generative active learning for image synthesis personalization. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 10669--10677, 2024 c

    2024

  80. [88]

    Towards highly realistic artistic style transfer via stable diffusion with step-aware and layer-aware prompt

    Zhanjie Zhang, Quanwei Zhang, Huaizhong Lin, Wei Xing, Juncheng Mo, Shuaicheng Huang, Jinheng Xie, Guangyuan Li, Junsheng Luan, Lei Zhao, et al. Towards highly realistic artistic style transfer via stable diffusion with step-aware and layer-aware prompt. In Proceedings of the ...

    2024

  81. [89]

    Artbank: Artistic style transfer with pre-trained diffusion model and implicit style prompt bank

    Zhanjie Zhang, Quanwei Zhang, Wei Xing, Guangyuan Li, Lei Zhao, Jiakai Sun, Zehua Lan, Junsheng Luan, Yiling Huang, and Huaizhong Lin. Artbank: Artistic style transfer with pre-trained diffusion model and implicit style prompt bank. In Proceedings of the AAAI Conference on Art...

    2024

  82. [90]

    Lgast: Towards high-quality arbitrary style transfer with local--global style learning

    Zhanjie Zhang, Yuxiang Li, Ruichen Xia, Mengyuan Yang, Yun Wang, Lei Zhao, and Wei Xing. Lgast: Towards high-quality arbitrary style transfer with local--global style learning. Neurocomputing, 623: 0 129434, 2025 a

    2025

  83. [91]

    U-stydit: Ultra-high quality artistic style transfer using diffusion transformers

    Zhanjie Zhang, Ao Ma, Ke Cao, Jing Wang, Shanyuan Liu, Yuhang Ma, Bo Cheng, Dawei Leng, and Yuhui Yin. U-stydit: Ultra-high quality artistic style transfer using diffusion transformers. arXiv preprint arXiv:2503.08157, 2025 b

    2025 arXiv

  84. [92]

    Spast: Arbitrary style transfer with style priors via pre-trained large-scale model

    Zhanjie Zhang, Quanwei Zhang, Junsheng Luan, Mengyuan Yang, Yun Wang, and Lei Zhao. Spast: Arbitrary style transfer with style priors via pre-trained large-scale model. Neural Networks, page 107556, 2025 c

    2025

  85. [93]

    Vectorsketcher: Learning to create a vector-based free-hand sketch

    Zhanjie Zhang, Quanwei Zhang, Junsheng Luan, Mengyuan Yang, Yun Wang, and Lei Zhao. Vectorsketcher: Learning to create a vector-based free-hand sketch. Engineering Applications of Artificial Intelligence, 156: 0 111005, 2025 d

    2025

  86. [94]

    Image generation from layout

    Bo Zhao, Lili Meng, Weidong Yin, and Leonid Sigal. Image generation from layout. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8584--8593, 2019

    2019

  87. [95]

    Uni-controlnet: All-in-one control to text-to-image diffusion models

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Systems, 36: 0 11127--11150, 2023

    2023

  88. [96]

    Layoutdiffusion: Controllable diffusion model for layout-to-image generation

    Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22490--22499, 2023

    2023

  89. [97]

    Enhancing detail preservation for customized text-to-image generation: A regularization-free approach

    Yufan Zhou, Ruiyi Zhang, Tong Sun, and Jinhui Xu. Enhancing detail preservation for customized text-to-image generation: A regularization-free approach. arXiv preprint arXiv:2305.13579, 2023

    2023 arXiv

  90. [98]

    Storydiffusion: Consistent self-attention for long-range image and video generation

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. Storydiffusion: Consistent self-attention for long-range image and video generation. Advances in Neural Information Processing Systems, 37: 0 110315--110340, 2025

    2025

  91. [99]

    Storymaker: Towards holistic consistent characters in text-to-image generation

    Zhengguang Zhou, Jing Li, Huaxia Li, Nemo Chen, and Xu Tang. Storymaker: Towards holistic consistent characters in text-to-image generation. arXiv preprint arXiv:2409.12576, 2024

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.