Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

IGD: Instructional Graphic Design with Multimodal Layer Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read IGD turns plain-language prompts into editable, layered graphic-design files.

desk verdict A solid editable-design system that overclaims superiority—layout actually trails DALL·E3 in its own table. read the letter →

arxiv 2507.09910 v1 pith:YDSQ5DHV submitted 2025-07-14 cs.CV

Yadong Qu , Shancheng Fang , Yuxin Wang , Xiaorui Wang , Zhineng Chen , Hongtao Xie , Yongdong Zhang This is my paper
classification cs.CV
keywords graphicdesigngenerationmultimodallargelanguagemodeldiffusionmodelseditablelayerslayoutvisualtextrenderinginstructionfollowingparametric
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Instructional Graphic Designer (IGD), a system that generates editable, multilayered graphic-design files directly from natural-language instructions. Its claim is that IGD is the first universal design model to produce multimodal layers—text, image, and graphic—in one stage, rather than only predicting layouts for pre-made assets or rendering a flat, non-editable image. This matters because it targets the two practical failures of prior work: diffusion models render illegible text and forbid editing, while layout-generation pipelines demand pre-designed elements. IGD instead predicts layer attributes, ordering, and placement with a multimodal language model and summons image content from a diffusion model, then renders the result through a design platform.

What carries the argument

The load-bearing mechanism is a standardized, token-friendly representation of design files, in which each multimodal layer (frame, group, graphic, text, image) is written as a sequence of special tokens and attributes such as position, color, font, and content. A multimodal large language model consumes this format and predicts the full stack of layers, including their order and layout; a diffusion model generates image content on demand when an <image_gen> token interrupts the text stream, and end-to-end training lets image-generation loss feed back into layout decisions. Supporting machinery includes converting simple solid-color and vector-like images into SVG paths to avoid overloading the diffusion model, and a bucket strategy that lets generated images match the varying sizes and aspect ratios the language model predicts.

What would settle it

Have human evaluators rate IGD and the baselines on the same prompts without knowing which system made which design; if human preference fails to rank IGD first on accuracy, harmony, or layout, the claimed superiority is not established. A simpler check: open an IGD output in the design platform and try to edit every layer; if some elements are rasterized or text is not true text, the editability claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that graphic design can be recast as parametric layer prediction plus image-asset generation. Given an instruction, IGD autoregressively emits a standardized string describing each layer's type, position, size, color, font, and content; when it hits an image-generation token, it pauses and uses a diffusion model to create the image, then resumes. Because every property is explicit, the output is a real editable design file rather than a pixel image. The authors report that this design outperforms diffusion-based and layout-generation baselines on OCR-based text accuracy, layout metrics, FID, and GPT-4o-assessed quality, and that it supports both Chinese and English across posters, slides, stickers, and marketing graphics.

Load-bearing premise

The paper's superiority claim depends on GPT-4o's five-dimension scores being a valid stand-in for human aesthetic judgment, but no human ratings or inter-rater agreement are reported.

Editorial extensions

If this is right

  • Design files generated by IGD are editable at the attribute and layer level, so users can change text, colors, positions, and images after generation.
  • The same architecture handles multiple scenarios—posters, slides, stickers, event graphics—because they share one standardized file format.
  • OCR-based evaluation shows IGD's rendered text is much more legible than diffusion baselines (86.83% vs 73.83% character F-measure for SD3.5), and the pre-render text accuracy reaches 97.39%.
  • End-to-end training couples the language model and diffusion model, so image style and layer layout stay coherent within a design.
  • Scalability is architectural: more data in the standardized format and larger language or diffusion backbones can be plugged in without changing the paradigm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If GPT-4o ratings do not track human aesthetic preference, the headline quality comparisons rest on a shaky proxy; a human study is the natural next check.
  • The parameter-prediction-plus-rendering paradigm should transfer to adjacent tasks such as UI mockups, document layouts, or interactive slide decks, since they share the same layered structure.
  • A testable extension is to feed an IGD-generated file into an editing loop, letting a user modify one layer and having the model re-layout the rest.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes IGD, a system that generates editable graphic design files from natural-language instructions by combining a multimodal large language model (Qwen2.5) with Stable Diffusion. The model predicts structured layer attributes (text, image, graphic) in a standardized format based on Penpot, renders text parametrically, and generates image assets through diffusion, with end-to-end training. The authors introduce a data pipeline with SVG conversion, text augmentation, and image tagging, and report experiments on 90k training samples with 1k English and 1k Chinese evaluation inputs. They compare against SD3.5, DALL·E3, AnyText, CanvaGPT, and OpenCOLE using GPT-4o scores across five dimensions, OCR-based character metrics, layout metrics, and FID.

Significance. If the claims were fully supported, this would be a meaningful advance: the system outputs editable, layered design files rather than flat images, and it addresses text legibility through parametric rendering, which the OCR results strongly support (IGD achieves 86.83 Char-F in Table 2 versus 73.83 for SD3.5). The standardized format and the use of SVG for simple graphical elements are sensible engineering choices that could support future scalable data collection. The authors also provide a useful comparison with layout-generation and diffusion baselines. However, the paper's central comparative claim of superior layout quality is contradicted by its own Table 1, and the primary evaluation relies on GPT-4o scores without human validation, statistical tests, or released per-prompt data. These issues currently prevent the claims from being fully established.

major comments (4)
  1. [Table 1] The central claim that IGD achieves superior layout quality is contradicted by the paper's own quantitative results. In Table 1, DALL·E3 obtains Layout scores of 85.2 (Chinese) and 82.4 (English), while IGD obtains 71.4 and 72.0. IGD also trails DALL·E3 on Harmony (82.6/88.8 vs 95.4/96.2) and Innovation (65.8/68.2 vs 86.8/88.6), and on Chinese Quality (79.8 vs 81.8). The only dimension where IGD is consistently ahead is Accuracy (88.4/81.4 vs 55.0/52.8). Thus, even taking the GPT-4o scores at face value, the abstract's and introduction's claims of generally 'superior experimental results' and better layout quality are not supported. The authors must either provide additional evidence (e.g., a human study or per-prompt analysis showing the gap is an artifact) or revise the claims to focus on the dimensions where IGD is actually superior, chiefly text accuracy and editability.
  2. [Section 4.1] The main quantitative evaluation uses GPT-4o as a judge for Quality, Harmony, Accuracy, Layout, and Innovation, but no human validation, inter-rater agreement, or statistical test is reported. GPT-4o's aesthetic judgments are not established as a valid proxy for human preference in this task, and the absence of error bars or significance tests means the reported differences (e.g., IGD's 79.8 vs DALL·E3's 81.8 on Chinese Quality) could easily be within noise. The paper should provide a human evaluation on a subsample, report agreement between GPT-4o and human raters, or at minimum release the per-prompt scores so the robustness of the comparisons can be audited.
  3. [Table 2] The quantitative layout comparison is incomplete. Rali, Rove, and Rcom are reported only for IGD and OpenCOLE; the diffusion-based baselines (SD3.5, DALL·E3, AnyText) are shown as '-'. This means the claim that IGD improves 'layer alignment' and 'text readability' relative to all baselines is not supported by those metrics. The authors should compute these layout metrics for all baselines or clearly restrict the claim to methods with editable layer structures.
  4. [Section 4.3] The ablation study is reported only qualitatively, with statements such as 'omitting the augmented pre-training leads to a decline in layout performance' and 'integrating LLM and SD improves spatial relationships and stylistic consistency.' No quantitative table or figure is provided for the ablations, so the reader cannot assess the magnitude or reliability of these effects. Since the ablation claims concern components that are central to the proposed method, quantitative results should be added.
minor comments (4)
  1. [Throughout] The manuscript contains several typos and grammatical errors, including 'capbilities', 'reseahcers', 'facilite', 'develope', 'constraint', and 'the the'; a careful proofreading pass is needed.
  2. [Figure 2] The XML example in Figure 2 has broken spacing inside several tags (e.g., '</ line_to >', '< t e x t>', '</ t e x t>'), which makes the standardized format hard to read. Use monospaced font and intact tags.
  3. [Section 4.1] The description of the evaluation data is ambiguous: the text says '1k each of English and Chinese user inputs are prepared for evaluation,' but Table 1's caption says 'we manually generate 100 evaluation samples' for CanvaGPT. Please clarify whether all methods are evaluated on the same 100 prompts per language or on different sets, and report how the 1k inputs are sampled.
  4. [References] Some references are incomplete, for example [1] (CanvaGPT) has no author list or venue, and [28] (NovelAI) is a blog post with no stable URL or version. Please provide complete citation information.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: IGD is an empirical systems paper with held-out evaluation, external baselines, and no prediction that reduces to its inputs.

full rationale

IGD makes no first-principles derivation; its central claim is that a trained MLLM+diffusion system can generate editable layered design files from natural language. The model is trained on 90k collected design files and evaluated on 1k held-out English and 1k held-out Chinese user inputs against external baselines (SD3.5, DALL·E3, AnyText, CanvaGPT, OpenCOLE), so the reported comparisons are not fitted to the evaluation set. GPT-4o, the judge, is a third-party system not authored by the paper's authors, and no parameter is fitted to its scores. The 'Training Samples' row in Table 1 is explicitly an additional sanity check, not evidence of generalization. The few self-citations ([30], [50], [52]) are related-work references for text recognition and glyph-control methods; none is load-bearing for IGD's architecture or results. No uniqueness theorem is imported, no ansatz is smuggled via citation, and no known result is merely renamed. Concerns about whether GPT-4o tracks human aesthetics or whether Table 1's Layout scores support the 'superior layout' claim are empirical/correctness issues, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper does not introduce new physical or mathematical entities. The standardized format is a representation protocol, not an invented entity. The free parameters listed are hand-chosen system hyperparameters that affect implementation details but are not the core scientific claim. The axioms are domain assumptions about tooling, evaluation, and data representativeness.

free parameters (2)
  • Aspect-ratio bucket set = not specified
    Images are grouped into pre-defined width/height buckets to handle scale variation; the bucket boundaries are hand-chosen hyperparameters, described in Section 3.2.3.
  • Number of dominant colors in k-means clustering = not specified
    Simple images are converted to SVG by k-means color clustering; the cluster count is a hand-chosen parameter affecting vectorization fidelity, described in Section 3.1.3.
assumptions (4)
  • domain assumption Penpot renders the standardized layer format into valid graphic design images.
    The whole pipeline depends on Penpot's rendering being faithful and complete; introduced in Section 3.1.1.
  • domain assumption GPT-4o scores correlate with human design quality.
    No human evaluation is provided, so the primary comparison metric rests on this unvalidated assumption; introduced in Section 4.1.
  • domain assumption OCR character metrics measure text legibility in design context.
    The paper uses OCR precision/recall as a proxy for visual text quality; introduced in Section 4.1.
  • domain assumption The 90k training corpus is representative of the target design scenarios.
    The model is trained on internet and paid design files; generalization to unseen user instructions is assumed; introduced in Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IGD: Instructional Graphic Design with Multimodal Layer Generation." pith.science (2026). https://pith.science/paper/YDSQ5DHV

@misc{pith2026250709910,
  author       = {Pith},
  title        = {Pith review of: IGD: Instructional Graphic Design with Multimodal Layer Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YDSQ5DHV}},
  note         = {Machine review of arXiv:2507.09910}
}
read the original abstract

Graphic design visually conveys information and data by creating and combining text, images and graphics. Two-stage methods that rely primarily on layout generation lack creativity and intelligence, making graphic design still labor-intensive. Existing diffusion-based methods generate non-editable graphic design files at image level with poor legibility in visual text rendering, which prevents them from achieving satisfactory and practical automated graphic design. In this paper, we propose Instructional Graphic Designer (IGD) to swiftly generate multimodal layers with editable flexibility with only natural language instructions. IGD adopts a new paradigm that leverages parametric rendering and image asset generation. First, we develop a design platform and establish a standardized format for multi-scenario design files, thus laying the foundation for scaling up data. Second, IGD utilizes the multimodal understanding and reasoning capabilities of MLLM to accomplish attribute prediction, sequencing and layout of layers. It also employs a diffusion model to generate image content for assets. By enabling end-to-end training, IGD architecturally supports scalability and extensibility in complex graphic design tasks. The superior experimental results demonstrate that IGD offers a new solution for graphic design.

Figures

Figures reproduced from arXiv: 2507.09910 by the authors.

Figure 1
Figure 1. Method comparison. Diffusion models struggle with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An example of representing a design file in the standard [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The pipeline of IGD. IGD generates standardized format strings according to instructions to achieve multi-scenario graphic [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Processes in multimodal and single modal input modes. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Comparison with the state-of-the-art methods. IGD effectively adheres to instructions by presenting the intended content and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Several samples to demonstrate the characteristics of IGD. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks

    cs.CV 2026-04 conditional novelty 7.0 of 10

    A 49-task layered-design benchmark shows frontier AI models leave most precision-heavy graphic design tasks unsolved, with only two tasks mostly solved.

Reference graph

Works this paper leans on

55 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Canvagpt - effortlessly design anything: presentations, lo- gos, social media posts and more. 6

    work page

  2. [2]

    Gpt-4v(ision) system card. 2023. 2

    work page 2023

  3. [3]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  4. [4]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  5. [5]

    Variational transformer networks for layout generation

    Diego Martin Arroyo, Janis Postels, and Federico Tombari. Variational transformer networks for layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 13642–13652, 2021. 6

    work page 2021

  6. [6]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 1, 2

    arXiv 2023

  7. [7]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 2

    arXiv 2023

  8. [8]

    ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 3

    arXiv 2022

Show all 55 references
  1. [9]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 1, 6

    2023

  2. [10]

    Textdiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. Advances in Neural Information Processing Sys- tems, 36, 2024. 2

    2024

  3. [11]

    Graphic design with large multimodal model

    Yutao Cheng, Zhao Zhang, Maoke Yang, Hui Nie, Chunyuan Li, Xinglong Wu, and Jie Shao. Graphic design with large multimodal model. arXiv preprint arXiv:2404.14368, 2024. 2, 3

    2024 arXiv

  4. [12]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhang- hao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yong- hao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. 1, 2

    2023

  5. [13]

    Dreamllm: Synergistic multimodal com- prehension and creation

    Runpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, et al. Dreamllm: Synergistic multimodal com- prehension and creation. arXiv preprint arXiv:2309.11499,

    arXiv

  6. [14]

    SVTR: scene text recognition with a single visual model

    Yongkun Du, Zhineng Chen, Caiyan Jia, Xiaoting Yin, Tian- lun Zheng, Chenxia Li, Yuning Du, and Yu-Gang Jiang. SVTR: scene text recognition with a single visual model. In IJCAI, pages 884–890, 2022. 3

    2022

  7. [15]

    Instruction-guided scene text recognition

    Yongkun Du, Zhineng Chen, Yuchen Su, Caiyan Jia, and Yu- Gang Jiang. Instruction-guided scene text recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(6):2723–2738, 2025. 3

    2025

  8. [16]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  9. [17]

    Layoutgpt: Compositional visual plan- ning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual plan- ning and generation with large language models. Advances in Neural Information Processing Systems, 36, 2024. 2, 3

    2024

  10. [18]

    Making llama see and draw with seed tokenizer

    Yuying Ge, Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. Making llama see and draw with seed tokenizer. arXiv preprint arXiv:2310.01218, 2023. 2

    2023 arXiv

  11. [19]

    Seed-x: Mul- timodal models with unified multi-granularity comprehen- sion and generation

    Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Mul- timodal models with unified multi-granularity comprehen- sion and generation. arXiv preprint arXiv:2404.14396, 2024. 2

    2024 arXiv

  12. [20]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 3

    2020

  13. [21]

    Opencole: Towards reproducible automatic graphic design generation

    Naoto Inoue, Kento Masui, Wataru Shimoda, and Kota Yam- aguchi. Opencole: Towards reproducible automatic graphic design generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8131–8135, 2024. 6

    2024

  14. [22]

    Cole: A hierarchical generation frame- work for multi-layered and editable graphic design

    Peidong Jia, Chenxuan Li, Yuhui Yuan, Zeyu Liu, Yichao Shen, Bohan Chen, Xingru Chen, Yinglin Zheng, Dong Chen, Ji Li, et al. Cole: A hierarchical generation frame- work for multi-layered and editable graphic design. arXiv preprint arXiv:2311.16974, 2023. 3

    2023 arXiv

  15. [23]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023. 2

    2023

  16. [24]

    Autoposter: A highly automatic and content-aware design system for ad- vertising poster generation

    Jinpeng Lin, Min Zhou, Ye Ma, Yifan Gao, Chenxi Fei, Yangjian Chen, Zhang Yu, and Tiezheng Ge. Autoposter: A highly automatic and content-aware design system for ad- vertising poster generation. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1250–1260,

  17. [25]

    Character-aware models improve visual text rendering

    Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mo- hammad Norouzi, and Noah Constant. Character-aware models improve visual text rendering. arXiv preprint arXiv:2212.10562, 2022. 3

    2022 arXiv

  18. [26]

    Glyph-byt5: A customized text encoder for accurate visual text rendering

    Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, and Yuhui Yuan. Glyph-byt5: A customized text encoder for accurate visual text rendering. In European Conference on Computer Vision , pages 361–377. Springer,

  19. [27]

    Glyphdraw: Learning to draw chinese characters in image synthesis models coher- ently

    Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, and Xiaodong Lin. Glyphdraw: Learning to draw chinese characters in image synthesis models coher- ently. arXiv preprint arXiv:2303.17870, 2, 2023. 3

    2023 arXiv

  20. [28]

    Novelai improvements on stable diffusion, 2023

    NovelAI. Novelai improvements on stable diffusion, 2023. 5

    2023

  21. [29]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 5

    2023 arXiv

  22. [30]

    Boosting semi- supervised scene text recognition via viewing and summariz- ing

    Yadong Qu, Yuxin Wang, Bangbang Zhou, Zixiao Wang, Hongtao Xie, and Yongdong Zhang. Boosting semi- supervised scene text recognition via viewing and summariz- ing. In Advances in Neural Information Processing Systems, pages 105503–105527. Curran Associates, Inc., 2024. 3

    2024

  23. [31]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  24. [32]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 3

    2020

  25. [33]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3, 4

    2022

  26. [34]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  27. [35]

    Potrace: a polygon-based tracing algorithm,

    Peter Selinger. Potrace: a polygon-based tracing algorithm,

  28. [36]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3

    2010 arXiv

  29. [37]

    Emu: Generative pretraining in multimodality

    Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Emu: Generative pretraining in multimodality. In The Twelfth International Conference on Learning Representations, 2023. 2

    2023

  30. [38]

    Chameleon: Mixed-modal early-fusion foundation models

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024. 2

    2024 arXiv

  31. [39]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1, 2

    2023 arXiv

  32. [40]

    Anytext: Multilingual visual text gen- eration and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text gen- eration and editing. arXiv preprint arXiv:2311.03054, 2023. 1, 2, 3, 6

    2023 arXiv

  33. [41]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 2

    2017

  34. [42]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024 arXiv

  35. [43]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 2

    2024 arXiv

  36. [44]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848, 2024

    arXiv

  37. [45]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024. 2

    2024 arXiv

  38. [46]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    arXiv

  39. [47]

    Posterllava: Constructing a uni- fied multi-modal layout generator with llm

    Tao Yang, Yingmin Luo, Zhongang Qi, Yang Wu, Ying Shan, and Chang Wen Chen. Posterllava: Constructing a uni- fied multi-modal layout generator with llm. arXiv preprint arXiv:2406.02884, 2024. 2, 3

    2024 arXiv

  40. [48]

    Glyphcontrol: Glyph conditional control for visual text generation

    Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, and Kai Chen. Glyphcontrol: Glyph conditional control for visual text generation. Advances in Neural Information Processing Systems, 36, 2024. 2, 3

    2024

  41. [49]

    mplug-owl: Modularization empowers large language models with multimodality

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178, 2023. 2

    2023 arXiv

  42. [50]

    How control information influences multilingual text image generation and editing? arXiv preprint arXiv:2407.11502 ,

    Boqiang Zhang, Zuan Gao, Yadong Qu, and Hongtao Xie. How control information influences multilingual text image generation and editing? arXiv preprint arXiv:2407.11502 ,

    arXiv

  43. [51]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 3

    2023

  44. [52]

    CDistNet: Perceiving multi- domain character distance for robust text recognition

    Tianlun Zheng, Zhineng Chen, Shancheng Fang, Hongtao Xie, and Yu-Gang Jiang. CDistNet: Perceiving multi- domain character distance for robust text recognition. Inter- national Journal of Computer Vision, 132(2):300–318, 2024. 3

    2024

  45. [53]

    Transfusion: Pre- dict the next token and diffuse images with one multi-modal model

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Pre- dict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408.11039, 2024. 2

    2024 arXiv

  46. [54]

    Composition-aware graphic layout gan for visual-textual presentation designs

    Min Zhou, Chenchen Xu, Ye Ma, Tiezheng Ge, Yuning Jiang, and Weiwei Xu. Composition-aware graphic layout gan for visual-textual presentation designs. arXiv preprint arXiv:2205.00303, 2022. 2, 3, 6

    2022 arXiv

  47. [55]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 2

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.