Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Rethinking Layered Graphic Design Generation with a Top-Down Approach

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Accordion turns flat AI-generated designs into editable layered graphics by planning from the whole reference image downward, and reports the top average on the DesignIntention benchmark.

desk verdict Accordion is a genuine first attempt at top-down layer decomposition of AI-generated designs, with honest limitations, but the headline GPT-4V scores don't prove editability — still worth refereeing. read the letter →

arxiv 2507.05601 v1 pith:YN6APXGI submitted 2025-07-08 cs.CV

Jingye Chen , Zhaowen Wang , Nanxuan Zhao , Li Zhang , Difan Liu , Jimei Yang , Qifeng Chen This is my paper
classification cs.CV
keywords layeredgraphicdesigntop-downgenerationvisionlanguagemodelplanningtextde-renderingAI-generatededitabilityrefinementIntentionbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that rasterized, hard-to-edit AI-generated designs can serve as global references for building editable layered graphics, provided the decomposition is planned top-down rather than assembled bottom-up. Accordion is a three-stage framework built around a vision-language model: it turns a short intention or sketch into a detailed prompt and reference image, derives a structured plan of background, objects, and vectorized text layers, then extracts and removes elements to reconstruct the layered design while replacing nonsensical AI text with prompt-appropriate wording. The authors report an average of 6.5 on the DesignIntention benchmark, above 6.0 for COLE and 6.3 for Open-COLE, and designer user studies favor its editability. If correct, the method makes the output of image-generation models usable in real editing workflows and opens a route from pixel designs to vector-like layers.

What carries the argument

The load-bearing mechanism is a three-stage top-down pipeline orchestrated by a single vision-language model (VLM). In Stage 1 the VLM expands a short intention or sketch into a detailed prompt for a text-to-image model, producing a global reference image. In Stage 2 the VLM reads that image and outputs a structured design plan as a sequence of dictionaries with normalized bounding boxes and renderable text attributes. In Stage 3 the plan drives image decomposition: a text-removal model erases the text regions using the planned boxes, SAM segments foreground objects from their planned boxes, an inpainting model removes each object to recover the background, and the VLM answers a questionnaire to pick the cleanest removal result. The text layer is re-rendered from the plan, so its content can be refined even when the reference contains gibberish. The framework is trained on Design39K augmented with AI-generated references whose text has been corrupted by inpainting, and it produces separate background, object, and vectorized text layers in a fixed bottom-to-top stacking order.

What would settle it

Run the pipeline on a test set of references where text overlaps or sits behind translucent objects, or where fonts are decorative and outside the 2,000-style set, and measure whether the extracted text layer matches the rendered text and whether object masks preserve the reported 68.4% IOU; if these cases degrade sharply, the claim that arbitrary AI-generated designs become editable layered designs fails. A second check is to replace the VLM plan with ground-truth boxes and attributes; if quality does not improve, planning accuracy is not the bottleneck the paper assumes.

Watch

Extended reading notes

Core claim

The central discovery is that a visually harmonious whole-image reference, instead of a sequential text-then-image or image-then-text plan, is a better organizing principle for layered design generation. Given a reference, the method parses it into a plan covering layer order, object bounding boxes, and text attributes such as content, color, font, alignment, line count, and angle, then uses text removal, SAM-based object extraction, and object removal to separate layers. Text is re-rendered as a vector layer, and the vision-language model selects among removal candidates to keep backgrounds clean. On the DesignIntention benchmark the method reports the top average across the five evaluation metrics, and it also handles text-to-template, adding text to backgrounds, text de-rendering, and design variations. The claim is the first successful conversion of AI-generated raster designs into editable layered designs with refined, meaningful text.

Load-bearing premise

The central premise is that the vision-language model's plan is accurate enough, with correct boxes, correct layer order, and text always above objects, for the segmentation and removal tools to split the reference image into clean layers, and that all needed typography falls inside the 2,000 predefined font set.

Editorial extensions

If this is right

  • Rasterized AI-generated posters become editable: text, objects, and background are separate layers that can be selected, translated, restyled, or swapped.
  • The framework stays compatible with newer image generators because the reference model is modular; upgrading the text-to-image model does not require retraining the planner.
  • Plan-first formulation lets text use available background space, producing longer and more information-dense copy, about 1.5 times longer than COLE in the reported adding-text comparison.
  • The same decomposition feeds design variation workflows, including upstream image-variation models, multiple inference-time plan branches, and downstream layout models that consume the extracted layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the plan already encodes text content and placement, the same VLM output could double as a quality check on the reference, automatically rejecting generated images whose text regions are empty or misaligned before extraction begins.
  • Relaxing the fixed text-above-objects assumption to per-element predicted z-order would extend the method to designs with text behind transparent graphics or interleaved with cutouts.
  • The predefined 2,000-font constraint suggests a concrete stress test: measuring how editability degrades on decorative or freeform lettering and whether a font-retrieval module plugged into the plan closes the gap.
  • Layered outputs could be reused as training data for layout generation models, turning a single design-generation system into a source of structured supervision for template synthesis.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Accordion, a three-stage framework for converting AI-generated rasterized design images into editable layered designs (background, object layers, and a vectorized text layer). The approach is top-down: Stage 1 generates a reference image from a user intention or sketch via a VLM-expanded prompt and a text-to-image model (Flux); Stage 2 uses a VLM to predict a design plan containing element orders, bounding boxes, and text attributes; Stage 3 extracts layers from the reference using SAM, an object/text removal model, and a VLM-based result selection module. The method is trained on an in-house dataset Design39K augmented with AI-generated reference images, and evaluated on the DesignIntention benchmark, a designer user study, and ablation experiments. The authors claim that Accordion is the first to convert AI-generated designs into editable layered designs and that it outperforms COLE and Open-COLE on several benchmark metrics.

Significance. If the central claim holds, Accordion addresses a practically important gap: making the output of image-generation models editable by non-experts. The paper is honest about limitations and provides a modular design that can benefit from improved vision expert models. The external DesignIntention benchmark and a small designer user study are positive elements, and the paper explicitly reports the numerical limitations (SAM IoU 68.4%, font accuracy 21.62%). However, the main quantitative evidence does not directly measure the property that constitutes the central contribution, namely layer fidelity and editability. The reported layer-level numbers are low, and the benchmark scores quantify image-level quality. The paper is therefore best read as a systems proposal with promising qualitative results, but its central claim is not yet established by the current evaluation.

major comments (4)
  1. [Table 1 and Section 4.2] The primary quantitative comparison uses GPT-4V image-level scores (design and layout, content relevance, typography and color, graphics and images, innovation). These metrics evaluate the appearance of the final rendered design, not whether the output layers are correctly segmented, clean, or truly editable. Since the paper's central contribution is 'editable layered designs,' the main evidence should include layer-level metrics such as object mask IoU, background reconstruction error, text-layer vectorization accuracy, and layer-stacking consistency. The current Table 1 does not measure editability, so the claim that Accordion produces practical layered designs is only indirectly supported.
  2. [Section 5 (limitations) and Table 2] The layer-fidelity numbers reported in the manuscript are low and directly affect the central claim: SAM achieves an IoU of 68.4%, font accuracy on original designs is 21.62%, and text recognition NED is 68.51% in Table 2. These numbers suggest that a substantial fraction of object masks and text style predictions are inaccurate, which would result in imperfect layers. The paper does not provide a quantitative evaluation of the final layered output (e.g., how often the extracted object masks align with the original reference, or how often the rendered text layers match the intended content and style). Without such evaluation, the editability claim remains fragile, especially given the acknowledged cumulative errors from misaligned bounding boxes.
  3. [Section 4.2, 'Text-to-template' paragraph] The comparison with COLE and Open-COLE is confounded by the use of different underlying text-to-image models. Accordion uses Flux to generate reference images, while COLE and Open-COLE use their own fine-tuned base models. The reported improvements in Table 1 (6.5 vs 6.3 for Open-COLE) may result from the higher visual quality of Flux references rather than from Accordion's top-down decomposition. The authors mention that Accordion can start from a user-provided reference, so a controlled experiment where COLE is applied to the same Flux references (or where Accordion is applied to COLE's generated images) would isolate the contribution of the decomposition stage. No such experiment is reported, and no error bars or significance tests are given for the 500-prompt benchmark.
  4. [Section 4.3 and Appendix J] The user study is described with very limited quantitative detail: 29 designers and 30 cases, with 73.5% of text-to-template cases deemed superior to COLE and 87.2% of sketch-to-description cases considered appropriate. The report does not state the number of test cases for each condition, the variance across designers, or a definition of 'appropriate.' More importantly, the study appears to ask whether the layered output helps further editing, but it does not measure whether the individual layers are faithful to the reference (e.g., object completeness, text correctness, background cleanliness). Given the low layer-level metrics in Table 2 and Section 5, the user study should be expanded or supplemented with objective layer-fidelity metrics.
minor comments (5)
  1. [Section 5, first paragraph] There is a typo: 'GPT evalaution score' should be 'GPT evaluation score'.
  2. [Appendix K, paragraph 1] The same sentence 'In Table 2, we present the evaluation results using various types of images as inputs' appears twice in succession.
  3. [Appendix H, first sentence] The phrase 'can produce non-layered designs' appears to be a typo; based on the content of the paragraph, it should read 'can produce non-square layered designs'.
  4. [Section 3.2] When stating that colors are mapped from [0,255] to [0,25], the paper does not explain why this range is chosen or whether it affects the VLM's learning; a brief justification or reference would improve clarity.
  5. [Section 4.2, 'De-rendering' subsection] The comparison with De-Render is purely qualitative (Figure 8). Since De-Render is an existing method with published quantitative results, reporting at least the text recognition accuracy or layer count error on a shared test set would strengthen the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the central claim is an empirical system result evaluated on an external benchmark, with acknowledged limitations that affect robustness rather than constructional circularity.

full rationale

Accordion's pipeline is a trained system, not a mathematical derivation. The VLM plan predictor is supervised on Design39K ground-truth layer annotations; the SAM/removal extraction is applied at inference; and the headline comparison is against the external DesignIntention benchmark using COLE's GPT-4V protocol plus a designer study (Table 1, Section 4.3). None of these steps define the target metric in terms of the model's own outputs. The inpainting-based augmentation (Section 4.1) creates synthetic nonsensical-text inputs from Design39K with ground-truth text taken from the original layered designs; this is standard synthetic supervision, not a fitted quantity renamed as a prediction. The paper's own limitations (Section 5: SAM IoU 68.4%, cumulative VLM bounding-box errors, text always above objects, 2,000-font constraint) weaken the editability guarantee but do not make the claim circular; they are acknowledged empirical failure modes. Self-citations (TextDiffuser, TextDiffuser-2, VLC) appear in related-work or downstream-variation contexts and are not used to justify the central result. No equation is shown to equal its input, and no parameter is fit to the evaluation benchmark and then reported as a prediction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the reliable operation of modular AI components (T2I, VLM planner, OCR, SAM, removal) and on the transfer of a VLM trained on in-house layered designs to AI-generated raster references. These are engineering assumptions rather than formal axioms, but they are load-bearing.

free parameters (4)
  • inpainting strength range = [0.5, 0.7]
    Hand-picked range for mimicking AI-generated nonsensical text; chosen because outside range gives insufficient or excessive inpainting changes.
  • VLM max output length = 1536
    Set to cover the average output sequence length of 728.32 in Design39K; hand-chosen.
  • LoRA training epochs = 6
    Chosen for training on 156,932 samples; no sensitivity analysis provided.
  • reference image normalization scale = 336
    Longer side scaled to 336 pixels following LLaVA conventions; affects VLM input resolution.
assumptions (5)
  • domain assumption Text layers are always above object layers in the visual hierarchy
    The paper's own limitation section states the extraction assumes text precedes objects, which simplifies the pipeline but restricts general applicability.
  • domain assumption SAM can segment objects from bounding boxes accurately enough
    Stage3 uses SAM conditioned on plan boxes; reported IOU is 68.4%, with failures on hollow or transparent objects, yet the pipeline depends on it.
  • domain assumption Removal models can cleanly erase text and objects
    ControlNet inpainting with the prompt 'nothing in the image' is used; the questionnaire selection mitigates but does not guarantee clean removal.
  • domain assumption Design39K training distribution transfers to AI-generated references
    The VLM plan generator is trained on in-house layered designs and applied to Flux-generated references; no explicit validation of this transfer is provided.
  • domain assumption Flux-generated images serve as representative references
    Reference creation uses Flux.1 schnell, and the paper notes Flux sometimes fails to generate text or follow prompts, so the reference quality is variable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking Layered Graphic Design Generation with a Top-Down Approach." pith.science (2026). https://pith.science/paper/YN6APXGI

@misc{pith2026250705601,
  author       = {Pith},
  title        = {Pith review of: Rethinking Layered Graphic Design Generation with a Top-Down Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YN6APXGI}},
  note         = {Machine review of arXiv:2507.05601}
}
read the original abstract

Graphic design is crucial for conveying ideas and messages. Designers usually organize their work into objects, backgrounds, and vectorized text layers to simplify editing. However, this workflow demands considerable expertise. With the rise of GenAI methods, an endless supply of high-quality graphic designs in pixel format has become more accessible, though these designs often lack editability. Despite this, non-layered designs still inspire human designers, influencing their choices in layouts and text styles, ultimately guiding the creation of layered designs. Motivated by this observation, we propose Accordion, a graphic design generation framework taking the first attempt to convert AI-generated designs into editable layered designs, meanwhile refining nonsensical AI-generated text with meaningful alternatives guided by user prompts. It is built around a vision language model (VLM) playing distinct roles in three curated stages. For each stage, we design prompts to guide the VLM in executing different tasks. Distinct from existing bottom-up methods (e.g., COLE and Open-COLE) that gradually generate elements to create layered designs, our approach works in a top-down manner by using the visually harmonious reference image as global guidance to decompose each layer. Additionally, it leverages multiple vision experts such as SAM and element removal models to facilitate the creation of graphic layers. We train our method using the in-house graphic design dataset Design39K, augmented with AI-generated design images coupled with refined ground truth created by a customized inpainting model. Experimental results and user studies by designers show that Accordion generates favorable results on the DesignIntention benchmark, including tasks such as text-to-template, adding text to background, and text de-rendering, and also excels in creating design variations.

Figures

Figures reproduced from arXiv: 2507.05601 by the authors.

Figure 1
Figure 1. Left: We convert AI-generated non-layered design refer￾ence images to layered designs by extracting background, objects, and vectorized text layers in a top-down manner with optional fur￾ther refinement. The layered representation facilitates easier edit￾ing. We include a gallery for more visualizations in Appendix A. Right: Existing bottom-up design approaches progressively gen￾erate elements on images in a predefi… view at source ↗
Figure 2
Figure 2. Overview of the proposed Accordion built around a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. In Stage2 design planning, VLM processes the reference image alongside a combined prompt to generate a comprehensive design [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Overview of the Stage3 layer generation. (a) Guided by the plan, the reference image is processed first with text removal, [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: For the same design, we utilize three types of references for the VLM, corresponding to three distinct training tasks: original [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of text-to-template show that our method uses references generated by Flux to create layered designs where [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results of adding text to the background show that our method not only places text harmoniously in terms of style and [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Qualitative results of text de-rendering. Our method groups text at the paragraph level to predict their attributes, thereby [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Our methods can support the design variations. [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 12
Figure 12. Figure 12: The VLM can convert the sketch draft into detailed prompt to generate references. [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: More examples about the questionnaire. The samples in the first row is for text removal, and the samples in the second row is [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 15
Figure 15. Figure 15: Samples generated by COLE and ours are concatenated for comparison. [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: User studies by designers to assess to task of sketch to design images. [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: User studies by designers to assess to task of text-to-templates. [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks

    cs.CV 2026-04 conditional novelty 7.0 of 10

    A 49-task layered-design benchmark shows frontier AI models leave most precision-heavy graphic design tasks unsolved, with only two tasks mostly solved.

Reference graph

Works this paper leans on

65 extracted references · 43 canonical work pages · cited by 1 Pith paper

  1. [1]

    Docsynth: a layout guided approach for controllable docu- ment image synthesis

    Sanket Biswas, Pau Riba, Josep Llad ´os, and Umapada Pal. Docsynth: a layout guided approach for controllable docu- ment image synthesis. In ICDAR, 2021. 2

    work page 2021

  2. [2]

    DocSynthv2: A Practical Autoregressive Modeling for Document Generation

    Sanket Biswas, Rajiv Jain, Vlad I Morariu, Jiuxiang Gu, Puneet Mathur, Curtis Wigington, Tong Sun, and Josep Llad´os. Docsynthv2: A practical autoregressive modeling for document generation. arXiv preprint arXiv:2406.08354,

    work page Pith review arXiv

  3. [3]

    Link: https://www.canva.com/magic-write/, 2024

    canva. Link: https://www.canva.com/magic-write/, 2024. 2

    work page 2024

  4. [4]

    Two-stage content-aware layout generation for poster designs

    Shang Chai, Liansheng Zhuang, Fengying Yan, and Zihan Zhou. Two-stage content-aware layout generation for poster designs. In ACMMM, 2023. 2

    work page 2023

  5. [5]

    Posta: A go-to framework for customized artistic poster gen- eration

    Haoyu Chen, Xiaojie Xu, Wenbo Li, Jingjing Ren, Tian Ye, Songhua Liu, Ying-Cong Chen, Lei Zhu, and Xinchao Wang. Posta: A go-to framework for customized artistic poster gen- eration. In CVPR, 2025. 2

    work page 2025

  6. [6]

    Textdiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. In NeurIPS, 2023. 1, 2, 3

    work page 2023

  7. [7]

    Textdiffuser-2: Unleashing the power of language models for text rendering

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser-2: Unleashing the power of language models for text rendering. In ECCV, 2024. 3, 17

    work page 2024

  8. [8]

    Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, 2024. 3

    work page 2024

Show all 65 references
  1. [9]

    Postercraft: Rethinking high-quality aesthetic poster generation in a unified framework

    SiXiang Chen, Jianyu Lai, Jialin Gao, Tian Ye, Haoyu Chen, Hengyu Shi, Shitong Shao, Yunlong Lin, Song Fei, Zhaohu Xing, et al. Postercraft: Rethinking high-quality aesthetic poster generation in a unified framework. arXiv preprint arXiv:2506.10741, 2025. 3

    2025 arXiv

  2. [10]

    Graphic design with large multimodal model

    Yutao Cheng, Zhao Zhang, Maoke Yang, Hui Nie, Chunyuan Li, Xinglong Wu, and Jie Shao. Graphic design with large multimodal model. arXiv preprint arXiv:2404.14368, 2024. 2

    2024 arXiv

  3. [11]

    Icdar2019 robust reading chal- lenge on arbitrary-shaped text-rrc-art

    Chee Kheng Chng, Yuliang Liu, Yipeng Sun, Chun Chet Ng, Canjie Luo, Zihan Ni, ChuanMing Fang, Shuaitao Zhang, Junyu Han, Errui Ding, et al. Icdar2019 robust reading chal- lenge on arbitrary-shaped text-rrc-art. In ICDAR, 2019. 18

    2019

  4. [12]

    Discover- ing the hidden vocabulary of dalle-2

    Giannis Daras and Alexandros G Dimakis. Discover- ing the hidden vocabulary of dalle-2. arXiv preprint arXiv:2206.00169, 2022. 3

    2022 arXiv

  5. [13]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In ICML, 2024. 3, 7

    2024

  6. [14]

    Link: https://blackforestlabs.ai/#get-flux, 2024

    flux. Link: https://blackforestlabs.ai/#get-flux, 2024. 6

    2024

  7. [15]

    Textpainter: Multi- modal text image generation with visual-harmony and text- comprehension for poster design

    Yifan Gao, Jinpeng Lin, Min Zhou, Chuanbin Liu, Hongtao Xie, Tiezheng Ge, and Yuning Jiang. Textpainter: Multi- modal text image generation with visual-harmony and text- comprehension for poster design. In ACMMM, 2023. 3

    2023

  8. [16]

    Layout- transformer: Layout generation and completion with self- attention

    Kamal Gupta, Justin Lazarow, Alessandro Achille, Larry S Davis, Vijay Mahadevan, and Abhinav Shrivastava. Layout- transformer: Layout generation and completion with self- attention. In ICCV, 2021. 2

    2021

  9. [17]

    Llms meet multimodal genera- tion and editing: A survey.arXiv preprint arXiv:2405.19334,

    Yingqing He, Zhaoyang Liu, Jingye Chen, Zeyue Tian, Hongyu Liu, Xiaowei Chi, Runtao Liu, Ruibin Yuan, Yazhou Xing, Wenhai Wang, et al. Llms meet multimodal genera- tion and editing: A survey.arXiv preprint arXiv:2405.19334,

    arXiv

  10. [18]

    Prompt-to-prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. In ICLR, 2023. 1

    2023

  11. [19]

    Retrieval-augmented layout transformer for content-aware layout generation

    Daichi Horita, Naoto Inoue, Kotaro Kikuchi, Kota Yam- aguchi, and Kiyoharu Aizawa. Retrieval-augmented layout transformer for content-aware layout generation. In CVPR,

  12. [20]

    Posterlayout: A new benchmark and ap- proach for content-aware visual-textual presentation layout

    Hsiao Yuan Hsu, Xiangteng He, Yuxin Peng, Hao Kong, and Qing Zhang. Posterlayout: A new benchmark and ap- proach for content-aware visual-textual presentation layout. In CVPR, 2023. 2

    2023

  13. [21]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2022. 5

    2022

  14. [22]

    Towards flexible multi-modal document models

    Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. Towards flexible multi-modal document models. In CVPR, 2023. 2

    2023

  15. [23]

    Opencole: Towards reproducible automatic graphic design generation

    Naoto Inoue, Kento Masui, Wataru Shimoda, and Kota Yam- aguchi. Opencole: Towards reproducible automatic graphic design generation. arXiv preprint arXiv:2406.08232, 2024. 2, 3, 7, 17

    2024 arXiv

  16. [24]

    Improving diffusion models for scene text editing with dual encoders

    Jiabao Ji, Guanhua Zhang, Zhaowen Wang, Bairu Hou, Zhifei Zhang, Brian Price, and Shiyu Chang. Improving diffusion models for scene text editing with dual encoders. Transactions on Machine Learning Research (TMLR), 2023. 3

    2023

  17. [25]

    Cole: A hierarchical generation frame- work for graphic design

    Peidong Jia, Chenxuan Li, Zeyu Liu, Yichao Shen, Xingru Chen, Yuhui Yuan, Yinglin Zheng, Dong Chen, Ji Li, Xi- aodong Xie, et al. Cole: A hierarchical generation frame- work for graphic design. arXiv preprint arXiv:2311.16974,

    arXiv

  18. [26]

    Designedit: Multi- layered latent decomposition and fusion for unified & accu- rate image editing

    Yueru Jia, Yuhui Yuan, Aosong Cheng, Chuke Wang, Ji Li, Huizhu Jia, and Shanghang Zhang. Designedit: Multi- layered latent decomposition and fusion for unified & accu- rate image editing. arXiv preprint arXiv:2403.14487, 2024. 1

    2024 arXiv

  19. [27]

    Coarse-to-fine generative modeling for graphic layouts

    Zhaoyun Jiang, Shizhao Sun, Jihua Zhu, Jian-Guang Lou, and Dongmei Zhang. Coarse-to-fine generative modeling for graphic layouts. In AAAI, 2022. 2

    2022

  20. [28]

    Layoutvae: Stochastic scene layout gen- eration from a label set

    Akash Abdu Jyothi, Thibaut Durand, Jiawei He, Leonid Si- gal, and Greg Mori. Layoutvae: Stochastic scene layout gen- eration from a label set. In ICCV, 2019

    2019

  21. [29]

    Multimodal markup docu- ment models for graphic design completion

    Kotaro Kikuchi, Naoto Inoue, Mayu Otani, Edgar Simo- Serra, and Kota Yamaguchi. Multimodal markup docu- ment models for graphic design completion. arXiv preprint arXiv:2409.19051, 2024. 2

    2024

  22. [30]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In ICCV, 2023. 2, 5, 17

    2023

  23. [31]

    Flux-text: A simple and advanced diffusion transformer baseline for scene text editing

    Rui Lan, Yancheng Bai, Xu Duan, Mingxing Li, Lei Sun, and Xiangxiang Chu. Flux-text: A simple and advanced diffusion transformer baseline for scene text editing. arXiv preprint arXiv:2505.03329, 2025. 3

    2025

  24. [32]

    Har- monious textual layout generation over natural images via deep aesthetics learning

    Chenhui Li, Peiying Zhang, and Changbo Wang. Har- monious textual layout generation over natural images via deep aesthetics learning. IEEE Transactions on Multimedia (TMM), 2021. 2

    2021

  25. [33]

    Layoutgan: Generating graphic layouts with wireframe discriminators

    Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, and Tingfa Xu. Layoutgan: Generating graphic layouts with wireframe discriminators. In ICLR, 2019

    2019

  26. [34]

    Planning and rendering: Towards end-to-end product poster generation

    Zhaochen Li, Fengheng Li, Wei Feng, Honghe Zhu, An Liu, Yaoyu Li, Zheng Zhang, Jingjing Lv, Xin Zhu, Junjie Shen, et al. Planning and rendering: Towards end-to-end product poster generation. arXiv preprint arXiv:2312.08822, 2023

    2023 arXiv

  27. [35]

    Textcengen: Attention-guided text-centric background adaptation for text-to-image generation

    Tianyi Liang, Jiangqi Liu, Sicheng Song, Shiqi Jiang, Yifei Huang, Changbo Wang, and Chenhui Li. Textcengen: Attention-guided text-centric background adaptation for text-to-image generation. arXiv preprint arXiv:2404.11824, 2024

    2024 arXiv

  28. [36]

    Autoposter: A highly automatic and content-aware design system for ad- vertising poster generation

    Jinpeng Lin, Min Zhou, Ye Ma, Yifan Gao, Chenxi Fei, Yangjian Chen, Zhang Yu, and Tiezheng Ge. Autoposter: A highly automatic and content-aware design system for ad- vertising poster generation. In ACMMM, 2023. 2

    2023

  29. [37]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023. 5

    2023

  30. [38]

    Glyph-byt5: A customized text encoder for accurate visual text rendering

    Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, and Yuhui Yuan. Glyph-byt5: A customized text encoder for accurate visual text rendering. In ECCV, 2024. 3

    2024

  31. [39]

    Glyph-byt5-v2: A strong aesthetic base- line for accurate multilingual visual text rendering

    Zeyu Liu, Weicong Liang, Yiming Zhao, Bohan Chen, Ji Li, and Yuhui Yuan. Glyph-byt5-v2: A strong aesthetic base- line for accurate multilingual visual text rendering. arXiv preprint arXiv:2406.10208, 2024

    2024 arXiv

  32. [40]

    Glyphdraw: Seamlessly ren- dering text with intricate spatial structures in text-to-image generation

    Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, and Xiaodong Lin. Glyphdraw: Seamlessly ren- dering text with intricate spatial structures in text-to-image generation. arXiv preprint arXiv:2303.17870, 2023

    2023 arXiv

  33. [41]

    Glyphdraw2: Automatic generation of complex glyph posters with diffusion models and large language models

    Jian Ma, Yonglin Deng, Chen Chen, Haonan Lu, and Zhenyu Yang. Glyphdraw2: Automatic generation of complex glyph posters with diffusion models and large language models. arXiv preprint arXiv:2407.02252, 2024. 3

    2024 arXiv

  34. [42]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In ICLR, 2023. 3

    2023

  35. [43]

    Art: Anonymous region transformer for variable multi-layer transparent image generation

    Yifan Pu, Yiming Zhao, Zhicong Tang, Ruihong Yin, Haox- ing Ye, Yuhui Yuan, Dong Chen, Jianmin Bao, Sirui Zhang, Yanbin Wang, et al. Art: Anonymous region transformer for variable multi-layer transparent image generation. In CVPR,

  36. [44]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 2, 3, 5, 17

    2022

  37. [45]

    Posterllama: Bridging design ability of langauge model to contents-aware layout generation

    Jaejung Seol, Seojun Kim, and Jaejun Yoo. Posterllama: Bridging design ability of langauge model to contents-aware layout generation. arXiv preprint arXiv:2404.00995, 2024. 2

    2024 arXiv

  38. [46]

    Visual layout composer: Image-vector dual diffusion model for de- sign layout generation

    Mohammad Amin Shabani, Zhaowen Wang, Difan Liu, Nanxuan Zhao, Jimei Yang, and Yasutaka Furukawa. Visual layout composer: Image-vector dual diffusion model for de- sign layout generation. In CVPR, 2024. 2, 9

    2024

  39. [47]

    De-rendering stylized texts

    Wataru Shimoda, Daichi Haraguchi, Seiichi Uchida, and Kota Yamaguchi. De-rendering stylized texts. In ICCV,

  40. [48]

    Mulan: A multi layer annotated dataset for controllable text-to-image generation

    Petru-Daniel Tudosiu, Yongxin Yang, Shifeng Zhang, Fei Chen, Steven McDonagh, Gerasimos Lampouras, Ignacio Ia- cobacci, and Sarah Parisot. Mulan: A multi layer annotated dataset for controllable text-to-image generation. In CVPR,

  41. [49]

    Anytext: Multilingual visual text gener- ation and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text gener- ation and editing. In ICLR, 2024. 3, 17

    2024

  42. [50]

    Versatile diffusion: Text, images and variations all in one diffusion model

    Xingqian Xu, Zhangyang Wang, Gong Zhang, Kai Wang, and Humphrey Shi. Versatile diffusion: Text, images and variations all in one diffusion model. In ICCV, 2023. 9

    2023

  43. [51]

    Canvasvae: Learning to generate vector graphic documents

    Kota Yamaguchi. Canvasvae: Learning to generate vector graphic documents. In ICCV, 2021. 2

    2021

  44. [52]

    Layoutdetr: detection transformer is a good mul- timodal layout designer

    Ning Yu, Chia-Chih Chen, Zeyuan Chen, Rui Meng, Gang Wu, Paul Josel, Juan Carlos Niebles, Caiming Xiong, and Ran Xu. Layoutdetr: detection transformer is a good mul- timodal layout designer. arXiv preprint arXiv:2212.09877 ,

    arXiv

  45. [53]

    Textctrl: Diffusion-based scene text editing with prior guidance control

    Weichao Zeng, Yan Shu, Zhenhang Li, Dongbao Yang, and Yu Zhou. Textctrl: Diffusion-based scene text editing with prior guidance control. In NeurIPS, 2024. 3

    2024

  46. [54]

    Layoutdiffusion: Improving graphic lay- out generation by discrete diffusion probabilistic models

    Junyi Zhang, Jiaqi Guo, Shizhao Sun, Jian-Guang Lou, and Dongmei Zhang. Layoutdiffusion: Improving graphic lay- out generation by discrete diffusion probabilistic models. In ICCV, 2023. 2

    2023

  47. [55]

    Transparent image layer diffusion using latent transparency

    Lvmin Zhang and Maneesh Agrawala. Transparent image layer diffusion using latent transparency. ACM TRANSAC- TIONS ON GRAPHICS (TOG), 2024. 2

    2024

  48. [56]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 5

    2023

  49. [57]

    Brush your text: Synthesize any scene text on im- ages via diffusion model

    Lingjun Zhang, Xinyuan Chen, Yaohui Wang, Yue Lu, and Yu Qiao. Brush your text: Synthesize any scene text on im- ages via diffusion model. In AAAI, 2024. 3

    2024

  50. [58]

    Text2layer: Layered image generation using latent diffusion model

    Xinyang Zhang, Wentian Zhao, Xin Lu, and Jeff Chien. Text2layer: Layered image generation using latent diffusion model. arXiv preprint arXiv:2307.09781, 2023. 2

    2023 arXiv

  51. [59]

    Layout-agnostic scene text image synthesis with diffusion models

    Qilong Zhangli, Jindong Jiang, Di Liu, Licheng Yu, Xi- aoliang Dai, Ankit Ramchandani, Guan Pang, Dimitris N Metaxas, and Praveen Krishnan. Layout-agnostic scene text image synthesis with diffusion models. In CVPR, 2024. 3

    2024

  52. [60]

    Modeling fonts in context: Font prediction on web designs

    Nanxuan Zhao, Ying Cao, and Rynson WH Lau. Modeling fonts in context: Font prediction on web designs. In Com- puter Graphics Forum, 2018. 2

    2018

  53. [61]

    Udifftext: A unified frame- work for high-quality text synthesis in arbitrary images via character-aware diffusion models

    Yiming Zhao and Zhouhui Lian. Udifftext: A unified frame- work for high-quality text synthesis in arbitrary images via character-aware diffusion models. In ECCV, 2024. 3

    2024

  54. [62]

    Harmonizing visual text comprehension and gen- eration

    Zhen Zhao, Jingqun Tang, Binghong Wu, Chunhui Lin, Shu Wei, Hao Liu, Xin Tan, Zhizhong Zhang, Can Huang, and Yuan Xie. Harmonizing visual text comprehension and gen- eration. In NeurIPS, 2024. 3

    2024

  55. [63]

    Content-aware generative modeling of graphic design layouts

    Xinru Zheng, Xiaotian Qiao, Ying Cao, and Rynson WH Lau. Content-aware generative modeling of graphic design layouts. ACM Transactions on Graphics (TOG), 2019. 2

    2019

  56. [64]

    Composition-aware graphic layout gan for visual-textual presentation designs

    Min Zhou, Chenchen Xu, Ye Ma, Tiezheng Ge, Yuning Jiang, and Weiwei Xu. Composition-aware graphic layout gan for visual-textual presentation designs. arXiv preprint arXiv:2205.00303, 2022

    2022 arXiv

  57. [65]

    Gelest life, that leavesbehind It a swath of glory,and often enough, a tear

    Wanrong Zhu, Jennifer Healey, Ruiyi Zhang, William Yang Wang, and Tong Sun. Automatic layout planning for visually-rich documents with instruction-following models. arXiv preprint arXiv:2404.15271, 2024. 2 Appendix A. Gallery for more text-to-template results We demonstrate mo...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.