Pith. sign in

REVIEW 4 major objections 5 minor 55 references

UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that conditioning a diffusion model on pixel-level text segmentation masks — which retain glyph shape, font, and color at full resolution — lets a single ControlNet-style architecture outperform multi-branch systems at…

desk verdict Promising but under-specified: the inference-time condition may not be the segmentation masks that make the approach new. read the letter →

arxiv 2507.00992 v2 pith:7GGXPAKE submitted 2025-07-01 cs.CV

Yuanrui Wang , Cong Han , Yafei Li , Zhipeng Jin , Xiawei Li , SiNan Du , Wen Tao , Yi Yang
show 3 more authors
Shuanglong Li Chun Yuan Liu Lin
This is my paper
classification cs.CV
keywords visualtextgenerationdiffusionmodelssegmentationmasksControlNetconditioningbilingualrenderingflowmatchingsmallsynthesistypographicstylepreservation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that text rendered inside AI-generated images comes out blurry or wrong largely because the usual conditioning signal — a pre-rendered glyph image — throws away the original font and color information. Its proposal is to condition the diffusion model on pixel-level text segmentation masks instead, because those masks carry glyph shape, position, font style, and color at full image resolution. The paper reports sentence accuracy of 0.8267 for Chinese and 0.9018 for English on the AnyText-benchmark, and 0.7925 on a new small-text benchmark where the best prior method reaches 0.3881. If those numbers hold, one clean control branch could replace the multi-module text-rendering pipelines used today.

What carries the argument

The central object is the pixel-level visual text mask used as the unified glyph condition. The mask is produced by a fine-tuned bilingual segmentation model; for a glyph region, the condition is either the segmentation edges plus the mask-multiplied original image when characters are large, or a blurred crop of the original bounded by the OCR-derived position mask when characters are small. This adaptive glyph condition, together with a glyph-region loss that weights the reconstruction error inside the mask, is what carries the argument: it preserves shape, font, and color at full resolution while compensating for segmentation failure on tiny characters.

What would settle it

Compare MiniText sentence accuracy when the inference condition is the raw rendered glyph image versus that same image passed through the segmentation model and adaptive blending; if the raw condition performs as well, segmentation conditioning is not the active source of the improvement.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that full typographic signal can be preserved in the conditioning pathway by replacing rendered glyph images with segmentation maps. A fine-tuned bilingual segmentation model extracts a mask from a source image; for small glyph regions, where segmentation is unreliable, the mask is adaptively blended with the original image region and a Gaussian-smoothed boundary. That glyph condition is fed through a ControlNet into a flow-matching diffusion transformer, and a glyph-region loss computes reconstruction error only inside the masked text areas. The result, the paper argues, is that a single ControlNet branch achieves state-of-the-art sentence accuracy on Chinese and English benchmarks and particularly strong performance on small and complex text, without the auxiliary modules used by prior methods.

Load-bearing premise

The method depends on the assumption that the text image drawn by the layout module at generation time is turned into the same type of text mask condition the model saw during training; the paper does not spell out that conversion step.

Editorial extensions

If this is right

  • A single ControlNet branch can carry all the typographic information that prior systems split across positional-encoding and style-control modules, so text-rendering pipelines need not grow extra branches.
  • Small-text generation, where prior methods collapse, becomes usable: the reported MiniText sentence accuracy is 0.7925 versus 0.3881 for the best prior method.
  • One condition format covers both Chinese and English, so multilingual visual text generation does not require language-specific auxiliary networks.
  • The new GlyphMM-3M and Poster-100K datasets plus the GlyphMM and MiniText benchmarks give the field a way to measure layout consistency and tiny-glyph quality that earlier benchmarks did not isolate.
  • The reported sample budget is modest (460,000 steps at batch size 16 from a pretrained initialization), suggesting segmentation-conditioned training is sample-efficient relative to prior glyph-rendering models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If mask conditioning transfers, any user-supplied stylized text image — a logo, a handwritten word, a custom font sample — could serve directly as the condition, turning the pipeline into a text-editing or font-transfer tool without retraining.
  • Because the adaptive blend exists to compensate for segmentation failures on tiny characters, better small-glyph segmentation should directly raise MiniText accuracy further.
  • The glyph-region loss is essentially importance weighting on the reconstruction objective, so the same trick could be applied to other diffusion tasks where a mask identifies the region that matters most.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes UniGlyph, a diffusion-based framework for visual text generation that replaces pre-rendered glyph images with pixel-level text segmentation masks as the conditioning signal. The method fine-tunes SAM-TS on a bilingual dataset to produce text masks, builds an adaptive glyph condition (Eq. 3) that combines Canny edges, masked image content, and a blurred small-character branch, and adds a glyph-region loss to a FLUX.1-dev-based DiT with ControlNet. The authors introduce two datasets (GlyphMM-3M, Poster-100K) and two benchmarks (GlyphMM-benchmark, MiniText-benchmark), and report state-of-the-art results on these benchmarks and on the AnyText-benchmark, especially for small text.

Significance. If the central claim is substantiated, the paper could simplify ControlNet-based text rendering by removing auxiliary modules for font/color control, and it demonstrates a promising direction for small-text generation. The construction of large bilingual text-image datasets and new benchmarks is a useful community contribution. However, the evaluation currently rests on an under-specified inference pipeline, single-seed results, a self-trained baseline, and benchmarks whose disjointness from the training data is not demonstrated. The core mechanism—segmentation-conditioned generation—is plausible but not yet rigorously established.

major comments (4)
  1. [Section 3, Eq. (3), Section 3.5] The paper does not state whether the rendered glyph image produced by the glyph renderer at inference is converted into the adaptive glyph condition G of Eq. (3) before being fed to the ControlNet. Section 3 says the segmentation model is "used during training" and Section 3.5 says only that the glyph renderer "serves as a condition for generation." If the raw rendered image is used directly, the inference-time condition distribution differs from the training-time G (no Canny edge emphasis, no small-text blurred branch, potentially different colors/backgrounds), which would invalidate the claimed segmentation-conditioned paradigm. Please specify the exact inference pipeline, including whether SAM-TS, PP-OCRv4, Canny, and adaptive blending are applied to the rendered image; if they are applied, update the text and figure accordingly; if they are not, the central claim is unsupported.
  2. [Section 4, Table 1, Table 2] The new benchmarks GlyphMM-benchmark and MiniText-benchmark are author-created, and the paper does not demonstrate that their images are disjoint from the GlyphMM-3M and Poster-100K training sets. Given the naming overlap and the use of Chinese poster images in both training and MiniText-benchmark, training/test contamination could inflate the reported gains. Please report the overlap analysis (e.g., exact/near-duplicate detection, or a clear statement of how benchmark images were selected from outside the training pools) and re-evaluate on a disjoint subset if overlap exists.
  3. [Section 5.2, Table 1] The quantitative comparison relies on results from a single seed (seed 0) with no error bars, on a self-trained GlyphDraw2 implementation with unavailable official weights, and on AnyText2 numbers quoted directly from its paper. These choices make it difficult to assess whether the reported improvements are significant or an artifact of re-implementation. Please provide multiple seeds (or at least variance estimates), clearly mark which numbers are from the original papers vs. re-implementations, and, for GlyphDraw2, either release the implementation or compare it against the official version on a small subset to quantify the re-implementation gap.
  4. [Section 5.4, Table 4] The ablation of the adaptive glyph condition does not support the text's claim that Gaussian blur improves small-text generation. Compared to w/o Gaussian Blur, UniGlyph with blur has slightly lower Sen.Acc (0.7849 vs. 0.7851) and lower NED (0.9507 vs. 0.9508); only ClipScore improves (0.7963 to 0.8097). Since the blur is specifically motivated to smooth small-text boundaries, the absence of an accuracy gain should be discussed, and the sentence "the generation accuracy did not decline" is contradicted by the reported numbers.
minor comments (5)
  1. [Section 5.4, Table 3] The differences in Sen.Acc across λ values in Table 3 are within the third decimal place (0.8179–0.8188), so the claim that λ=1 is optimal is not robust without error bars or significance testing.
  2. [Section 5.1.2] The calculation "16 × 460,000 = 7.36 million samples" is unclear: please specify that 460,000 is the total number of training steps (300k + 160k) and clarify the relationship between the stated subset of GlyphMM-3M and the actual unique images used.
  3. [Section 5.2] The phrase "our effective training data volume" conflates batch-size-adjusted steps with unique data samples; this is not a meaningful measure of data consumption or sample efficiency, and the claim of "sample efficiency" should be made with a more direct comparison (e.g., number of unique training images).
  4. [Section 3.2] There is a typo "Gaussain" in the text following Eq. (3), and the sentence "which contains comprehensive glyph information" after Eq. (1) is a sentence fragment.
  5. [Section 5.4] In the paragraph before Table 4, "wWe validated" should be "We validated."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is independently grounded by the external AnyText-benchmark comparison.

full rationale

The paper's main derivation is not circular. Training uses a fine-tuned SAM-TS segmentation model to produce pixel-level masks (Eq. 1), an adaptive glyph condition built from those masks (Eq. 3), a flow-matching objective (Eq. 5), and a glyph-region loss (Eq. 7). The reported state-of-the-art results on the AnyText-benchmark — an externally defined benchmark with published baselines (AnyText, GlyphControl, GlyphDraw2, Glyph-ByT5-v2) — provide independent evidence for the effectiveness of the approach. That result does not reduce to a fit by construction, because the benchmark and its metrics are not defined in terms of the paper's own glyph-conditioning formula. The two newly introduced benchmarks, GlyphMM-benchmark and MiniText-benchmark, are author-created, and the paper does not state explicitly whether they are disjoint from the GlyphMM-3M and Poster-100K training sets; if they overlapped, those particular numbers would carry less evidential weight. However, the absence of this statement is an evaluation-transparency concern, not a demonstrated circular derivation, and the AnyText-benchmark result stands independently. The potential ambiguity about whether the inference-time rendered glyph image is passed through the same segmentation and adaptive-blending pipeline described in Eq. 3 is a methodological clarity issue, not a self-referential loop: it does not make any reported metric equal to a training input by construction. The paper also does not rely on any load-bearing self-citation; citations to Hi-SAM, BTS, FLUX, and PP-OCRv4 are external sources that supply components or training data rather than the paper's own conclusions. Overall, the derivation chain is self-contained with respect to the central claim, and no specific circular step can be exhibited from the paper's equations or text.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a handful of explicit modeling choices (threshold, loss weight, blur) and assumptions about the quality and representativeness of the segmentation model and benchmarks. No code, data, or benchmark releases are provided, so these choices cannot be independently checked.

free parameters (4)
  • adaptive blending threshold T = 4900 pixels
    Introduced in Eq. 2-3 to switch between segmentation-derived condition and blurred crop for small glyph regions; chosen by the authors based on observed segmentation inaccuracies, not derived from data.
  • glyph region loss weight lambda = 1
    Selected via ablation in Table 3; differences between lambda values are within 0.001 Sen.Acc, so the choice is not clearly determined.
  • Gaussian blur sigma = not specified
    Used in boundary smoothing for small glyph crops in Eq. 3; value not reported.
  • CFG scale and control scale = 0.9 for both
    Fixed for all methods; not tuned per model, which may disadvantage some baselines.
assumptions (4)
  • domain assumption The fine-tuned SAM-TS segmentation model produces masks that preserve shape, position, font style, and color at full resolution.
    Central premise of the conditioning signal; not quantitatively validated in the paper beyond qualitative figures.
  • standard math Flow matching loss (Eq. 5) and the VAE encoder/decoder behave as standard in FLUX.1-dev training.
    Standard background for diffusion model training.
  • domain assumption The threshold T=4900 and the adaptive blending strategy generalize across scripts (Chinese, English), fonts, and image resolutions.
    The threshold is a fixed constant; no sensitivity analysis across font sizes or scripts is provided.
  • domain assumption The GlyphMM-benchmark and MiniText-benchmark are drawn from a distribution representative of real text-image tasks and disjoint from the training sets.
    No release or overlap analysis is given; benchmarks are author-created.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis." pith.science (2026). https://pith.science/paper/7GGXPAKE

@misc{pith2026250700992,
  author       = {Pith},
  title        = {Pith review of: UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7GGXPAKE}},
  note         = {Machine review of arXiv:2507.00992}
}
read the original abstract

Text-to-image generation has greatly advanced content creation, yet accurately rendering visual text remains a key challenge due to blurred glyphs, semantic drift, and limited style control. Existing methods often rely on pre-rendered glyph images as conditions, but these struggle to retain original font styles and color cues, necessitating complex multi-branch designs that increase model overhead and reduce flexibility. To address these issues, we propose a segmentation-guided framework that uses pixel-level visual text masks -- rich in glyph shape, color, and spatial detail -- as unified conditional inputs. Our method introduces two core components: (1) a fine-tuned bilingual segmentation model for precise text mask extraction, and (2) a streamlined diffusion model augmented with adaptive glyph conditioning and a region-specific loss to preserve textual fidelity in both content and style. Our approach achieves state-of-the-art performance on the AnyText benchmark, significantly surpassing prior methods in both Chinese and English settings. To enable more rigorous evaluation, we also introduce two new benchmarks: GlyphMM-benchmark for testing layout and glyph consistency in complex typesetting, and MiniText-benchmark for assessing generation quality in small-scale text regions. Experimental results show that our model outperforms existing methods by a large margin in both scenarios, particularly excelling at small text rendering and complex layout preservation, validating its strong generalization and deployment readiness.

Figures

Figures reproduced from arXiv: 2507.00992 by the authors.

Figure 1
Figure 1. The UniGlyph framework: (1) A dual-objective diffusion model (flow matching + glyph region loss) ensures glyph feature [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The segmentation results are presented in three columns: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The segmentation performance on extremely compact [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison between UniGlyph and state-of-the-art models, primarily illustrating English text generation, with the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparative results of UniGlyph and other state-of-the-art models in small-character generation, highlighting UniGlyph’s [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 31 canonical work pages

  1. [1]

    BlackForestLab. Flux.1. https : / / blackforestlabs . ai / announcing - black - forest-labs/, 2024. 1

    work page 2024

  2. [2]

    Textdiffuser-2: Unleashing the power of language models for text rendering

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser-2: Unleashing the power of language models for text rendering. arXiv preprint arXiv:2311.16465, 2023. 2

    arXiv 2023

  3. [3]

    Textdiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. arXiv preprint, abs/2305.10855, 2023. 1, 2

    arXiv 2023

  4. [4]

    Control3d: Towards controllable text-to-3d generation,

    Yang Chen, Yingwei Pan, Yehao Li, Ting Yao, and Tao Mei. Control3d: Towards controllable text-to-3d generation,

    work page

  5. [5]

    Whoever started the interference should end it: Guiding data-free model merging via task vectors

    Runxi Cheng, Feng Xiong, Yongxian Wei, Wanyun Zhu, and Chun Yuan. Whoever started the interference should end it: Guiding data-free model merging via task vectors. arXiv preprint arXiv:2503.08099, 2025. 1

    arXiv 2025

  6. [6]

    ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts

    Sinan Du, Guosheng Zhang, Keyao Wang, Yuanrui Wang, Haixiao Yue, Gang Zhang, Errui Ding, Jingdong Wang, Zhengzhuo Xu, and Chun Yuan. Alore: Efficient visual adaptation via aggregating low rank experts. arXiv preprint arXiv:2412.08341, 2024

    work page Pith review arXiv 2024

  7. [7]

    Scaling rec- tified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rec- tified flow transformers for high-resolution image synthesis. In ICML, 2024. 1, 4, 5

    work page 2024

  8. [8]

    Layoutgpt: Compositional visual plan- ning and generation with large language models, 2023

    Weixi Feng, Wanrong Zhu, Tsu jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual plan- ning and generation with large language models, 2023. 2

    work page 2023

Show all 55 references
  1. [9]

    Wukong: 100 million large-scale chinese cross-modal pre-training dataset and A foundation frame- work

    Jiaxi Gu, Xiaojun Meng, Guansong Lu, Lu Hou, Minzhe Niu, Hang Xu, Xiaodan Liang, Wei Zhang, Xin Jiang, and Chunjing Xu. Wukong: 100 million large-scale chinese cross-modal pre-training dataset and A foundation frame- work. CoRR, abs/2202.06767, 2022. 5

    2022 arXiv

  2. [10]

    En- hancing logits distillation with plug&play kendall’s τ rank- ing loss, 2025

    Yuchen Guan, Runxi Cheng, Kang Liu, and Chun Yuan. En- hancing logits distillation with plug&play kendall’s τ rank- ing loss, 2025. 1

    2025

  3. [11]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. CoRR,

  4. [12]

    Composer: Creative and controllable im- age synthesis with composable conditions

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: Creative and controllable im- age synthesis with composable conditions. arXiv preprint, abs/2302.09778, 2023. 2

    2023 arXiv

  5. [13]

    Improving diffusion models for scene text editing with dual encoders,

    Jiabao Ji, Guanhua Zhang, Zhaowen Wang, Bairu Hou, Zhifei Zhang, Brian Price, and Shiyu Chang. Improving diffusion models for scene text editing with dual encoders,

  6. [14]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding vari- ational bayes. In ICLR, 2014. 4

    2014

  7. [15]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 3

    2023 arXiv

  8. [16]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 5

    2024

  9. [17]

    Layoutprompter: Awaken the design ability of large language models, 2023

    Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang James Yang, Jian-Guang Lou, and Dongmei Zhang. Layoutprompter: Awaken the design ability of large language models, 2023. 2

    2023

  10. [18]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi- milian Nickel, and Matt Le. Flow matching for generative modeling, 2023. 4

    2023

  11. [19]

    Character-aware models improve visual text rendering

    Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mo- hammad Norouzi, and Noah Constant. Character-aware models improve visual text rendering. InACL, pages 16270– 16297, 2023. 2

    2023

  12. [20]

    Glyph-byt5: A customized text encoder for accurate visual text rendering.arXiv preprint arXiv:2403.09622, 2024

    Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, and Yuhui Yuan. Glyph-byt5: A customized text encoder for accurate visual text rendering.arXiv preprint arXiv:2403.09622, 2024. 2

    2024 arXiv

  13. [21]

    Glyph-byt5-v2: A strong aesthetic base- line for accurate multilingual visual text rendering

    Zeyu Liu, Weicong Liang, Yiming Zhao, Bohan Chen, Ji Li, and Yuhui Yuan. Glyph-byt5-v2: A strong aesthetic base- line for accurate multilingual visual text rendering. arXiv preprint arXiv:2406.10208, 2024. 1, 5

    2024 arXiv

  14. [22]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. 5

    2019

  15. [23]

    Glyphdraw2: Automatic generation of complex glyph posters with diffusion models and large language models

    Jian Ma, Yonglin Deng, Chen Chen, Haonan Lu, and Zhenyu Yang. Glyphdraw2: Automatic generation of complex glyph posters with diffusion models and large language models. CoRR, 2024. 1, 2, 5

    2024

  16. [24]

    Chargen: High accurate character- level visual text generation model with multimodal encoder,

    Lichen Ma, Tiezhu Yue, Pei Fu, Yujie Zhong, Kai Zhou, Xi- aoming Wei, and Jie Hu. Chargen: High accurate character- level visual text generation model with multimodal encoder,

  17. [25]

    Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation

    Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, et al. Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation. In SIGGRAPH Asia 2024 Conference Papers, pages 1–12, 2024. 1

    2024

  18. [26]

    Follow-your-creation: Empowering 4d creation through video inpainting

    Yue Ma, Kunyu Feng, Xinhua Zhang, Hongyu Liu, David Junhao Zhang, Jinbo Xing, Yinhan Zhang, Ayden Yang, Zeyu Wang, and Qifeng Chen. Follow-your-creation: Empowering 4d creation through video inpainting. arXiv preprint arXiv:2506.04590, 2025

    2025 arXiv

  19. [27]

    Follow-your-click: Open-domain regional image animation via motion prompts

    Yue Ma, Yingqing He, Hongfa Wang, Andong Wang, Leqi Shen, Chenyang Qi, Jixuan Ying, Chengfei Cai, Zhifeng Li, Heung-Yeung Shum, et al. Follow-your-click: Open-domain regional image animation via motion prompts. In Proceed- ings of the AAAI Conference on Artificial Intelligence...

    2025

  20. [28]

    Follow-your-motion: Video motion transfer via efficient spatial-temporal decoupled finetuning

    Yue Ma, Yulong Liu, Qiyuan Zhu, Ayden Yang, Kunyu Feng, Xinhua Zhang, Zhifeng Li, Sirui Han, Chenyang Qi, and Qifeng Chen. Follow-your-motion: Video motion transfer via efficient spatial-temporal decoupled finetuning. arXiv preprint arXiv:2506.05207, 2025. 1

    2025

  21. [29]

    T2i-adapter: Learning 9 adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Jian Zhang, Zhon- gang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning 9 adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint, abs/2302.08453, 2023. 2

    2023 arXiv

  22. [30]

    OpenAI. Dall·e3. https://openai.com/index/ dall-e-3/, 2023. 1

    2023

  23. [31]

    Pp-ocrv4

    PaddlePaddle. Pp-ocrv4. https : / / github . com / PaddlePaddle/PaddleOCR/blob/release/2.7/ doc/doc_ch/PP-OCRv4_introduction.md , 2023. 4

    2023

  24. [32]

    Customtext: Customized tex- tual image generation using diffusion models, 2024

    Shubham Paliwal, Arushi Jain, Monika Sharma, Vikram Jamwal, and Lovekesh Vig. Customtext: Customized tex- tual image generation using diffusion models, 2024. 2

    2024

  25. [33]

    Scalable diffusion models with transformers, 2023

    William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. 5

    2023

  26. [34]

    LAION- 400M: open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. LAION- 400M: open dataset of clip-filtered 400 million image-text pairs. CoRR, abs/2111.02114, 2021. 5

    2021 arXiv

  27. [35]

    Gsrender: Dedupli- cated occupancy prediction via weakly supervised 3d gaus- sian splatting, 2024

    Qianpu Sun, Changyong Shu, Sifan Zhou, Zichen Yu, Yan Chen, Dawei Yang, and Yuan Chun. Gsrender: Dedupli- cated occupancy prediction via weakly supervised 3d gaus- sian splatting, 2024. 1

    2024

  28. [36]

    Llama: Open and efficient foundation lan- guage models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aure- lien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. Llama: Open and efficient foundation la...

    2023

  29. [37]

    Llama 2: Open foundation and fine- tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fer- nandes, Jere...

    2023

  30. [38]

    Anytext: Multilingual visual text gener- ation and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text gener- ation and editing. arXiv, 2023. 1, 2, 5, 7

    2023

  31. [39]

    Anytext2: Vi- sual text generation and editing with customizable attributes,

    Yuxiang Tuo, Yifeng Geng, and Liefeng Bo. Anytext2: Vi- sual text generation and editing with customizable attributes,

  32. [40]

    Rectified diffusion: Straightness is not your need in rectified flow, 2024

    Fu-Yun Wang, Ling Yang, Zhaoyang Huang, Mengdi Wang, and Hongsheng Li. Rectified diffusion: Straightness is not your need in rectified flow, 2024. 4

    2024

  33. [41]

    Perceive, understand and restore: Real-world image super- resolution with autoregressive multimodal generative mod- els, 2025

    Hongyang Wei, Shuaizheng Liu, Chun Yuan, and Lei Zhang. Perceive, understand and restore: Real-world image super- resolution with autoregressive multimodal generative mod- els, 2025. 1

    2025

  34. [42]

    Modeling multi-task model merg- ing as adaptive projective gradient descent

    Yongxian Wei, Anke Tang, Li Shen, Zixuan Hu, Chun Yuan, and Xiaochun Cao. Modeling multi-task model merg- ing as adaptive projective gradient descent. arXiv preprint arXiv:2501.01230, 2025. 1

    2025 arXiv

  35. [43]

    Bts: a bi-lingual benchmark for text segmentation in the wild

    Xixi Xu, Zhongang Qi, Jianqi Ma, Honglun Zhang, Ying Shan, and Xiaohu Qie. Bts: a bi-lingual benchmark for text segmentation in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19152–19162, 2022. 3, 5

    2022

  36. [44]

    Chartbench: A benchmark for complex visual reasoning in charts

    Zhengzhuo Xu, Sinan Du, Yiyan Qi, Chengjin Xu, Chun Yuan, and Jian Guo. Chartbench: A benchmark for complex visual reasoning in charts. arXiv preprint arXiv:2312.15915,

    arXiv

  37. [45]

    Chartmoe: Mixture of di- versely aligned expert connector for chart understanding

    Zhengzhuo Xu, Bowen Qu, Yiyan Qi, SiNan Du, Chengjin Xu, Chun Yuan, and Jian Guo. Chartmoe: Mixture of di- versely aligned expert connector for chart understanding. In The Thirteenth International Conference on Learning Rep- resentations, 2025. 1

    2025

  38. [46]

    Glyphcontrol: Glyph condi- tional control for visual text generation

    Yukang Yang, Dongnan Gui, Yuhui Yuan, Haisong Ding, Han Hu, and Kai Chen. Glyphcontrol: Glyph condi- tional control for visual text generation. arXiv preprint , abs/2305.18259, 2023. 2, 5

    2023 arXiv

  39. [47]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint, 2023. 2

    2023

  40. [48]

    Hi-sam: Marrying segment anything model for hierarchical text segmentation

    Maoyuan Ye, Jing Zhang, Juhua Liu, Chenyu Liu, Baocai Yin, Cong Liu, Bo Du, and Dacheng Tao. Hi-sam: Marrying segment anything model for hierarchical text segmentation. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, pages 1–16, 2024. 2, 3, 5

    2024

  41. [49]

    Artist: Improving the generation of text-rich images with disentan- gled diffusion models and large language models, 2024

    Jianyi Zhang, Yufan Zhou, Jiuxiang Gu, Curtis Wigington, Tong Yu, Yiran Chen, Tong Sun, and Ruiyi Zhang. Artist: Improving the generation of text-rich images with disentan- gled diffusion models and large language models, 2024. 2

    2024

  42. [50]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. arXiv preprint, abs/2302.05543, 2023. 2, 5

    2023 arXiv

  43. [51]

    Brush your text: Synthesize any scene text on im- ages via diffusion model, 2023

    Lingjun Zhang, Xinyuan Chen, Yaohui Wang, Yue Lu, and Yu Qiao. Brush your text: Synthesize any scene text on im- ages via diffusion model, 2023. 2

    2023

  44. [52]

    Metaxas, and Praveen Krishnan

    Qilong Zhangli, Jindong Jiang, Di Liu, Licheng Yu, Xi- aoliang Dai, Ankit Ramchandani, Guan Pang, Dimitris N. Metaxas, and Praveen Krishnan. Layout agnostic scene text image synthesis with diffusion models, 2024. 2

    2024

  45. [53]

    Udifftext: A unified frame- work for high-quality text synthesis in arbitrary images via character-aware diffusion models, 2023

    Yiming Zhao and Zhouhui Lian. Udifftext: A unified frame- work for high-quality text synthesis in arbitrary images via character-aware diffusion models, 2023. 2

    2023

  46. [54]

    A task is worth one word: Learning with task prompts for high-quality versatile image inpainting, 2023

    Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting, 2023. 1

    2023

  47. [55]

    Colorflow: Retrieval- augmented image sequence colorization, 2024

    Junhao Zhuang, Xuan Ju, Zhaoyang Zhang, Yong Liu, Shiyi Zhang, Chun Yuan, and Ying Shan. Colorflow: Retrieval- augmented image sequence colorization, 2024. 1 10

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.