Pith. sign in

REVIEW 3 major objections 5 minor 44 references

FontAdapter: Instant Font Adaptation in Visual Text Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FontAdapter claims that a single reference glyph image lets one text-to-image model render arbitrary text in any unseen font within seconds, without per-font fine-tuning.

desk verdict FontAdapter is a solid, practical method for instant font adaptation, but the paper overstates its case by never measuring against the per-font LoRA fine-tuning it claims to replace. read the letter →

arxiv 2506.05843 v1 pith:TBOE3KUW submitted 2025-06-06 cs.CV

Myungkyu Koo , Subin Kim , Sangkyung Kwak , Jaehyun Nam , Seojin Kim , Jinwoo Shin This is my paper
classification cs.CV
keywords visualtextgenerationfontcustomizationtext-to-imagediffusionreference-basedstyletransfertwo-stagecurriculumlearningsyntheticpaireddatasetsglyph-agnosticrepresentationzero-shotadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that font customization in visual text generation can be made instant: one model takes a single reference glyph image and renders any requested text in that font, including fonts never seen during training, in about 11 seconds on one GPU and without per-font fine-tuning. The authors identify why naive training fails—models copy reference glyphs or backgrounds instead of learning the font—and propose a two-stage curriculum that first extracts font attributes from isolated glyph pairs and then integrates the style into realistic scenes. Paired synthetic datasets, where the same font appears with different words, force the model to learn glyph-agnostic style. If correct, this turns font adaptation from a per-font optimization task into a general, real-time capability, while preserving text readability and prompt alignment.

What carries the argument

The central mechanism is a two-stage curriculum implemented as image-conditioning layers on top of a frozen text-to-image diffusion model. A Resampler module converts the reference glyph image into image prompt tokens, which enter the diffusion transformer through image-image attention layers running parallel to text attention. In the first stage, reference and target are black glyphs on white backgrounds with different words in the same font, so the only shared signal is the font style itself. In the second stage, targets become colorized scene-text images with realistic backgrounds while the Resampler is frozen, preserving the font representation learned earlier; a final domain-alignment pass refines synthetic scenes with SDEdit and expert-model samples. All training updates only the added image-conditioning layers, leaving base model weights frozen.

What would settle it

Hold out entire letters rather than whole words: train FontAdapter using only words composed of the letters A-M and then evaluate words composed only of N-Z in the same fonts. If the held-out letters score substantially lower on Max-IoU and HOG-similarity than the trained letters, font style is not glyph-agnostic and the core generalization claim fails.

Watch

Extended reading notes

Core claim

FontAdapter establishes that font style is a transferable, glyph-agnostic attribute that a single diffusion model can learn from paired images of the same font rendered with different words. Directly training on font-specific datasets degrades fine details, so the authors split learning into two stages: first, the model generates text-only images from a reference glyph, learning nuanced font attributes without background interference; second, with the Resampler frozen to protect the learned style, it generates scene-text images that embed the style into natural backgrounds. In the paper's evaluation on 9,000 samples across 300 held-out fonts, FontAdapter surpasses both caption-based generation and IPAdapter-Instruct on all font-similarity metrics (Max-IoU 0.4293 vs 0.3105/0.3118, HOG-sim 0.6138 vs 0.4451/0.4481), keeps word accuracy above its own base model (0.5303 vs 0.3963), and preserves prompt alignment. The authors present this as evidence that one glyph reference carries enough information to define a font for characters never seen next to that reference during training.

Load-bearing premise

Font style is a stable attribute that can be read off from a single glyph and transferred to characters that never appear next to it in training.

Editorial extensions

If this is right

  • Users can render any chosen font from a single glyph reference in seconds, with no per-font fine-tuning and no stored font-specific weights.
  • Text accuracy and prompt alignment are preserved under font conditioning, with word accuracy improving over the base SD3 model, so font adaptation does not come at the cost of readability or scene fidelity.
  • Font style transfers across languages from the same reference, because the learned style is glyph-agnostic rather than tied to particular characters.
  • Interpolating the Resampler's image prompt tokens yields smooth blends between two fonts, indicating a continuous, usable font-style space.
  • SDEdit-based visual text editing can change text, font, or color while keeping the original font style, covering tasks beyond generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If font style is truly glyph-agnostic, the same two-stage recipe should extend to numerals, punctuation, and non-Latin scripts once the backbone can render them; a clean test is to hold out whole character classes, not just words, during training.
  • The smooth font blending via token interpolation suggests the Resampler embeds fonts in an ordered, low-dimensional space; arithmetic on font tokens could enable style navigation beyond pairwise blends.
  • The paper's own limitation note says its metrics emphasize overall glyph structure over fine details, so a perceptual or vision-language-model evaluation would be needed to confirm that fine font features survive at scale.
  • Because reference glyphs can be obtained by off-the-shelf text segmentation, real-world photographs of signs could serve directly as font references, making font capture from the wild a natural extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces FontAdapter, a training-time method that lets a text-to-image diffusion model (built on SD3 and IPAdapter-Instruct) adapt an arbitrary font at inference time from a single reference glyph image, without per-font fine-tuning. The method uses a two-stage curriculum: first learning font-attribute extraction from paired text-only images, then learning to render the adapted font in scene-text images, followed by a domain-alignment stage that refines synthetic data with SDEdit and expert LoRA models. The authors report quantitative gains over Qwen-VL captioning and IPAdapter-Instruct on font similarity, text accuracy, and prompt alignment, and demonstrate applications including visual text editing, font blending, and cross-lingual font transfer. The central claim is that FontAdapter replaces costly per-font fine-tuning (about 40 minutes on 4 RTX 3090 GPUs for SD3 LoRA) with an 11-second adaptation on a single RTX 3090, while preserving high-quality font fidelity.

Significance. If the central claim holds, FontAdapter would be a practical contribution to visual text generation, addressing a real bottleneck in font customization: the cost of fine-tuning for each new font. The paper has several strengths: it ships a concrete training scheme with ablations that support the two-stage curriculum; it reports consistent quantitative improvements over two baselines across all reported metrics; it includes a user study; and it demonstrates a range of applications. The synthetic paired-dataset construction is a useful idea in itself. However, the significance is currently undercut by the absence of a direct comparison against the method the paper claims to replace (per-font fine-tuning), and by an internal inconsistency in the reported evaluation size. These issues are not presentation-level; they affect the strength of the central speed-quality tradeoff claim.

major comments (3)
  1. [Section 4.1, Table 1; Section 3.4; Appendix A] The central claim that FontAdapter replaces per-font fine-tuning is never tested quantitatively. The abstract and Section 1 position the 11-second adaptation as the key advantage over the 'about 40 minutes on 4 RTX 3090 GPUs' LoRA fine-tuning of SD3, but Table 1 compares FontAdapter only against Qwen-VL captioning and IPAdapter-Instruct. The paper already constructs per-font expert LoRA models during domain alignment (Section 3.4 and Appendix A); these could serve as a direct baseline. Without a comparison, the reader cannot calibrate the speed-quality tradeoff: a 40-minute LoRA might produce substantially higher font fidelity, in which case the practical contribution is a tradeoff that the paper does not measure. The user study (Table 4) also compares only against the two weak baselines. This omission is load-bearing and should be addressed by reporting per-font LoRA numbers on the same evaluation suite.
  2. [Section 4, first paragraph] The reported evaluation count is internally inconsistent. The paper states that 300 evaluation fonts are combined with 1,000 new words and that 300 prompts per complexity level (simple, moderate, complex) yield 900 prompts, then 'for each evaluation font, we randomly sample 10 prompts, generating 9,000 samples in total.' With 300 fonts and 10 prompts per font, the total is 3,000 samples, not 9,000. The discrepancy by a factor of three needs to be corrected, and the sampling procedure clarified (e.g., whether each font is paired with 10 distinct prompts from the 900, or whether 9,000 is intended with 30 prompts per font).
  3. [Section 3.4, Appendix A, and Section 4 (Evaluation Metrics)] The domain alignment filter uses the same metrics that are later reported as evaluation outcomes. Specifically, Appendix A states that refined samples are retained only if Max-IoU > 0.59 and HOG-sim > 0.80, and Section 4 uses Max-IoU and HOG-similarity as headline font-similarity metrics. Since the training data is selected to score highly under these metrics, the reported gains could be partly an artifact of optimizing the training set toward the evaluation measure, even though evaluation fonts are held out. The thresholds are also heuristic free parameters without sensitivity analysis. The paper should provide an ablation or analysis showing that the conclusions are robust to the choice of thresholds and to the use of these metrics in data filtering, or should use a held-out metric for the filter that is distinct from the evaluation metric.
minor comments (5)
  1. [Abstract and Section 4.1] The '11 seconds' claim is not accompanied by a measurement protocol; please specify the hardware, batch size, number of denoising steps, and whether the time includes preprocessing of the reference glyph or only the generation pass.
  2. [Table 4 and Appendix F] The user study description would benefit from clarifying how the 4,600 responses are distributed across the 50 sampled images per model and the two evaluation aspects, and whether each user saw all pairs or a subset.
  3. [Section 3.2] The text says empty regions are 'manually labeled with four corner coordinates' for 500 backgrounds; please clarify whether all boxes are hand-annotated and whether any automated verification was applied, as this affects reproducibility.
  4. [Appendix H] The limitation paragraph acknowledges that the proposed metrics capture overall glyph structure rather than fine details; this is a useful caveat, but it also tempers the quantitative claims in Table 1 and should be referenced earlier in the main text.
  5. [Section 4.2, Table 3] The row labels 'Same', 'Mixed', and 'Different' would be clearer if the text defined the 1:3 mixing ratio before the table, rather than only in the surrounding paragraph.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FontAdapter's training and evaluation are separated by held-out fonts and words, and the only metric overlap is a training-data quality filter that does not force the reported results.

full rationale

FontAdapter's claimed derivation chain is a two-stage curriculum trained on paired synthetic data and evaluated on 300 held-out online fonts plus a held-out 1,000-word dictionary. The font-attribute extraction objective does not define the target results in terms of the evaluation metrics. The domain-alignment filter in Section 3.4 and Appendix A uses the heuristic criterion 'Max-IoU > 0.59 ∧ HOG-sim. > 0.80' to select SDEdit-refined synthetic training samples, and Section 4 reports Max-IoU and HOG-similarity on generated images. While this shares metric names, it is not a reduction of the claim to its inputs: the filter selects training samples, the evaluation is performed on unseen fonts and words, and the reported values (e.g., Max-IoU 0.4293, HOG-sim 0.6138) are not entailed by the threshold. No load-bearing self-citation chain is present; reference [6] is a minor related-work citation by overlapping authors and is not used to justify the method. The absence of a per-font LoRA baseline is a completeness and calibration concern, not a circularity concern. Overall, the derivation is self-contained and the evaluation is not forced by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the transferability of font style from a single glyph, the representativeness of synthetic data, and the reliability of pretrained components. One ad-hoc element is the metric-based filter in domain alignment, which is a hand-chosen threshold tied to the evaluation metrics. No new physical or conceptual entities are introduced.

free parameters (5)
  • Domain alignment filter thresholds (Max-IoU > 0.59, HOG-sim > 0.80) = Max-IoU 0.59, HOG-sim 0.80
    Heuristic thresholds used to filter SDEdit-refined training samples; they use the same font accuracy metrics as the evaluation, creating a mild selection loop.
  • SDEdit strength values = 6 and 7.5
    Chosen to balance realism and glyph preservation in the domain alignment stage.
  • Training epochs = 50 (each stage) + 10 (domain alignment)
    Hyperparameters selected empirically to reach performance saturation.
  • Learning rate = 7.5e-8 (two-stage), 1e-8 (domain alignment)
    Standard tuning for fine-tuning diffusion adapters.
  • Batch size = 48
    Choice based on available GPU memory.
assumptions (5)
  • domain assumption Font style is a transferable property: a single glyph reference is sufficient to define a font style applicable to other characters.
    The whole method assumes glyph-agnostic font attributes can be extracted from one reference glyph; introduced in Section 3.1 and Section 3.2.
  • domain assumption Synthetic data generated from online fonts and SD3 backgrounds is representative enough for real-world generalization.
    The method relies on synthetic paired datasets and SDEdit/expert refinement to bridge the distribution gap; Sections 3.2 and 3.4.
  • domain assumption Pretrained IPAdapter-Instruct and SD3 provide a valid base for font extraction and rendering.
    The model is initialized from IPAdapter-Instruct pretrained on SD3; Section 3.3 implementation details.
  • standard math The conditional flow matching objective is a valid training loss for the diffusion model.
    Equation (1) uses the flow matching loss from prior work [19].
  • ad hoc to paper The heuristic filter thresholds preserve quality and do not systematically bias the training data.
    Section 3.4 uses Max-IoU>0.59 and HOG-sim.>0.80 to filter refined images; no rigorous justification is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FontAdapter: Instant Font Adaptation in Visual Text Generation." pith.science (2026). https://pith.science/paper/TBOE3KUW

@misc{pith2026250605843,
  author       = {Pith},
  title        = {Pith review of: FontAdapter: Instant Font Adaptation in Visual Text Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TBOE3KUW}},
  note         = {Machine review of arXiv:2506.05843}
}
read the original abstract

Text-to-image diffusion models have significantly improved the seamless integration of visual text into diverse image contexts. Recent approaches further improve control over font styles through fine-tuning with predefined font dictionaries. However, adapting unseen fonts outside the preset is computationally expensive, often requiring tens of minutes, making real-time customization impractical. In this paper, we present FontAdapter, a framework that enables visual text generation in unseen fonts within seconds, conditioned on a reference glyph image. To this end, we find that direct training on font datasets fails to capture nuanced font attributes, limiting generalization to new glyphs. To overcome this, we propose a two-stage curriculum learning approach: FontAdapter first learns to extract font attributes from isolated glyphs and then integrates these styles into diverse natural backgrounds. To support this two-stage training scheme, we construct synthetic datasets tailored to each stage, leveraging large-scale online fonts effectively. Experiments demonstrate that FontAdapter enables high-quality, robust font customization across unseen fonts without additional fine-tuning during inference. Furthermore, it supports visual text editing, font style blending, and cross-lingual font transfer, positioning FontAdapter as a versatile framework for font customization tasks.

Figures

Figures reproduced from arXiv: 2506.05843 by the authors.

Figure 1
Figure 1. Font customization with FontAdapter. In this paper, we present FontAdapter, a framework for font customization that instantly adapts a wide range of font styles across diverse image contexts. (a) shows that FontAdapter enables fast font customization, while successfully capturing both the nuances and intricate details of unique font styles. (b) and (c) highlight the versatility of FontAdapter, including controlled e… view at source ↗
Figure 2
Figure 2. Overview. (a) In the first stage, the model is trained to generate a text-only image, allowing the model to effectively understand font attributes. (b) In the second stage, the model is trained to generate a scene-text image, allowing the model to learn the incorporation of visual text and the background. The Resampler module is frozen in this stage to preserve the font attributes previously learned. (c) In the MM-D… view at source ↗
Figure 3
Figure 3. Examples of synthetic paired datasets. Text-only im￾ages are paired with each other in the first stage (top row) and are paired with scene-text images in the second stage (bottom row). Note that each pair share same font styles while differs in text. Text-only Images and Scene-text Images. To gener￾ate text-only images, we render black glyphs centered on white backgrounds. For scene-text images, where colorized glyp… view at source ↗
Figures from the paper (17 more)
Figure 5
Figure 5. Figure 5: Font customization results over training. Font attributes learned in the first stage remain consistent throughout training. ate high-quality samples using 50 expert models, each fine￾tuned on 3–5 scene-text images, specializing in a single font style (see Appendix A fo…
Figure 6
Figure 6. Figure 6: Qualitative comparison of font customization. Each sample is generated with a single font reference, without fine-tuning. FontAdapter demonstrates robust generalizability to diverse unique fonts while naturally integrating visual text into complex scenes. For example, …
Figure 7
Figure 7. Figure 7: Results from different training schemes. Fine details of reference fonts stay preserved when the Resampler is frozen. †Each sign denotes whether to freeze the Resampler on each stage. Resampler Freezing. We also examine the impact of freez￾ing the Resampler by unfreezi…
Figure 8
Figure 8. Figure 8: Font blending. Each sample between two font references is generated with interpolated image prompt tokens, where the ratio is indicated above each sample. Font features shift gradually according to the ratio, showing strong potential for designing new font styles. “eng…
Figure 9
Figure 9. Figure 9: Cross-lingual font reference. Although FontAdapter is not trained on other languages than English, it effectively captures font features from other language glyphs. Cross-lingual Font Reference. FontAdapter exhibits unique font customization capabilities, even in cross…
Figure 11
Figure 11. Figure 11: More examples from IPAdapter-Instruct. The pre-trained IPAdapter-Instruct fails to effectively disentangle font styles from realistic reference images, especially when the refer￾ence glyph is embedded in real scenes. F. User Study We conduct a user study ( [PITH_FULL…
Figure 12
Figure 12. Figure 12 [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Examples of scene-text images. Each background image is generated using SD3 and manually labeled with corner coordinates of a text region, marked with dashed lines. The glyphs are then warped onto the bounding box, while preserving their aspect ratio [PITH_FULL_IMAGE…
Figure 14
Figure 14. Figure 14: Examples of expert samples. Each expert model is fine-tuned using 4 visual text images of a specific font (shown in the first column). The rest of the samples in each row are generated by the corresponding expert model, which demonstrate consistent font styles seamles…
Figure 15
Figure 15. Figure 15: Visualization of evaluation pipeline. First, 1)prompt alignment is assessed by measuring image-text similarity between the generated image and its input prompt. Next, 2)text accuracy is evaluated using an off-the-shelf OCR model (PP-OCRv3 [17]). Lastly, for 3)font sim…
Figure 16
Figure 16. Figure 16: Font reference examples [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 17
Figure 17. Figure 17: Examples of measuring max-IoU and HOG-similarity. Ground-truth glyphs are highlighted in yellow, with the more similar visual text (#1) in green and the less similar one (#2) in red. (a) and (b) show examples of bold and thin font styles, respectively. In both cases, …
Figure 18
Figure 18. Figure 18: User study. We conduct a user study to evaluate font similarity and visual text quality. Users are presented with the global questions shown at the top of the figure and choose the better sample based on each criterion. To ensure consistent evaluation, each criterion …
Figure 19
Figure 19. Figure 19: More examples of visual text generation by FontAdapter. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: More examples of visual text editing by FontAdapter. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]
Figure 21
Figure 21. Figure 21: Examples of multi-color text control by FontAdapter. Text color can be easily controlled via textual prompting, without affecting the quality of font customization—demonstrating effective disentanglement between font styles and other glyph features. Reference Referenc…
Figure 22
Figure 22. Figure 22: More examples of font blending by FontAdapter. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_22.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 20 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35: 0 23716--23736, 2022

    2022

  3. [3]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 1 0 (2): 0 3, 2023

    arXiv 2023

  4. [4]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2 0 (3): 0 8, 2023

    2023

  5. [5]

    BlackForestLab . Flux.1. https://blackforestlabs.ai/announcing-black-forest-labs/, 2024. Accessed: 2024-10-29

    work page 2024

  6. [6]

    Anydoor: Zero-shot object-level image customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6593--6602, 2024

    work page 2024

  7. [7]

    Improving diffusion models for virtual try-on

    Yisol Choi, Sangkyung Kwak, Kyungmin Lee, Hyungwon Choi, and Jinwoo Shin. Improving diffusion models for virtual try-on. arXiv preprint arXiv:2403.05139, 2024

    arXiv 2024

  8. [8]

    Deepfloyd if

    DeepFloyd. Deepfloyd if. https://github.com/deep-floyd/IF, 2023

    work page 2023

Show all 44 references
  1. [9]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  2. [10]

    Neural transformation fields for arbitrary-styled font generation

    Bin Fu, Junjun He, Jianjun Wang, and Yu Qiao. Neural transformation fields for arbitrary-styled font generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22438--22447, 2023

    2023

  3. [11]

    Generate like experts: Multi-stage font generation by incorporating font transfer process into diffusion models

    Bin Fu, Fanghua Yu, Anran Liu, Zixuan Wang, Jie Wen, Junjun He, and Yu Qiao. Generate like experts: Multi-stage font generation by incorporating font transfer process into diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  4. [12]

    Lcm-lookahead for encoder-based text-to-image personalization

    Rinon Gal, Or Lichter, Elad Richardson, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Lcm-lookahead for encoder-based text-to-image personalization. arXiv preprint arXiv:2404.03620, 2024

    2024 arXiv

  5. [13]

    Diff-font: Diffusion model for robust one-shot font generation

    Haibin He, Xinyuan Chen, Chaoyue Wang, Juhua Liu, Bo Du, Dacheng Tao, and Qiao Yu. Diff-font: Diffusion model for robust one-shot font generation. International Journal of Computer Vision, pages 1--15, 2024 a

    2024

  6. [14]

    Disenvisioner: Disentangled and enriched visual prompt for customized image generation

    Jing He, LI Haodong, Guibao Shen, CAI Yingjie, Weichao Qiu, Ying-Cong Chen, et al. Disenvisioner: Disentangled and enriched visual prompt for customized image generation. In The Thirteenth International Conference on Learning Representations, 2024 b

    2024

  7. [15]

    Imagine yourself: Tuning-free personalized image generation

    Zecheng He, Bo Sun, Felix Juefei-Xu, Haoyu Ma, Ankit Ramchandani, Vincent Cheung, Siddharth Shah, Anmol Kalia, Harihar Subramanyam, Alireza Zareian, et al. Imagine yourself: Tuning-free personalized image generation. arXiv preprint arXiv:2409.13346, 2024 c

    2024 arXiv

  8. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  9. [17]

    Improving diffusion models for scene text editing with dual encoders

    Jiabao Ji, Guanhua Zhang, Zhaowen Wang, Bairu Hou, Zhifei Zhang, Brian Price, and Shiyu Chang. Improving diffusion models for scene text editing with dual encoders. arXiv preprint arXiv:2304.05568, 2023

    2023 arXiv

  10. [18]

    Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system

    Chenxia Li, Weiwei Liu, Ruoyu Guo, Xiaoting Yin, Kaitao Jiang, Yongkun Du, Yuning Du, Lingfeng Zhu, Baohua Lai, Xiaoguang Hu, et al. Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system. arXiv preprint arXiv:2206.03001, 2022

    2022 arXiv

  11. [19]

    Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing

    Dongxu Li, Junnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. Advances in Neural Information Processing Systems, 36: 0 30146--30166, 2023

    2023

  12. [20]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022

    2022 arXiv

  13. [21]

    Glyph-byt5-v2: a strong aesthetic baseline for accurate multilingual visual text rendering

    Zeyu Liu, Weicong Liang, Yiming Zhao, Bohan Chen, Lin Liang, Lijuan Wang, Ji Li, and Yuhui Yuan. Glyph-byt5-v2: a strong aesthetic baseline for accurate multilingual visual text rendering. arXiv preprint arXiv:2406.10208, 2024

    2024 arXiv

  14. [22]

    Glyph-byt5: A customized text encoder for accurate visual text rendering

    Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, and Yuhui Yuan. Glyph-byt5: A customized text encoder for accurate visual text rendering. In European Conference on Computer Vision, pages 361--377. Springer, 2025

    2025

  15. [23]

    Sdedit: Guided image synthesis and editing with stochastic differential equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021

    2021 arXiv

  16. [24]

    Dall-e 3: Text-to-image generation model

    OpenAI. Dall-e 3: Text-to-image generation model. https://openai.com/dall-e-3, 2023

    2023

  17. [25]

    Gpt-4o system card

    OpenAI. Gpt-4o system card. https://openai.com/index/gpt-4o-system-card/, 2024

    2024

  18. [26]

    Few shot font generation via transferring similarity guided global style and quantization local style

    Wei Pan, Anna Zhu, Xinyu Zhou, Brian Kenji Iwana, and Shilin Li. Few shot font generation via transferring similarity guided global style and quantization local style. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19506--19516, 2023

    2023

  19. [27]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  20. [28]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695, 2022

    2022

  21. [29]

    Ipadapter-instruct: Resolving ambiguity in image-based conditioning using instruct prompts

    Ciara Rowles, Shimon Vainer, Dante De Nigris, Slava Elizarov, Konstantin Kutsy, and Simon Donn \'e . Ipadapter-instruct: Resolving ambiguity in image-based conditioning using instruct prompts. arXiv preprint arXiv:2408.03209, 2024

    2024 arXiv

  22. [30]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  23. [31]

    Instantbooth: Personalized text-to-image generation without test-time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. Instantbooth: Personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8543--8552, 2024

    2024

  24. [32]

    Few-shot font generation by learning fine-grained local styles

    Licheng Tang, Yiyang Cai, Jiaming Liu, Zhibin Hong, Mingming Gong, Minhu Fan, Junyu Han, Jingtuo Liu, Errui Ding, and Jingdong Wang. Few-shot font generation by learning fine-grained local styles. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni...

    2022

  25. [33]

    Anytext: Multilingual visual text generation and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text generation and editing. arXiv preprint arXiv:2311.03054, 2023

    2023 arXiv

  26. [34]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

    2017

  27. [35]

    Cf-font: Content fusion for few-shot font generation

    Chi Wang, Min Zhou, Tiezheng Ge, Yuning Jiang, Hujun Bao, and Weiwei Xu. Cf-font: Content fusion for few-shot font generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1858--1867, 2023

    2023

  28. [36]

    Deep high-resolution representation learning for visual recognition

    Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al. Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence, 43 0 (10): ...

    2020

  29. [37]

    Ms-diffusion: Multi-subject zero-shot image personalization with layout guidance

    X Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. Ms-diffusion: Multi-subject zero-shot image personalization with layout guidance. arXiv preprint arXiv:2406.07209, 2024

    2024 arXiv

  30. [38]

    Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15943--15953, 2023

    2023

  31. [39]

    Rethinking text segmentation: A novel dataset and a text-specific refinement approach

    Xingqian Xu, Zhifei Zhang, Zhaowen Wang, Brian Price, Zhonghao Wang, and Humphrey Shi. Rethinking text segmentation: A novel dataset and a text-specific refinement approach. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12045--12055, 2021

    2021

  32. [40]

    Glyphcontrol: glyph conditional control for visual text generation

    Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, and Kai Chen. Glyphcontrol: glyph conditional control for visual text generation. Advances in Neural Information Processing Systems, 36: 0 44050--44066, 2023

    2023

  33. [41]

    Fontdiffuser: One-shot font generation via denoising diffusion with multi-scale content aggregation and style contrastive learning

    Zhenhua Yang, Dezhi Peng, Yuxin Kong, Yuyi Zhang, Cong Yao, and Lianwen Jin. Fontdiffuser: One-shot font generation via denoising diffusion with multi-scale content aggregation and style contrastive learning. In Proceedings of the AAAI conference on artificial intelligence, pa...

    2024

  34. [42]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  35. [43]

    Jedi: Joint-image diffusion models for finetuning-free personalized text-to-image generation

    Yu Zeng, Vishal M Patel, Haochen Wang, Xun Huang, Ting-Chun Wang, Ming-Yu Liu, and Yogesh Balaji. Jedi: Joint-image diffusion models for finetuning-free personalized text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2024

  36. [44]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975--11986, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.