Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

FIX-CLIP: Dual-Branch Hierarchical Contrastive Learning via Synthetic Captions for Better Understanding of Long Text

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FIX-CLIP claims that a dual-branch contrastive training scheme—masked images matched with short captions, raw images matched with long synthetic captions—extends CLIP-style text encoding to 248 tokens and improves both long-text and…

desk verdict Solid long-text CLIP training recipe with a genuine internal-validity flaw in the headline numbers: the ShareGPT4V-1k column is in-distribution, so the 13%/5% claims need re-computing. read the letter →

arxiv 2507.10095 v2 pith:USXFY7WS submitted 2025-07-14 cs.CV

Bingchao Wang , Zhiwei Ning , Jianyu Ding , Xuanang Gao , Yin Li , Dongsheng Jiang , Jie Yang , Wei Liu This is my paper
classification cs.CV
keywords CLIPlong-textimage-textretrievalsyntheticcaptionsdual-branchcontrastivelearningregionalpromptshierarchicalfeaturealignmenttext-to-imagegenerationzero-shotclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FIX-CLIP sets out to remove CLIP's 77-token ceiling on text input without paying the usual price of forgetting short-text skills. The paper's central claim is that a dual-branch objective—matching masked images to short captions and raw images to long captions—plus regional prompts and hierarchical alignment of intermediate features lets the model understand captions up to 248 tokens. Trained on up to 30M synthetic long captions generated by a multimodal LLM from 20 prompts, FIX-CLIP reports higher Recall@1 than prior long-text CLIP methods on human-authored long-caption benchmarks such as DCI and IIW, while also improving on COCO and Flickr30k short-text retrieval. Because the resulting text encoder can be dropped into Stable Diffusion XL in place of the original CLIP text encoder, the claim matters beyond retrieval: long-prompt text-to-image generation could inherit the same gains.

What carries the argument

Three mechanisms carry the argument. (1) Dual-branch training: one branch contrasts masked images (75% of patches replaced with learnable zero-initialized embeddings) against short captions, the other contrasts raw images against long captions, so the two text-length regimes are learned from matched visual inputs. (2) Regional prompts with unidirectional mask: in each image-encoder block, M learnable prompt tokens attend only to their own contiguous block of patch embeddings, while patches and the [CLS] token keep full attention, giving local features without global degradation. (3) Hierarchical feature alignment: encoder layers are split into groups; each group's tokens are aggregated with Gaussian weights into a Group Middle Feature, projected and normalized, and aligned across modalities with an InfoNCE loss on the deeper groups; these losses are added to the short and long contrastive losses. The synthetic caption pipeline—20 diverse prompts applied to Llama3-LLaVA-NE XT-8b over 30M images, with filtering of repeated or meaningless text—supplies the training signal that these mechanisms act on.

What would settle it

Retrain FIX-CLIP on the same 30M synthetic captions but with the main object nouns adversarially swapped for plausible alternatives (e.g., 'yellow bus' to 'yellow coach', 'fountain' to 'water feature') while keeping images fixed; if Recall@1 on DCI and IIW stays near the reported values, the model is matching caption style rather than image content, and the claimed long-text understanding is not what the numbers say.

Watch

Extended reading notes

Core claim

On the paper's own terms, FIX-CLIP's discovery is that long-text and short-text image-text alignment need not compete. The dual-branch pipeline preserves the original 77 position embeddings for short text, keeps the pre-trained parameters, and expands positions 21–77 by interpolation to reach 248 positions for long text; masked images are paired with short captions and raw images with long captions, so the model maintains the short-text feature space while learning a longer one. Learnable regional prompts, each attending only to a contiguous band of image patches through a unidirectional mask, supply local visual detail without corrupting the patch embeddings. A hierarchical feature alignment loss matches group-aggregated intermediate features from the image and text encoders, with the deepest groups weighted most heavily. With 30M synthetic long captions, the paper reports a 13% average gain over the previous best method on long-text benchmarks and a 5% gain on short-text benchmarks, and shows the text encoder improving long-prompt text-to-image generation when inserted into a diffusion model.

Load-bearing premise

The load-bearing premise is that the 30M synthetic long captions produced by Llama3-LLaVA-NeXT-8b from 20 prompts are accurate and diverse enough to teach genuine image-to-long-text alignment; if those captions contain systematic hallucination or a uniform style that survives the simple repeat filter, the reported gains on human-authored benchmarks could be inflated and may not transfer to other domains.

Editorial extensions

If this is right

  • CLIP-style dual encoders can serve captions of roughly 248 tokens while keeping or improving zero-shot short-text retrieval, removing a hard input-length ceiling for downstream models.
  • The same text encoder can replace the standard CLIP text encoder in a diffusion model, improving adherence to long prompts (positions, materials, colors, counts) without retraining the diffusion backbone.
  • Scaling MLLM-synthesized long captions from 5M to 30M steadily raises long-text retrieval, suggesting that synthetic captions, not just human captions, are a usable scaling axis.
  • Incremental training on synthetic long captions can beat methods trained from scratch on comparable or larger data, lowering the compute needed to add long-text ability.
  • Zero-shot robustness on adversarial classification sets such as ImageNet-A and ImageNet-O is maintained or improved, so long-text training need not cost distribution robustness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gains come largely from the uniform narrative style of the 20 synthesis prompts, transferring to domains with very different caption styles, such as medical imaging or technical diagrams, may require re-synthesizing or adapting the prompts; a domain-transfer evaluation would expose this.
  • The unidirectional-mask regional prompts could be a general recipe for injecting coarse region-level grounding into any ViT-based image encoder without dense annotations, so the mechanism may outlive the long-text setting.
  • The hierarchical Group Middle Feature alignment effectively uses shallow-to-deep features as a curriculum; one testable extension is weighting the group losses by caption length or by estimated regional informativeness rather than by fixed depth.
  • A controlled study holding compute fixed and varying only the number of synthesis prompts would separate prompt-diversity effects from data-scale effects in the reported scaling curve.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FIX-CLIP, an incremental fine-tuning method that extends CLIP to long text (up to 248 tokens) while preserving short-text retrieval. Three components are introduced: a dual-branch contrastive objective pairing masked images with short captions and raw images with long captions; learnable regional prompts with unidirectional attention masks in the image encoder; and hierarchical alignment of intermediate layer features. The authors also synthesize long captions for roughly 30M images using Llama3-LLaVA-NeXT-8b and train at 5M, 15M, and 30M scales. Experiments report state-of-the-art long-text retrieval on DCI, IIW, ShareGPT4V-1k, and Urban-1k, strong short-text results on COCO and Flickr30k, and qualitative text-to-image generation with SDXL.

Significance. If the long-text gains on held-out human-authored benchmarks are confirmed after removing the in-distribution ShareGPT4V-1k evaluation, FIX-CLIP would be a practically valuable incremental recipe: it improves long-text retrieval by large margins over Long-CLIP on DCI and IIW while keeping short-text COCO/Flickr scores at or above Long-CLIP. The three modules are ablated separately, the code is released, and the synthetic-data scaling study from 5M to 30M is a useful reference. The main caveats are benchmark contamination, selection of hyperparameters on the reported benchmarks, and some internal inconsistencies in the ablation reporting.

major comments (4)
  1. [Sec. 4.1, Table 1, Appendix Table 9] ShareGPT4V-1k is presented as a zero-shot long-text benchmark, but it is a 1k subsample of the ShareGPT4V training set, and Table 8 shows that ShareGPT4V is part of the training mixture for the 1M, 5M, 15M, and 30M models. The near-perfect scores in this column therefore measure training-set retrieval. Because the 'Avg' column in Table 1 includes this benchmark, the headline margins in Sec. 4.3 are inflated. Please recompute all long-text averages and the 13%/5% claim using only DCI, IIW, and Urban-1k, or explicitly label ShareGPT4V-1k as a memorization check.
  2. [Sec. 4.4, Tables 6-7] The hyperparameters M=4, hierarchical groups [4,6], and GMF weights 0.2/0.4/0.8 are selected by maximizing DCI and COCO scores in the ablations, and the same DCI and COCO results are then reported in Tables 1-2 as the main results. This creates a selection-on-the-test-set risk: the reported margins may be partly due to tuning on the evaluation benchmarks. Please hold out a validation split for model selection and report final numbers on an untouched test portion, or at minimum disclose the selection procedure and its potential effect.
  3. [Sec. 3.3, Eq. (3)] The mask indexing is inconsistent: the sequence is defined as [CLS, R_1, ..., R_M, P_1, ..., P_N] with j in [1, M], yet the formula gives b_j = 1 + M + j*floor(N/M). For j=1 the first regional prompt attends to patch floor(N/M)+1 rather than the first patch, and for j=M the interval extends beyond the sequence length. Please restate the formula with 0-based prompt indices (j=0, ..., M-1) or the correct 1-based offset, and align it with the released code.
  4. [Sec. 4.4, Table 4] The narrative does not match the numbers: row 1 shows DCI T2I dropping from 67.4 to 61.4 when the dual-branch pipeline is added, yet the text says the dual-branch pipeline gives a '4% boost in R1' on DCI; row 2 (HF alone) drops DCI T2I to 58.5, yet the text says HF gives 'decent gain for all benchmarks'; and row 4 (DB+RP) drops DCI T2I and COCO I2T relative to row 1, yet the text says RP 'improves the performance in each task.' Please correct the description or the table so that the component-wise contributions are reported accurately.
minor comments (6)
  1. [Abstract and Fig. 2 caption] The abstract contains 'under-stream tasks,' which appears to be a typo, and the Figure 2 caption has missing spaces in 'Thisimagecaptures' and 'T-Mobilestore'; please proofread these passages.
  2. [Sec. 4.3] The sentence 'FIX-CLIP outperforms state-of-the-art approaches by 13% and 5%' does not specify the baseline model or data scale; please make the comparison explicit and recompute after removing ShareGPT4V-1k.
  3. [Tables 3-6] Several table headers and captions contain missing spaces, such as 'ofFIX-CLIP' and 'capabilities ofFIX-CLIP'; please fix these typographical issues.
  4. [Sec. 4.5, Figs. 5 and 10] The text-to-image generation evaluation is purely qualitative; please add a quantitative metric such as image-text alignment or user study, or explicitly scope the claim as a qualitative demonstration.
  5. [Tables 1-3] No variance or number of runs is reported; for margins of 0.1-1.0 points (e.g., COCO R@1 between FIX-CLIP 15M and 30M in Table 2), reporting standard deviations over at least three seeds would increase confidence.
  6. [Sec. 3.2] The phrase 'only the positional embedding (PE) in Eq.(1) is learnable' is unclear because Eq. (1) is a deterministic interpolation formula; please clarify that the expanded position embedding parameters are updated during training.

Circularity Check

2 steps flagged · score 4.0 of 10

Zero-shot long-text average includes ShareGPT4V-1k, which overlaps the ShareGPT4V training set, and DCI/COCO are used for model selection; the core method still has independent external support.

  1. self definitional [Sec. 4.1 (Experimental Setup), Table 1, and Appendix Tables 8-9]
    "we evaluate method on datasets with long captions, including ShareGPT4V-1k [7], Urban-1k [63], DCI [52], and IIW [15] and report the Recall at 1 (R@1) metric. ... For results without specifically indicating data scales, the training dataset is ShareGPT4V [7], which contains 1M long-text-image pairs."

    FIX-CLIP 1M is trained on ShareGPT4V (1M pairs), and ShareGPT4V-1k is a 1k sample of that same dataset, as the shared citation [7] and the appendix's benchmark statistics indicate. The column labeled 'ShareGPT4V-1k' in Table 1 is therefore a retrieval test on the training distribution: the model has seen the exact image-text pairs during contrastive training. The near-saturated scores (95.5/94.1 for the 1M model; 98.6/98.5 for the 30M model) reflect training-set retrieval, not zero-shot generalization. Since Table 1's 'Avg' and Sec. 4.3's 'outperforms state-of-the-art approaches by 13%' include this column, the headline long-text margin is partly an artifact of training/evaluation overlap.

  2. fitted input called prediction [Sec. 4.4 (Ablation studies), Tables 6-7; Sec. 4.3 (Scalability Analysis)]
    "Finally, we set the weights for GMF loss as 0.2, 0.4, and 0.8 for 4-th, 5-th, and 6-th groups, respectively. ... When the number of prompts is set to 4, our approach achieves the best performance on average."

    The final configuration is selected by inspecting the DCI and COCO R@1 columns in Tables 6-7: the hierarchical range [4,6] is chosen because it yields the best DCI numbers, and the number of regional prompts M=4 is chosen because it 'achieves the best performance on average.' The same DCI and COCO numbers are then reported as headline results in Tables 1-2 and summarized in Sec. 4.3. Thus the reported gains on DCI and COCO are the selected best-of-grid values on those benchmarks, not independent predictions. This is benchmark-driven model selection: the 'prediction' on DCI and COCO is statistically forced to be the maximum over the ablated configurations.

full rationale

FIX-CLIP is an empirical systems paper, not a derivation from first principles, and the proposed architecture itself is not circular: the dual-branch pipeline, regional prompts, and hierarchical alignment are all trained and then measured on external benchmarks. No load-bearing self-citation chain or imported uniqueness theorem is present. The circularity burden is concentrated in the evaluation protocol. First, Table 1 presents ShareGPT4V-1k as a zero-shot long-text benchmark, but ShareGPT4V is also the default training set, so that column is a training-set retrieval check rather than generalization; the long-text 'Avg' and the 13% headline margin include it. Second, the hyperparameters and module choices (number of regional prompts, hierarchical group range, group weights) are selected from ablations measured on DCI and COCO, and the same DCI and COCO results are then reported as headline SOTA achievements, which is a fitted-input-called-prediction pattern. Still, the method retains substantial independent support: the held-out IIW and Urban-1k long-text columns and the short-text Flickr/COCO results (modulo COCO selection) are external to the training set and do not collapse when the contaminated column is removed. Accordingly, this is partial circularity in the reported evaluation, not a circular derivation of the central method. Score 4.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central result rests on the quality of 30M synthetic captions, the validity of masked-image training for short text, and the choice of several free hyperparameters (M, G, group weights, mask ratio) that are set by hand or by ablations on the evaluation benchmarks. No new physical entities are introduced.

free parameters (5)
  • Mask ratio alpha = 0.75
    Hand-set based on MAE; no ablation reported in the paper.
  • Number of regional prompts M = 4
    Selected via ablation on DCI and COCO in Table 6; this is tuning on the evaluation benchmarks.
  • Hierarchical group count G = 6
    Chosen without ablation; all Transformer blocks divided into 6 groups.
  • Hierarchical alignment start group K and weights = K=4, weights 0.2, 0.4, 0.8
    Tuned on DCI benchmark in Table 7.
  • Position embedding expansion factor = 4 (length 248)
    Chosen to meet long-text length requirements; no ablation on this factor, only long vs short PE.
assumptions (4)
  • domain assumption Synthetic captions generated by Llama3-LLaVA-NeXT-8b are sufficiently accurate and diverse to train a general long-text image-text encoder.
    Invoked in Sec 3.1; the method's success depends on this, and the paper only applies a simple filter for repeats, not a factuality check.
  • domain assumption Randomly masking 75% of image patches preserves enough semantic information for contrastive alignment with short captions.
    Invoked in Sec 3.2, citing MAE; the paper does not ablate the mask ratio.
  • domain assumption Position embedding interpolation from 77 to 248 tokens preserves short-text performance while enabling long-text encoding.
    Invoked in Sec 3.2 following Long-CLIP; only frozen first 20 PE are kept, the rest is interpolated and learned.
  • ad hoc to paper The ShareGPT4V-1k evaluation subset is disjoint from the ShareGPT4V training set.
    Assumed in Sec 4.1/4.2; the paper does not state a split, and since training uses all of ShareGPT4V, this assumption is questionable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FIX-CLIP: Dual-Branch Hierarchical Contrastive Learning via Synthetic Captions for Better Understanding of Long Text." pith.science (2026). https://pith.science/paper/USXFY7WS

@misc{pith2026250710095,
  author       = {Pith},
  title        = {Pith review of: FIX-CLIP: Dual-Branch Hierarchical Contrastive Learning via Synthetic Captions for Better Understanding of Long Text},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/USXFY7WS}},
  note         = {Machine review of arXiv:2507.10095}
}
abstract

CLIP has shown promising performance across many short-text tasks in a zero-shot manner. However, limited by the input length of the text encoder, CLIP struggles on under-stream tasks with long-text inputs ($>77$ tokens). To remedy this issue, we propose FIX-CLIP, which includes three novel modules: (1) A dual-branch training pipeline that aligns short and long texts with masked and raw images, respectively, which boosts the long-text representation while preserving the short-text ability. (2) Multiple learnable regional prompts with unidirectional masks in Transformer layers for regional information extraction. (3) A hierarchical feature alignment module in the intermediate encoder layers to promote the consistency of multi-scale features. Furthermore, we collect 30M images and utilize existing MLLMs to synthesize long-text captions for training. Extensive experiments show that FIX-CLIP achieves state-of-the-art performance on both long-text and short-text retrieval benchmarks. For downstream applications, we reveal that FIX-CLIP's text encoder delivers promising performance in a plug-and-play manner for diffusion models with long-text input. The code is available at https://github.com/bcwang-sjtu/Fix-CLIP.

Figures

Figures reproduced from arXiv: 2507.10095 by the authors.

Figure 1
Figure 1. We compare FIX-CLIP with CLIP [44], LoTLIP [55], and Long-CLIP [63] on B/16 model. FIX-CLIP achieves competi￾tive performance across long-text and short-text retrieval tasks. els. Recently, PixArt-α [4] uses Flan-T5 as the text encoder to increase the length of input tokens from 77 to 120 and injects the obtained text features into DiT [41] to alleviate the deficiency in long-text understanding. Following Long￾CLIP … view at source ↗
Figure 2
Figure 2. Comparison of FIX-CLIP against Long-CLIP [63] in image-to-text and text-to-image retrieval tasks with long-text captions. The key texts related to the correct elements are marked in green, and the red texts indicate the wrong elements. tail features is crucial, recent works [1, 48, 53] make an effort to address the issue by dividing the input images into several regions and matching each region with the corre￾spondi… view at source ↗
Figure 3
Figure 3. Overview of FIX-CLIP. The image w/o mask aligns with a long caption, while the masked image aligns with a short caption. In the image encoder, regional prompts are employed with the unidirectional mask to extract the regional information. The hierarchical alignment module is designed to associate the middle aggregation features between the image encoder and the text encoder. is the dimension of the channel. Subseque… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Unidirectional mask map is proposed to achieve the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Comparison on the text-to-image generation performance. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Some incorrect examples from our re-captioned dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 8
Figure 8. Figure 8: Visualization of the Effects of Unidirectional Masking [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Similarity Heatmap between text and image features in different models. (a) presents a comparative analysis between our model [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: More Text-to-Image Generation examples. Images generated by [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Eigen Neural Network: Unlocking Generalizable Vision with Eigenbasis

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    Reparameterizing layer weights in a learned orthonormal eigenbasis is claimed to improve ImageNet classification, cross-modal retrieval, and enable a faster backpropagation-free variant that surpasses standard backpro...

Reference graph

Works this paper leans on

94 extracted references · 52 canonical work pages · cited by 1 Pith paper

  1. [1]

    ComAlign: Compositional Alignment in Vision-Language Models

    Ali Abdollah, Amirmohammad Izadi, Armin Saghafian, Reza Vahidimajd, Mohammad Mozafari, Amirreza Mirzaei, Mo- hammadmahdi Samiei, and Mahdieh Soleymani Baghshah. Comalign: Compositional alignment in vision-language mod- els. arXiv preprint arXiv:2409.08206, 2024. 2

    work page Pith review arXiv 2024

  2. [2]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Dosovitskiy Alexey. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv: 2010.11929, 2020. 6

    arXiv 2010

  3. [3]

    Internlm2 technical report

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internlm2 technical report. arXiv preprint arXiv:2403.17297, 2024. 1

    arXiv 2024

  4. [4]

    Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023. 1

    arXiv

  5. [5]

    Prism- layers: Open data for high-quality multi-layer transparent image generative models

    Junwen Chen, Heyang Jiang, Yanbin Wang, Keming Wu, Ji Li, Chao Zhang, Keiji Yanai, Dong Chen, and Yuhui Yuan. Prism- layers: Open data for high-quality multi-layer transparent image generative models. arXiv preprint arXiv:2505.22523,

    arXiv

  6. [6]

    Shikra: Unleashing multimodal llm’s referential dialogue magic

    Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195,

    arXiv

  7. [7]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12793, 2023. 3, 5, 6, 1, 2

    arXiv 2023

  8. [8]

    Microsoft coco captions: Data collection and evaluation server

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedan- tam, Saurabh Gupta, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325 , 2015. 5, 7, 2, 3

    arXiv 2015

Show all 94 references
  1. [9]

    Reproducible scal- ing laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuh- mann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scal- ing laws for contrastive language-image learning. In Proceed- ings of the IEEE/CVF Conference on Computer ...

    2023

  2. [10]

    Instructblip: Towards general-purpose vision- language models with instruction tuning, 2023

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision- language models with instruction tuning, 2023. 3

    2023

  3. [11]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 5, 7

    2009

  4. [12]

    Maskclip: Masked self-distillation advances contrastive language-image pretraining

    Xiaoyi Dong, Jianmin Bao, Yinglin Zheng, Ting Zhang, Dongdong Chen, Hao Yang, Ming Zeng, Weiming Zhang, Lu Yuan, Dong Chen, et al. Maskclip: Masked self-distillation advances contrastive language-image pretraining. In Proceed- ings of the IEEE/CVF Conference on Computer Vision...

    2023

  5. [13]

    Go to zero: Towards zero-shot motion generation with million-scale data

    Ke Fan, Shunlin Lu, Minyue Dai, Runyi Yu, Lixing Xiao, Zhiyang Dou, Junting Dong, Lizhuang Ma, and Jingbo Wang. Go to zero: Towards zero-shot motion generation with million-scale data. arXiv preprint arXiv:2507.07095 ,

    arXiv

  6. [14]

    Improving clip training with language rewrites

    Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, and Yon- glong Tian. Improving clip training with language rewrites. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  7. [15]

    Im- ageinwords: Unlocking hyper-detailed image descriptions

    Roopal Garg, Andrea Burns, Burcu Karagol Ayan, Yonatan Bitton, Ceslee Montgomery, Yasumasa Onoe, Andrew Bun- ner, Ranjay Krishna, Jason Baldridge, and Radu Soricut. Im- ageinwords: Unlocking hyper-detailed image descriptions. arXiv preprint arXiv:2405.02793, 2024. 5, 2, 3, 4

    2024 arXiv

  8. [16]

    Open- vocabulary object detection via vision and language knowl- edge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open- vocabulary object detection via vision and language knowl- edge distillation. arXiv preprint arXiv:2104.13921 , 2021. 2

    2021 arXiv

  9. [17]

    Synthclip: Are we ready for a fully synthetic clip training? arXiv preprint arXiv:2402.01832, 2024

    Hasan Abed Al Kader Hammoud, Hani Itani, Fabio Pizzati, Philip Torr, Adel Bibi, and Bernard Ghanem. Synthclip: Are we ready for a fully synthetic clip training? arXiv preprint arXiv:2402.01832, 2024. 2

    2024 arXiv

  10. [18]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020. 5

    2020

  11. [19]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 3

    2022

  12. [20]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15262–15271, 2021. 5, 6, 7

    2021

  13. [21]

    Dynamicid: Zero-shot multi-id image personalization with flexible facial editability

    Xirui Hu, Jiahao Wang, Hao Chen, Weizhan Zhang, Benqi Wang, Yikun Li, and Haishun Nan. Dynamicid: Zero-shot multi-id image personalization with flexible facial editability

  14. [22]

    Vcoder: Ver- satile vision encoders for multimodal large language models

    Jitesh Jain, Jianwei Yang, and Humphrey Shi. Vcoder: Ver- satile vision encoders for multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27992–28002, 2024. 2

    2024

  15. [23]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International confer- ence on machine learning , pages 4904–49...

    2021

  16. [24]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of compute...

    2017

  17. [25]

    Krizhevsky and G

    A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images.Handbook of Systemic Autoimmune Diseases, 1(4), 2009. 5, 7

    2009

  18. [26]

    Veclip: Improving clip training via visual-enriched captions

    Zhengfeng Lai, Haotian Zhang, Bowen Zhang, Wentao Wu, Haoping Bai, Aleksei Timofeev, Xianzhi Du, Zhe Gan, Jiu- long Shan, Chen-Nee Chuah, et al. Veclip: Improving clip training via visual-enriched captions. In European Confer- ence on Computer Vision , pages 111–127. Springer, 2025. 2

    2025

  19. [27]

    Proxyclip: Proxy atten- tion improves clip for open-vocabulary segmentation

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Proxyclip: Proxy atten- tion improves clip for open-vocabulary segmentation. arXiv preprint arXiv:2408.04883, 2024. 2

    2024 arXiv

  20. [28]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. arXiv preprint arXiv:2201.03546, 2022. 2

    2022 arXiv

  21. [29]

    Blip: Bootstrapping language-image pre-training for unified vision- language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision- language understanding and generation. In International conference on machine learning, pages 12888–12900. PMLR,

  22. [30]

    Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR,

  23. [31]

    Grounded language- image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language- image pre-training. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, p...

    2022

  24. [32]

    Scaling language-image pre-training via masking

    Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichten- hofer, and Kaiming He. Scaling language-image pre-training via masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23390– 23400, 2023. 2, 7

    2023

  25. [33]

    Cascade-clip: Cascaded vision-language embeddings alignment for zero-shot semantic segmentation

    Yunheng Li, ZhongYu Li, Quansheng Zeng, Qibin Hou, and Ming-Ming Cheng. Cascade-clip: Cascaded vision-language embeddings alignment for zero-shot semantic segmentation. arXiv preprint arXiv:2406.00670, 2024. 2

    2024 arXiv

  26. [34]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 3

    2024

  27. [35]

    Llava-next: Improved reason- ing, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reason- ing, ocr, and world knowledge, 2024. 3, 1, 2

    2024

  28. [36]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 3

    2017 arXiv

  29. [37]

    Open vocabulary semantic segmentation with patch aligned con- trastive learning

    Jishnu Mukhoti, Tsung-Yu Lin, Omid Poursaeed, Rui Wang, Ashish Shah, Philip HS Torr, and Ser-Nam Lim. Open vocabulary semantic segmentation with patch aligned con- trastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19...

    2023

  30. [38]

    Tulip: Token-length upgraded clip

    Ivona Najdenkoska, Mohammad Mahdi Derakhshani, Yuki M Asano, Nanne van Noord, Marcel Worring, and Cees GM Snoek. Tulip: Token-length upgraded clip. arXiv preprint arXiv:2410.10034, 2024. 3, 6, 7

    2024 arXiv

  31. [39]

    Wonderturbo: Generating interac- tive 3d world in 0.72 seconds, 2025

    Chaojun Ni, Xiaofeng Wang, Zheng Zhu, Weijie Wang, Haoyun Li, Guosheng Zhao, Jie Li, Wenkang Qin, Guan Huang, and Wenjun Mei. Wonderturbo: Generating interac- tive 3d world in 0.72 seconds, 2025. 1

    2025

  32. [40]

    Im2text: Describing images using 1 million captioned photographs

    Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2text: Describing images using 1 million captioned photographs. Advances in neural information processing systems , 24, 2011. 3, 1, 2

    2011

  33. [41]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  34. [42]

    Bizgen: Advancing article-level visual text rendering for info- graphics generation

    Yuyang Peng, Shishi Xiao, Keming Wu, Qisheng Liao, Bo- han Chen, Kevin Lin, Danqing Huang, Ji Li, and Yuhui Yuan. Bizgen: Advancing article-level visual text rendering for info- graphics generation. In Proceedings of the Computer Vision and Pattern Recognition Conference , pag...

  35. [43]

    Flickr30k entities: Collecting region-to-phrase correspon- dences for richer image-to-sentence models

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspon- dences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer visi...

    2015

  36. [44]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning ,...

    2021

  37. [45]

    Do imagenet classifiers generalize to im- agenet? In International conference on machine learning , pages 5389–5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to im- agenet? In International conference on machine learning , pages 5389–5400. PMLR, 2019. 5, 7

    2019

  38. [46]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjorn Ommer. High-resolution image syn- thesis with latent diffusion models. In 2022 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR),

    2022

  39. [47]

    Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning. In Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pape...

    2018

  40. [48]

    Umg-clip: A unified multi-granularity vision generalist for open-world understanding

    Bowen Shi, Peisen Zhao, Zichen Wang, Yuhang Zhang, Yaom- ing Wang, Jin Li, Wenrui Dai, Junni Zou, Hongkai Xiong, Qi Tian, et al. Umg-clip: A unified multi-granularity vision generalist for open-world understanding. In European Con- ference on Computer Vision, pages 259–277. Sp...

    2025

  41. [49]

    Localizing objects with self-supervised trans- formers and no labels

    Oriane Sim´eoni, Gilles Puy, Huy V V o, Simon Roburin, Spy- ros Gidaris, Andrei Bursuc, Patrick P ´erez, Renaud Marlet, and Jean Ponce. Localizing objects with self-supervised trans- formers and no labels. arXiv preprint arXiv:2109.14279 ,

    arXiv

  42. [50]

    Eva-clip: Improved training techniques for clip at scale

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint arXiv:2303.15389, 2023. 2, 7

    2023 arXiv

  43. [51]

    Yfcc100m: The new data in multimedia research

    Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li- Jia Li. Yfcc100m: The new data in multimedia research. Communications of the ACM, 59(2):64–73, 2016. 3, 1, 2

    2016

  44. [52]

    A picture is worth more than 77 text tokens: Evaluating clip-style models on dense captions

    Jack Urbanek, Florian Bordes, Pietro Astolfi, Mary Williamson, Vasu Sharma, and Adriana Romero-Soriano. A picture is worth more than 77 text tokens: Evaluating clip-style models on dense captions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  45. [53]

    Position-guided text prompt for vision-language pre- training

    Jinpeng Wang, Pan Zhou, Mike Zheng Shou, and Shuicheng Yan. Position-guided text prompt for vision-language pre- training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23242– 23251, 2023. 2

    2023

  46. [54]

    Image as a foreign lan- guage: Beit pretraining for vision and vision-language tasks

    Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, et al. Image as a foreign lan- guage: Beit pretraining for vision and vision-language tasks. In Proceedings of the IEEE/CVF Conference ...

    2023

  47. [55]

    Lotlip: Improving language-image pre-training for long text understanding

    Wei Wu, Kecheng Zheng, Shuailei Ma, Fan Lu, Yuxin Guo, Yifei Zhang, Wei Chen, Qingpei Guo, Yujun Shen, and Zheng- Jun Zha. Lotlip: Improving language-image pre-training for long text understanding. arXiv preprint arXiv:2410.05249,

    arXiv

  48. [56]

    Flair: Vlm with fine- grained language-informed image representations

    Rui Xiao, Sanghwan Kim, Mariana-Iuliana Georgescu, Zeynep Akata, and Stephan Alaniz. Flair: Vlm with fine- grained language-informed image representations. arXiv preprint arXiv:2412.03561, 2024. 3, 6

    2024 arXiv

  49. [57]

    Demystifying clip data

    Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Rus- sell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demystifying clip data. arXiv preprint arXiv:2309.16671, 2023. 2

    2023 arXiv

  50. [58]

    Filip: Fine-grained interactive language-image pre-training

    Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. Filip: Fine-grained interactive language-image pre-training. arXiv preprint arXiv:2111.07783, 2021. 2

    2021 arXiv

  51. [59]

    Fb-diff: Fourier basis-guided diffusion for temporal interpolation of 4d medical imaging,

    Xin You, Runze Yang, Chuyan Zhang, Zhongliang Jiang, Jie Yang, and Nassir Navab. Fb-diff: Fourier basis-guided diffusion for temporal interpolation of 4d medical imaging,

  52. [60]

    Temporal differential fields for 4d motion modeling via image-to-video synthesis, 2025

    Xin You, Minghui Zhang, Hanxiao Zhang, Jie Yang, and Nassir Navab. Temporal differential fields for 4d motion modeling via image-to-video synthesis, 2025. 2

    2025

  53. [61]

    Capsfu- sion: Rethinking image-text data at scale

    Qiying Yu, Quan Sun, Xiaosong Zhang, Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, and Jingjing Liu. Capsfu- sion: Rethinking image-text data at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14022–14032, 2024. 2

    2024

  54. [62]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023. 6

    2023

  55. [63]

    Long-clip: Unlocking the long-text capability of clip

    Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-clip: Unlocking the long-text capability of clip. arXiv preprint arXiv:2403.15378, 2024. 1, 2, 3, 5, 6, 7, 8, 4

    2024 arXiv

  56. [64]

    Vision-language models for vision tasks: A survey

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence,

  57. [65]

    Exploring regional clues in clip for zero-shot semantic seg- mentation

    Yi Zhang, Meng-Hao Guo, Miao Wang, and Shi-Min Hu. Exploring regional clues in clip for zero-shot semantic seg- mentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3270–3280,

  58. [66]

    Dreamlip: Language- image pre-training with long captions

    Kecheng Zheng, Yifei Zhang, Wei Wu, Fan Lu, Shuailei Ma, Xin Jin, Wei Chen, and Yujun Shen. Dreamlip: Language- image pre-training with long captions. In European Confer- ence on Computer Vision, pages 73–90. Springer, 2025. 1, 2, 3, 7

    2025

  59. [67]

    Zegclip: Towards adapting clip for zero-shot semantic segmentation

    Ziqin Zhou, Yinjie Lei, Bowen Zhang, Lingqiao Liu, and Yifan Liu. Zegclip: Towards adapting clip for zero-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11175–11185, 2023. 2 FIX-CLIP: Dual-Branch Hierarc...

    2023

  60. [68]

    During the re-caption process, samples are randomly taken from the following 20 prompts

    Prompting Templates for Long-text Caption Synthesis To ensure the diversity of the synthesis long-text captions, we have set up multiple prompts to instruct Llama3-LLaV A- NeXT-8b [35] to generate long-text captions with detailed descriptions. During the re-caption process, sa...

  61. [69]

    Provide a comprehensive description of this image, including all visual elements, their spatial relationships, and the overall atmosphere

  62. [70]

    Generate a detailed caption explaining what’s happening in this image, covering actions, subjects, environment, and temporal context

  63. [71]

    Analyze this image in detail, describing the main subjects, background, lighting, colors, and composition

  64. [72]

    Write an extensive caption that captures both the explicit visual content and implicit context or story behind this image

  65. [73]

    Describe this image as if explaining it to someone who cannot see it, including all relevant details and visual nuances

  66. [74]

    Break down the scene components in this image, detailing the foreground, middle ground, and background elements

  67. [75]

    Describe the environmental context, lighting conditions, time of day, and weather elements visible in this image

  68. [76]

    Analyze the spatial arrangement and relationships between all objects and subjects in this image

  69. [77]

    Detail the setting of this scene, including architectural elements, natural features, and atmospheric conditions

  70. [78]

    Explain the visual dynamics of this scene, including movement, direction, and flow of elements

  71. [79]

    Elaborate on the image’s details such as the objects’ textures, the direction of shadows, and how they contribute to the overall look

  72. [80]

    Describe the image from top to bottom and left to right, highlighting every element and its significance within the frame

  73. [81]

    Generate a caption that delves into the emotional undertones suggested by the image’s colors, expressions of the subjects, and the setting

  74. [82]

    Analyze the image to explain how the placement of elements affects the flow and balance within the visual space

  75. [83]

    Write a detailed description of the image that includes the sizes of the objects relative to each other and their proximity

  76. [84]

    Describe the image in terms of the contrast between light and dark areas and how it shapes the perception of the scene

  77. [85]

    Generate a caption that interprets the possible narrative connections between different elements in the image

  78. [86]

    Analyze the image to explain how the colors interact with each other and what mood they create together

  79. [87]

    Write a detailed description of the image that covers the small details often overlooked, like tiny patterns on objects

  80. [88]

    Describe the image by focusing on the perspective used and how it makes the viewer experience the scene

  81. [89]

    We apply a simple filtering method on captions to reduce repeated words, meaningless sentences, and short results

    Abnormal Synthesized Captions While synthesized captions provide detailed descriptions, MLLMs usually bring hallucination elements. We apply a simple filtering method on captions to reduce repeated words, meaningless sentences, and short results. Fig. 6 shows some abnormal syn...

  82. [90]

    5th of October, there’s a significant event highlighted in blue - the launch

    Details of the Setup 8.1. Details of the training datasets Our model’s training corpus comprises six distinct datasets, as enumerated in Tab. 8. The ShareGPT4V [7] dataset, previ- ously employed in Long-CLIP [63] implementation, exhibits exceptional annotation quality. The rem...

    2012

  83. [91]

    Shared Prompts

    Raw Short Caption versus Synthesis Short Caption We identified quality limitations in the raw short captions within our training dataset through empirical observation. To address this constraint, we proposed an alternative approach utilizing synthetically generated short capti...

  84. [92]

    8, the regional prompts obtain stronger responses in the corresponding local patches

    Visualization of the Effects of Unidirectional Masking and Region Prompts In Fig. 8, the regional prompts obtain stronger responses in the corresponding local patches. The red boxes visualize how regional prompts incorporate local features, highlighting the role of Unidirectio...

  85. [93]

    Visualization of the Similarity Heatmap We visualize the heatmap of similarity between image fea- tures and text features, and compare our results with those of CLIP [44] and Long-CLIP [63], as shown in Fig. 9. To evaluate the performance on short texts, the prompt is set as ”...

  86. [94]

    We replace the original text encoder in the stable-diffusion model with that in Long- CLIP [63] or ours

    Analysis of Text-to-Image Generation Ex- amples In this section, we showcase more text-to-image generation examples in long captions to demonstrate the enhancement in understanding long texts. We replace the original text encoder in the stable-diffusion model with that in Long...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.