{"id":"155df979-7393-47c4-ba56-f578621f0121","arxiv_id":"2608.05260","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Training only the text encoder on multi-sentence paragraphs improves long-description image retrieval by over 14 points on DOCCI and removes the need for context-window architecture tricks.","lead":"This paper tests whether training a vision-language model on long paragraph descriptions instead of short captions makes it better at finding images from detailed text. A paragraph-trained text encoder matched a larger Long-CLIP model on one benchmark and beat it by 14 points on another, with no architecture changes.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ShareGPT4V 'match' may be inflated by training-set overlap: the benchmark's 30K LLaVA-Pretrain images derive from CC3M, the same source as the 500K training set, and no overlap filtering is reported.","rationale":"The reader's weakest assumption identifies exactly the most load-bearing concern. The central claim has two quantitative pillars: matching Long-CLIP-L on ShareGPT4V and beating it on DOCCI. The DOCCI pillar is unaffected by overlap because DOCCI images are not in CC3M, and it is the paper's own 'true test of generalization.' However, the ShareGPT4V 'match' is an explicit part of the abstract, conclusion, and Sec. 4.2, and the paper's assertion 'without having seen any ShareGPT4V data during training' is directly falsifiable if the LLaVA-Pretrain portion of the eval set overlaps with the CC3M training sample. No overlap filtering, image-ID deduplication, or sensitivity analysis is reported, leaving the number unverified. The internal controlled ablation (C1–C10) and the truncation analysis provide independent evidence that paragraph granularity helps long-text retrieval, so a full rejection is not warranted. A CONDITIONAL verdict requiring the overlap check, artifact release, and a re-evaluation on the disjoint subset is the correct outcome. Since the reader already assigned CONDITIONAL, the verdict should remain unchanged.","tokens_in":24402,"tokens_out":15928,"duration_ms":139408,"concrete_test":"Compute the exact image-level overlap between the paper's 500K CC3M training subset and the 30K LLaVA-Pretrain images in the ShareGPT4V 80K eval set using image IDs or URLs. Then re-evaluate C4 and C5 I2T R@1 only on the non-overlapping ShareGPT4V subset. If the scores drop by more than ~2 points and fall below Long-CLIP-L's 52.6%, the 'match' claim is an artifact of training-set leakage. Also report the overlap count in the paper as a reproducibility footnote.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim (abstract, Sec. 4.2) is that paragraph-supervised models 'match Long-CLIP-L on ShareGPT4V' without having seen ShareGPT4V data. The ShareGPT4V evaluation set is described in Table 2 as a fixed 80K sample comprising 50K COCO + 30K LLaVA-Pretrain images. LLaVA-Pretrain is a subset of CC3M, and the paper's training set is described as '500K CC3M images' (Sec. 3.1). If the 500K sample overlaps substantially with those 30K LLaVA-Pretrain images, then during fine-tuning the text encoder was trained on synthetic paragraphs for those exact images. At evaluation, the frozen vision encoder produces the same image embedding the model learned to align with paragraph text, giving an inflated I2T R@1 for the 'unseen' ShareGPT4V set. The reported 51.7–52.0% is within 0.6–0.9 points of Long-CLIP-L's 52.6%; even a modest leakage effect could erase this apparent match. The paper does not report any overlap filtering or quantify the overlap. The DOCCI result is on a genuinely distinct dataset and is not affected, so the overall direction may survive, but the ShareGPT4V pillar of the central claim is not currently supported without an overlap analysis.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how the granularity of text supervision affects contrastive vision-language retrieval. The authors construct a synthetic dataset from 500K CC3M images using Qwen2-VL to generate five positive captions, five hard-negative captions, and one multi-sentence paragraph per image, with Llama 3.2 Vision scoring paragraph faithfulness and triggering regeneration below score 5. They fine-tune only the BLIP text encoder and text projection, freezing the vision encoder, across 10 configurations that vary caption count, paragraph presence, and hard-negative inclusion. Evaluations are on Flickr30k, COCO (full-set), an 80K ShareGPT4V sample, and DOCCI. The main claims are that paragraph-supervised models match Long-CLIP-L on ShareGPT4V and exceed it by more than 14 points on DOCCI image-to-text R@1 without architectural changes; caption-only fine-tuning degrades long-description retrieval; more caption diversity helps short captions but hurts long descriptions; hard negatives are harmful under text-only fine-tuning; and truncation analysis shows paragraph-trained models use longer token budgets effectively.","tokens_in":24654,"tokens_out":13726,"duration_ms":116268,"significance":"If the results hold, the paper provides a clean empirical dissociation of text granularity from architectural and data-scale confounds. The frozen-vision design across 10 configurations is a useful experimental contribution, and the DOCCI benchmark is genuinely unseen by all training stages, so the >14-point I2T gain over Long-CLIP-L is a substantial result. The truncation analysis offers a plausible mechanism rather than a black-box comparison. The paper contains no fitted constants and its evidence is empirical comparisons against external benchmarks, which is a strength. The main weakness is the ShareGPT4V evaluation: because the 80K sample includes 30K LLaVA-Pretrain images from the CC3M source used for training, the 'unseen' claim for that benchmark is not currently established. The central contribution may survive on DOCCI alone, but the headline claim needs to be repaired with an overlap analysis.","major_comments":[{"comment":"The claim that paragraph-supervised models 'match Long-CLIP-L on ShareGPT4V ... without having seen any ShareGPT4V data' is not supported as stated because the evaluation set overlaps with the training data source. The 80K ShareGPT4V sample is described as 50K COCO images plus 30K LLaVA-Pretrain images, and LLaVA-Pretrain is built from CC3M, the same dataset from which the 500K training images are sampled in Sec. 3.1. If those 30K images are present in the 500K training set, the text encoder was fine-tuned on synthetic paragraphs for exactly those evaluation images, which can inflate the I2T R@1 reported for C4-C6. No overlap filtering or overlap statistics are reported anywhere, and the Limitations section (Sec. 6) does not mention this risk. Please compute the overlap by image ID or perceptual hash, report results on the overlapping and non-overlapping subsets separately, and either remove or carefully qualify the 'unseen' claim. The DOCCI results are on a distinct dataset and are not affected by this issue.","section":"Sec. 4.2, Table 2"},{"comment":"The full-set evaluation protocol is a deliberate departure from the standard 1K/5K Flickr30k and COCO splits, but the manuscript does not state whether all baselines (BLIP0, CLIP0, Long-CLIP-B, Long-CLIP-L) were evaluated by the authors with identical preprocessing, tokenization, and retrieval code. Because full-set numbers are not directly available from prior publications, the comparisons cannot be verified without this information. In addition, Long-CLIP is described as fine-tuned on 1M ShareGPT4V pairs (Sec. 4.2), and the ShareGPT4V evaluation sample in Table 2 includes 50K COCO images; evaluating Long-CLIP on the full COCO train2017 set may therefore be favorable to Long-CLIP because of training-data overlap. Please state explicitly how each baseline was obtained, add any overlap filtering, and consider reporting standard-split results in a supplementary table for comparability with the literature.","section":"Sec. 3.4, Table 1"}],"minor_comments":[{"comment":"The token-budget treatment of ShareGPT4V differs between Table 2 (77 tokens) and Table 4 (128 tokens for the same C5 checkpoint), yielding 51.7% versus 63.1% I2T R@1. The protocol note in Table 4 is important and should be stated in Sec. 4.2 as well, so that the reader understands that the 'match' claim is a conservative, 77-token comparison and that the model improves substantially at its native training budget.","section":"Sec. 4.2 vs. Table 4"},{"comment":"The paper should specify the exact positive and negative counts for the 'Negative' rows in Table 3; as written, the reader cannot tell how many positive captions accompany the 1, 3, or 5 hard negatives.","section":"Sec. 5.1, Table 3"},{"comment":"The column header 'CFG5' appears to be a typo for C5; please correct it.","section":"Sec. 5.2, Table 4"},{"comment":"The hard-negative finding is stated as if it applies to hard negatives generally, but the experiments use synthetic factually-incorrect captions in the denominator with a frozen vision encoder; the conclusions should be scoped to this setup, since standard in-batch hard negatives may behave differently.","section":"Sec. 6"},{"comment":"The statement 'Code and dataset are available here' does not include a URL; please provide a working link or repository identifier.","section":"Abstract and Sec. 3.4"},{"comment":"The truncation analysis uses 10K samples from ShareGPT4V and DOCCI without specifying the selection method or random seed; please add this information for reproducibility.","section":"Sec. 4.3, Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The ShareGPT4V overlap concern is the main obstacle to publication. If the overlap with the 500K CC3M training set is substantial, the abstract's 'match' claim must be revised or dropped; however, the DOCCI result is independent and the paper's overall direction is likely sound. The full-set evaluation protocol also needs to be documented more carefully. I would like to see the overlap analysis and the baseline-evaluation details before a final decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a look, and the referee time is justified, but the headline claim needs a carve-out. The DOCCI result is the real finding: a BLIP text encoder fine-tuned on paragraphs, with the vision encoder frozen, beats Long-CLIP-L by 14 points on image-to-text R@1 on a benchmark none of the models were trained on. That is a large, believable effect, and it is backed by a clean controlled design: ten text-supervision configurations, a shared frozen vision encoder, and a synthetic paragraph pipeline with a quality-scoring loop. The truncation analysis is the most instructive piece—paragraph-trained models keep gaining recall as the token budget grows to 128, while caption-trained models plateau or drop past 60 tokens. That is a genuine mechanistic observation.\n\nThe soft spots are real but fixable. The most serious is the ShareGPT4V evaluation: the 80K sample includes 30K LLaVA-Pretrain images, which are CC3M images, and the training set is 500K CC3M images. The paper reports no overlap filtering. A random 500K sample of CC3M would overlap a random 30K LLaVA-Pretrain sample by thousands of images (roughly 15% of that subset). At evaluation, the model has already been fine-tuned to align those exact images with long paragraph text, so the \"matches Long-CLIP-L on ShareGPT4V without having seen the data\" claim is inflated by an unknown amount. The gap to Long-CLIP-L is 0.6-0.9 points; even partial overlap could erase it. This is not a fatal flaw because the DOCCI result is unaffected, but the abstract should be toned down.\n\nTwo smaller issues. The paper uses a nonstandard full-set retrieval protocol on Flickr30k and COCO but does not state explicitly that the Long-CLIP and CLIP baselines were recomputed under the same protocol. If those numbers come from the original papers, the comparison is invalid. And there are no error bars anywhere; the ShareGPT4V difference is within the range where seed-to-seed variance matters.\n\nThe hard-negative finding—collapse from ~50% to ~25% with a single hard negative—is surprising and well documented; I'd want to see it replicated under a second seed before treating it as robust, but the direction is plausible given the frozen vision encoder can't co-adapt. The data-efficiency result that 25% of paragraphs does best on short-caption retrieval is similarly provisional.\n\nNo circularity or invented entities; the claims are empirical comparisons against external benchmarks. The paper is honest about its scope.\n\nFor a referee: send it out, but require an overlap analysis for ShareGPT4V, an explicit statement that baselines were rerun on the same splits, and error bars on the headline numbers. The authors have a solid contribution underneath; it deserves a proper review rather than a desk rejection.","headline":"The DOCCI result is real and useful; the ShareGPT4V 'match' is likely inflated by train/eval overlap, so the headline claim needs qualification before it can be trusted.","tokens_in":25198,"tokens_out":7275,"would_cite":true,"duration_ms":61829,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Paragraph supervision, not a longer context window, may be the real key to long-description retrieval.","keywords":["vision-language retrieval","text supervision granularity","paragraph descriptions","contrastive learning","BLIP text encoder fine-tuning","long-description retrieval","hard negatives","truncation analysis"],"falsifier":"Re-run the ShareGPT4V evaluation excluding the 30K LLaVA-Pretrain images that overlap with CC3M (or use a strictly disjoint held-out set), and check whether the paragraph-trained model still matches Long-CLIP-L. Additionally, retrain the C5 configuration with paragraphs truncated to 60 tokens and verify whether the DOCCI and ShareGPT4V gains disappear, which would confirm that the benefit comes from processing the full paragraph rather than from the training data distribution alone.","tokens_in":24189,"feed_emoji":"🖼️","tokens_out":1686,"duration_ms":14939,"temperature":0.7,"pith_summary":"This paper argues that the granularity of the text used to train a vision-language model, rather than its architectural context length, is the main lever for retrieving images from long, detailed descriptions. By fine-tuning only the text encoder of a frozen BLIP model on either single captions or multi-sentence paragraphs, the authors isolate text granularity as the sole variable. Their paragraph-trained model matches Long-CLIP-L on ShareGPT4V and outperforms it by over 14 points on DOCCI image-to-text retrieval, without any positional-embedding or architecture changes. A sympathetic reader would care because it suggests a cheaper, data-centric path to long-text retrieval that competes with architectural extensions.","feed_headline":"Richer captions beat wider context windows for long-text retrieval","feed_subtitle":"Fine-tuning a text encoder on paragraphs matches Long-CLIP without changing the architecture","key_machinery":"The central object is the text-supervision granularity, varied across ten configurations from a single caption to a multi-sentence paragraph. The mechanism is a controlled training setup that freezes the BLIP vision encoder and fine-tunes only the text encoder and projection, so any performance difference is attributable to the text input. The paper also relies on a synthetic data pipeline using Qwen2-VL to generate diverse captions, hard negatives, and quality-scored paragraphs (with Llama 3.2 Vision as a scorer), and on a truncation analysis that shows how recall changes with token budget.","core_discovery":"The paper's central claim is that training text granularity alone determines long-text retrieval performance, and that paragraph-level supervision is sufficient to match or beat an architecture built for long contexts. Holding the vision encoder fixed and fine-tuning only the BLIP text encoder, the authors find that a configuration using the original caption plus one generated paragraph (C5) reaches 66.1% I2T R@1 on DOCCI versus 51.6% for Long-CLIP-L, and matches Long-CLIP-L on ShareGPT4V (around 52% vs. 52.6%). They also report a mechanistic distinction: paragraph-trained models improve monotonically as token budget grows from 20 to 128, whereas caption-only training plateaus or degrades beyond 60 tokens, indicating that paragraph supervision teaches the text encoder to use extended sequences productively.","pith_inferences":["The paper's finding suggests a data-centric alternative to architectural fixes: even a modest amount of paragraph data (25% of 500K images) captures most of the long-text benefit, implying that collecting or generating paragraph annotations could be a cost-effective upgrade for existing CLIP-style models.","The degradation of caption-only training on long descriptions hints that the text encoder, when fine-tuned on short texts, actively narrows its representational capacity; this could be tested by evaluating such models on zero-shot long-text tasks beyond retrieval, such as image captioning with detailed prompts.","The success of paragraphs over multiple captions, despite the latter providing more textual diversity, suggests that coherence and structure matter more than raw token count; this could be probed by ablating paragraph structure (e.g., scrambling sentences) while keeping the same vocabulary.","The hard-negative collapse in the text-only fine-tuning regime may be a specific instance of a broader phenomenon: when one encoder is frozen, negative sampling that relies on joint adaptation can push the trainable encoder into a degenerate subspace; this hypothesis could be tested by unfreezing the vision encoder for a few epochs and observing whether the hard-negative damage reverses."],"forward_implications":["If paragraph supervision is the main driver, practitioners can improve long-description retrieval by investing in richer text annotations rather than extending context windows or modifying positional embeddings.","Caption-only fine-tuning on synthetic captions may actively degrade long-description retrieval, so data curation for retrieval should prioritize paragraph-level descriptions.","Hard negative captions, typically beneficial in contrastive learning, are harmful in a text-only fine-tuning setting with a frozen vision encoder, suggesting that negative sampling strategies need to be re-evaluated for partial-update training.","The monotonic improvement of paragraph-trained models with token budget implies that longer paragraphs (beyond the 128-token limit) could unlock further gains, and that the text encoder's capacity is not the bottleneck.","The Pareto analysis shows that paragraph-containing configurations achieve large long-description gains with only modest short-caption trade-offs, offering a practical operating point for retrieval systems that must handle both regimes."],"supporting_citations":[{"why":"Long-CLIP is the primary baseline that extends CLIP's token limit through positional-embedding interpolation; the paper's headline comparison is against Long-CLIP-L on ShareGPT4V and DOCCI.","marker":"[24]"},{"why":"BLIP supplies the backbone model (ViT-B, base-COCO) whose text encoder is fine-tuned while the vision encoder stays frozen.","marker":"[7]"},{"why":"CC3M is the source of the 500K training images used to generate the synthetic captions, hard negatives, and paragraphs.","marker":"[18]"},{"why":"Qwen2-VL generates the diverse captions, hard negative captions, and paragraphs in the synthetic pipeline.","marker":"[21]"},{"why":"Llama 3.2 Vision scores paragraph quality in the feedback loop, filtering paragraphs below score 5.","marker":"[3]"},{"why":"ShareGPT4V is one of the two long-description benchmarks used for evaluation; the paper compares against its 80K subset.","marker":"[1]"},{"why":"DOCCI is the other long-description benchmark, unseen by all models, serving as the test of generalization.","marker":"[10]"},{"why":"InfoNCE is the contrastive loss that the paper extends to multi-positive and hard-negative variants for the training configurations.","marker":"[11]"},{"why":"DiHT is cited as a representative work that advocates hard negatives in vision-language pretraining; the paper contrasts its findings with this prior claim.","marker":"[12]"}],"fun_headline_variants":["Paragraph supervision beats Long-CLIP without architecture changes","Train on paragraphs to improve long-text retrieval, not captions","Long-text retrieval is determined by text granularity, not context window","Match Long-CLIP performance with no architecture change: use paragraphs","For long descriptions, paragraph supervision outperforms caption training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The ShareGPT4V evaluation sample is assumed to be unseen during training, but 30K of its images come from LLaVA-Pretrain, which is built from CC3M, the same source as the paper's 500K training set—so those images may have been seen with other captions during fine-tuning.","fun_headline_variants_meta":{"raw":{"variants":["Paragraph supervision beats Long-CLIP without architecture changes","Train on paragraphs to improve long-text retrieval, not captions","Long-text retrieval is determined by text granularity, not context window","Match Long-CLIP performance with no architecture change: use paragraphs","For long descriptions, paragraph supervision outperforms caption training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000902,"raw_usage":{"total_tokens":3903,"prompt_tokens":985,"completion_tokens":2918,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":2834}},"tokens_in":601,"tokens_out":2918,"duration_ms":20206,"temperature":1.0,"reasoning_tokens":2834,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T16:53:52.957526+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the ShareGPT4V evaluation excluding the 30K LLaVA-Pretrain images that overlap with CC3M (or use a strictly disjoint held-out set), and check whether the paragraph-trained model still matches Long-CLIP-L. Additionally, retrain the C5 configuration with paragraphs truncated to 60 tokens and verify whether the DOCCI and ShareGPT4V gains disappear, which would confirm that the benefit comes from processing the full paragraph rather than from the training data distribution alone.","supporting_citations":[{"cited_title":"In: European Conference on Computer Vision (ECCV)","cited_arxiv_id":null,"evidence_quote":"Long-CLIP is the primary baseline that extends CLIP's token limit through positional-embedding interpolation; the paper's headline comparison is against Long-CLIP-L on ShareGPT4V and DOCCI."},{"cited_title":"In: International Con- ference on Machine Learning (ICML)","cited_arxiv_id":null,"evidence_quote":"BLIP supplies the backbone model (ViT-B, base-COCO) whose text encoder is fine-tuned while the vision encoder stays frozen."},{"cited_title":"In: Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","cited_arxiv_id":null,"evidence_quote":"CC3M is the source of the 500K training images used to generate the synthetic captions, hard negatives, and paragraphs."},{"cited_title":"In: Eu- ropean Conference on Computer Vision (ECCV)","cited_arxiv_id":null,"evidence_quote":"ShareGPT4V is one of the two long-description benchmarks used for evaluation; the paper compares against its 80K subset."},{"cited_title":"In: ECCV (2024)","cited_arxiv_id":null,"evidence_quote":"DOCCI is the other long-description benchmark, unseen by all models, serving as the test of generalization."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","cited_arxiv_id":null,"evidence_quote":"DiHT is cited as a representative work that advocates hard negatives in vision-language pretraining; the paper contrasts its findings with this prior claim."}],"review_version":1}