{"id":"b51d5835-6b77-4d9a-9344-6ae110f408e0","arxiv_id":"2505.18434","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A training-time pipeline that synthesizes diverse negation captions from batch neighbors improves CLIP's negation accuracy on matching and generation benchmarks, and a new NEG-TTOI benchmark measures negation handling in text-to-image generation.","lead":"This paper teaches CLIP to understand negation like 'no dog' or 'without a ball' by generating practice captions on the fly during training, adding only 2.5% overhead. The authors also release a 2,000-prompt benchmark to test whether text-to-image models avoid drawing objects that are mentioned as absent.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The training-time negation labels are generated without verifying that the negated object is truly absent, so false labels may corrupt the core training signal and undermine the central claim.","rationale":"The reader's weakest assumption accurately identifies the most load-bearing risk: the generated negation captions are not verified against the image content, so the training signal may teach incorrect negation semantics. This concern is more fundamental than the abstract's overclaim about retrieval SOTA, because it directly challenges the mechanism by which TNG-CLIP is supposed to acquire negation awareness. The paper's own ablations (Table 6) show that removing the compositional or full negation alignments changes matching accuracy, confirming that these generated captions are load-bearing components of the loss rather than incidental noise. Since the paper provides no estimate of how often the generated negation object is actually absent, the central claim should be accepted only conditionally on an empirical label-correctness audit. The reader already reached CONDITIONAL, so my read does not change that verdict, but it sharpens the condition: measure the false-absence rate before trusting the training-time generation mechanism.","tokens_in":15179,"tokens_out":5706,"duration_ms":53013,"concrete_test":"Sample 1,000 compositional-negation pairs from actual training batches after the Section 3.1.1 similarity selection, and use an open-vocabulary object detector or human annotators to check whether O_n is actually absent from I_i. Report the false-absence rate, i.e., the fraction of pairs where O_n is present in the image. If this rate is non-trivial (e.g., more than 5%), the training signal is corrupted; the authors should also retrain with O_n filtered to verified-absent objects and compare NegBench-MSCOCO matching accuracy to see whether the current unverified generation is actually responsible for the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's core training signal is generated without any verification that the negated object is actually absent from the paired image. In Sections 3.1.1–3.1.2, the negation object O_n for image-text pair P_o is taken from the caption of the most visually similar pair P_s (Eq. 2), and in Section 3.1.3 this O_n is used to build a compositional negation caption 'A, but not O_n around' paired with I_i. But neither the pipeline nor any reported analysis checks whether O_n is present in I_i; existing image-text captions commonly omit objects that are visually present, and visually similar images frequently share objects. Thus a caption asserting 'no boy around' can be paired with an image that in fact contains a boy. Similarly, Section 3.1.4 pairs a full-negation caption from a random other sample with I_i without checking that the negated scene is absent from I_i. Because the text-to-image objective in Section 3.2 directly aligns these generated captions with I_i, false negation labels are treated as true supervision. The paper's central claim that TNG-CLIP learns correct negation semantics from training-time generation is therefore contingent on an unmeasured label-correctness rate, and the paper provides no evidence that this rate is high.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TNG-CLIP, a method for improving negation understanding in CLIP by generating negation captions dynamically during training. For each image-text pair in a batch, the pipeline finds the visually most similar pair via cosine similarity of frozen image features, extracts a noun from the similar pair's caption, and uses template-based rewriting to produce a compositional negation caption (e.g., \"A dog playing a ball, but not a boy around\") and a full negation caption sampled from another random pair. Each image is then associated with three captions (original, compositional negation, full negation) and trained with an asymmetric objective: the text-to-image loss aligns all three captions with the correct image, while the image-to-text loss uses randomly assigned labels as noise. The paper also introduces NEG-TTOI, a benchmark of 2000 text-to-image generation prompts with positive and negative questions. Experiments on NegBench, VALSE-Existence, and image generation with Stable Diffusion report strong gains over prior negation-aware CLIP baselines, with a reported 2.5% training-time overhead.","tokens_in":15439,"tokens_out":5667,"duration_ms":50511,"significance":"If the reported results hold, the paper's contribution is practically valuable: it offers a way to improve negation understanding with no per-sample LLM calls, no pre-generated negation dataset, and low training overhead, while extending evaluation to text-to-image generation. The paper also ships useful ablation evidence for the dynamic-data design and for the asymmetric noise objective, and it provides a new benchmark that could support future work. The central idea is plausible, but the correctness of the automatically generated training labels is not verified, which is a load-bearing gap; the abstract also overclaims retrieval-state-of-the-art. These issues are fixable, but they need to be addressed before the claims can be accepted.","major_comments":[{"comment":"The training-time negation labels are generated without verifying that the negated object is actually absent from the paired image. For the compositional negation caption, the negation object O_n is extracted from the caption of the most similar image-text pair P_s (Eq. 2) and then asserted to be absent from I_i, but neither the pipeline nor any reported analysis checks this. For the full negation caption, a random other sample's caption is paired with I_i without checking that the negated scene is absent. Since the text-to-image loss in Section 3.2 directly aligns these captions with I_i, false labels such as \"not a boy around\" on an image that contains a boy are trained as correct supervision. Similar images often share common objects, and image captions routinely omit visible objects, so this is a concrete risk rather than a hypothetical one. The paper should provide a quantitative audit of label correctness on a sample of generated captions, or add a filtering/verification step, before claiming that TNG-CLIP learns correct negation semantics.","section":"Sections 3.1.1-3.1.4, Eq. (2), and Section 3.2"},{"comment":"The abstract claims state-of-the-art performance on \"image-to-text matching, text-to-image retrieval, and image generation,\" but Table 1 shows that TNG-CLIP's Neg-R@5 on text-to-image retrieval is 61.11, lower than NegCLIP's 64.41. The main text itself states that TNG-CLIP \"slightly underperforms the NegCLIP model on retrieval tasks.\" The abstract should be corrected to avoid claiming retrieval state-of-the-art, or the claim should be restricted to the matching and generation tasks.","section":"Abstract and Table 1"},{"comment":"The random-label image-to-text objective is central to the proposed training recipe, but it is justified only by the informal argument that negation captions are out-of-distribution and that noise prevents overfitting. Table 6 provides an ablation showing that removing the noise degrades retrieval, which is useful, but the design is not compared with a more principled alternative such as using the original captions for image-to-text alignment with a weighting scheme or a temperature-scaled soft label. As written, the reader cannot tell whether the benefit comes from the specific noise mechanism or from simply reducing the strength of the image-to-text alignment. A targeted comparison would strengthen the paper's central training contribution.","section":"Section 3.2, Eq. (4)"}],"minor_comments":[{"comment":"Typo: \"denoated\" should be \"denoted\".","section":"Section 3.1.4"},{"comment":"Typo: \"mdoel's\" should be \"model's\".","section":"Section 6"},{"comment":"Typo: \"imaget-to-text\" should be \"image-to-text\".","section":"Section 5.1, Valse-Existence bullet"},{"comment":"Typo: \"Generaiton\" should be \"Generation\".","section":"Table 8 caption"},{"comment":"The sentence \"we replace thecapwith the provided original caption\" should read \"we replace {cap} with the provided original caption\" for clarity.","section":"Appendix A.5"},{"comment":"Related work cites \"Truong et al.\" and \"Zhang et al.\" without years or reference list entries; please add the full citations. Also, Alhamoud et al. 2025a and 2025b point to the same arXiv preprint; please disambiguate.","section":"References"},{"comment":"The GitHub repository link is a placeholder text \"here\"; provide the actual URL or remove the footnote.","section":"Footnote 1"},{"comment":"Main results are reported without variance or significance tests; given the small margins in some comparisons, adding error bars or repeated-run statistics would improve confidence.","section":"Tables 1-3, 5"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: genuinely new idea—generating negation captions on the fly from batch neighbors instead of pre-generating an LLM dataset—and the paper backs it with careful ablations. The soft spots are real, though not fatal.\n\nWhat's new: prior negation-aware CLIP methods (NegCLIP, CoN-CLIP, NegationCLIP, CC12M-NegFull) rely on large pre-generated negation datasets, which are expensive and fixed. TNG-CLIP picks the most visually similar image-text pair in the batch, extracts a noun from its caption that is semantically distant from the original caption, and templates it into compositional and full negation captions. This is cheap (2.5% overhead) and gives every epoch different negatives. The dynamic-dataset comparison is a genuine plus: it beats a fixed dataset at comparable compute, and the ablation of the asymmetric noise-augmented objective is thorough. The NEG-TTOI generation benchmark is a useful addition, with LLM verification in construction and 2000 samples.\n\nThe biggest soft spot: labels are never verified for correctness. Sections 3.1.1–3.1.3 generate 'but not {object} around' for an image without checking that the object is actually absent. COCO captions omit many visible objects, and the most similar image often shares objects with the query. Full-negation captions are paired randomly without checking absence either. The paper reports no statistics on false-label rate. Since the text-to-image objective treats these captions as true positives, false labels directly train wrong semantics. This is load-bearing for the claim that TNG-CLIP learns 'correct negation semantics'—even though the model still improves across benchmarks, the training signal is noisier than claimed. The stress-test note is on target. A peer-review version needs either a filtering step or a quantification of label correctness.\n\nSecond: the abstract says SOTA on text-to-image retrieval, but Table 1 shows NegCLIP beats TNG-CLIP on both R@5 and Neg-R@5. The paper's own Section 5.1 admits this. The abstract should be corrected.\n\nThird: the code link is a placeholder ('can be found here'). No URL. Minor but annoying.\n\nCitation pattern looks fine; the relevant negation and dynamic-dataset literature is covered. The 'first benchmark' claim for NEG-TTOI is defensible because Park et al. only had a 107-prompt experiment.\n\nWho this is for: anyone fine-tuning CLIP for negation, or using CLIP as a text encoder for generation. It deserves a serious referee. I'd send it to review, conditional on the label-verification analysis, corrected retrieval claim, and code release.","headline":"A clever training-time negation data-augmentation trick with strong empirical results, but the training labels are unverified and the abstract overclaims retrieval SOTA.","tokens_in":15933,"tokens_out":4137,"would_cite":true,"duration_ms":36723,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training-time negation caption generation makes CLIP negation-aware and sets state-of-the-art results across matching, retrieval, and generation.","keywords":["negation understanding","CLIP fine-tuning","contrastive learning","image-text matching","text-to-image retrieval","text-to-image generation","training-time data augmentation","negation benchmark"],"falsifier":"Run an object detector over query images during training and compute the fraction of generated compositional captions whose negated object is actually present in the image; if TNG-CLIP's gains depend on that fraction, the value of the method rests on label correctness. A direct test is to filter out all generated captions with a false negation and retrain: if accuracy does not drop, the method was learning from false labels.","tokens_in":1749,"feed_emoji":"🚫","tokens_out":2190,"duration_ms":78222,"temperature":0.7,"pith_summary":"TNG-CLIP is a fine-tuning procedure that makes CLIP negation-aware without any pre-built negation dataset or per-sample LLM calls. During each training batch, the method finds the visually most similar image-caption pair for every example, extracts a noun from that neighbor's caption, and uses template patterns to synthesize two kinds of negation captions: a compositional one (the original caption, but not the noun) and a full negation of some other caption in the batch. These dynamically generated captions are fed into an asymmetric contrastive objective, adding about 2.5% training time. The paper also contributes Neg-TtoI, a 2000-prompt benchmark for text-to-image generation under negation, and reports state-of-the-art results on image-to-text matching, text-to-image retrieval, and generation benchmarks. If correct, negation understanding can be improved cheaply and without reliance on LLM-generated training data.","feed_headline":"On-the-fly negation captions make CLIP negation-aware","feed_subtitle":"2.5% extra training time delivers top scores in matching, retrieval, and text-to-image generation.","key_machinery":"The central mechanism is batch-internal, template-based negation caption generation coupled with an asymmetric noise-augmented contrastive loss. The generator identifies a negation object from a visually similar neighbor's caption using cosine similarity, part-of-speech tagging, and semantic-distance filtering; then it applies one of 46 compositional or 18 full-negation patterns to build captions. Because the neighbor and the template are chosen per batch and per epoch, the same image receives different negation captions across training. The objective matters as much as the generation: the text-to-image term aligns original and negation captions with the image, while the image-to-text term uses random label noise, motivated by the claim that negation captions are out-of-distribution for pretrained CLIP, so that fine-tuning does not overfit. The Neg-TtoI benchmark, built from MS-COCO captions via LLM rewriting and question-answer verification, supplies the first text-to-image negation evaluation.","core_discovery":"The paper's central discovery claim is that negation understanding in CLIP can be trained from negation captions synthesized on the fly from other captions in the same batch, instead of from a fixed, LLM-generated negation dataset. For each image-text pair, the pipeline embeds the batch images, picks the most similar image by cosine similarity, extracts noun phrases from that image's caption with a part-of-speech tagger, filters the noun through a lexical network to keep it semantically distant from the original caption, and inserts it into a randomly chosen negation template to form a compositional negation caption. A second full-negation caption is formed by negating an unrelated caption from another random pair. Each image is then paired with its original caption, its compositional negation, and a full negation from another sample. Using an asymmetric objective, the text-to-image direction aligns all three captions to the image while the image-to-text direction injects random label noise, and the visual encoder is frozen throughout. The paper reports that this procedure outperforms existing negation-aware CLIP variants on NegBench and Valse-Existence, and that plugging the fine-tuned text encoder into a latent diffusion model improves generation on the proposed Neg-TtoI benchmark.","pith_inferences":["The unverified negation label is the soft spot: the noun borrowed from the most similar image's caption may already be present in the query image but unmentioned, so some training captions are false. A cheap detector-based or caption-consistency filter on the selected noun would test whether removing those false labels improves or changes the results.","Because part of the gain is attributed to random label noise in the image-to-text loss, the method suggests that negation fine-tuning succeeds by preventing overfitting to out-of-distribution text rather than only by adding negation examples; future work could separate these two effects by adding noise to an equal number of non-negation captions.","The same training-time construction could extend beyond image-text pairs to other contrastive or generative objectives, for instance visual question answering where a question about an absent object needs an aligned negative answer.","Neg-TtoI's LLM-based judge checks surface semantics; harder negatives that confuse the generator with visually similar distractor objects would reveal whether the model truly understands absence or only avoids the literal word."],"forward_implications":["Fine-tuning with dynamically generated negation captions costs only about 2.5% extra training time, so the method removes the main computational barrier to negation-aware CLIP.","A single text-encoder fine-tune transfers to image generation: replacing a diffusion model's text encoder with TNG-CLIP improves accuracy on Neg-TtoI over the original model and over CoN-CLIP.","Dynamic per-epoch negation captions outperform a fixed dataset of the same generated captions (51.61 vs 49.52 average accuracy on NegBench-MSCOCO matching), suggesting that diversity itself helps generalization.","The proposed Neg-TtoI benchmark gives a reusable 2000-prompt test for whether generative models both include the desired objects and omit the negated ones.","Zero-shot image classification remains near pretrained-CLIP levels, indicating the negation fine-tune does not cause catastrophic forgetting."],"supporting_citations":[{"why":"Supplies the CLIP architecture and pretrained weights that TNG-CLIP fine-tunes.","marker":"Radford et al., 2021"},{"why":"Provides NegBench evaluation and the CC12M-NegFull baseline that TNG-CLIP is compared against.","marker":"Alhamoud et al., 2025a"},{"why":"NegCLIP baseline; the retrieval comparison TNG-CLIP is measured against.","marker":"Yuksekgonul et al., 2023"},{"why":"CoN-CLIP baseline and the precedent of freezing the visual encoder while fine-tuning the text encoder.","marker":"Singh et al., 2024"},{"why":"NegationCLIP baseline and the strategy of replacing a diffusion model's text encoder to test negation in generation.","marker":"Park et al., 2025"},{"why":"Stable Diffusion backbone whose text encoder is replaced with the fine-tuned CLIP for Neg-TtoI experiments.","marker":"Rombach et al., 2022"},{"why":"Provides the part-of-speech tagger used to extract candidate negation nouns from captions.","marker":"Bird et al., 2009"},{"why":"WordNet supplies the semantic-distance filter for choosing a negation object distant from the original caption.","marker":"Miller, 1995"},{"why":"GPT-4o generates the negation templates and the Neg-TtoI benchmark prompts and verification questions.","marker":"OpenAI et al., 2024"},{"why":"The theoretical argument that fine-tuning on out-of-distribution labels can distort features, motivating the noisy image-to-text loss.","marker":"Kumar et al., 2022"}],"fun_headline_variants":["Train CLIP to negate in 2.5% extra time","Synthesize negation captions during training","Negation-aware CLIP with on-the-fly captions","Add 2.5% training time to make CLIP negation-aware","Negation captions generated mid-training, no LLM needed"],"cache_read_input_tokens":18176,"weakest_assumption_plain":"The load-bearing premise is that the noun selected from the most similar image's caption is genuinely absent from the original image, but the pipeline never verifies this absence, so false negation labels can be fed into training.","fun_headline_variants_meta":{"raw":{"variants":["Train CLIP to negate in 2.5% extra time","Synthesize negation captions during training","Negation-aware CLIP with on-the-fly captions","Add 2.5% training time to make CLIP negation-aware","Negation captions generated mid-training, no LLM needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000563,"raw_usage":{"total_tokens":2688,"prompt_tokens":980,"completion_tokens":1708,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":1622}},"tokens_in":596,"tokens_out":1708,"duration_ms":11140,"temperature":1.0,"reasoning_tokens":1622,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:30:43.287328+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run an object detector over query images during training and compute the fraction of generated compositional captions whose negated object is actually present in the image; if TNG-CLIP's gains depend on that fraction, the value of the method rests on label correctness. A direct test is to filter out all generated captions with a false negation and retrain: if accuracy does not drop, the method was learning from false labels.","supporting_citations":[],"review_version":1}