{"id":"2c5291d3-5f03-4977-85f5-f59fc0aaab2d","arxiv_id":"2502.01385","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Backdoor-poisoned CLIP training samples have unusually sparse local neighborhoods in representation space, which makes them detectable by standard local outlier detectors with high accuracy.","lead":"This paper shows that backdoor-poisoned images in CLIP training data stand out as statistical outliers in the model's learned representation space, so simple outlier-detection algorithms can find them. A smart generalist would read it because it offers a cheap, practical way to clean large web-scale training datasets of hidden backdoor attacks, before they get baked into deployed models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unintentional backdoor claim is vulnerable to initialization bias in trigger recovery; missing same-target random-init control.","rationale":"The reader's weakest_assumption concerns the sparsity geometry underlying the detection method, and the reader mentions the trigger-recovery initialization separately in the rationale. I identify the trigger-recovery bias as the single most load-bearing concern because it is concrete, testable, and directly threatens a headline claim of the paper, whereas the sparsity assumption is supported by extensive experiments across architectures and datasets and is already qualified by the paper's own low-ASR results (e.g., ViT clean-label AUC ~71%). The paper's provided counter-example (Appendix B.4) is not a valid control: it changes both the target class and the initialization simultaneously, so it cannot rule out that the high ASR for the birthday cake target stems from starting the optimization at the target image. A same-target random-initialization experiment would settle this cleanly. If the concern lands, the unintentional backdoor claim should be substantially softened, but the primary detection contribution and the filtering defense are independent and remain supported. Hence the overall verdict stays conditional: the paper is acceptable in principle but needs this additional control before the unintentional-backdoor finding can be taken at face value.","tokens_in":32795,"tokens_out":8323,"duration_ms":76114,"concrete_test":"Re-run the trigger-recovery procedure of Section 5.3 with the same birthday cake target caption but initialize Δ from random Gaussian noise (and also from a random natural image) on both the authors' CC3M-trained RN50 and the OpenCLIP RN50. Additionally, run a positive control: recover a trigger for a randomly chosen target class (e.g., “great white shark”) initialized with that class's image. If the birthday cake ASR under random initialization drops to the 13–45% range seen in the shark control, the original 92–99% ASR is attributable to initialization bias. If ASR remains above ~80% with random initialization, the unintentional backdoor claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's secondary claim (Section 5.3) that a natural backdoor exists in CC3M and persists in an OpenCLIP model rests on a trigger-recovery optimization (Appendix B.4, Eq. 2) where the mask and pattern are initialized with the birthday cake image (“Δ is initialized using a birthday cake example”). The recovered trigger achieves 92–99% ASR, but this could largely be an artifact of optimization starting near the target image and refining it into a universal adversarial patch rather than evidence of a learned backdoor. The control in Appendix B.4 (Figure 7) uses a different target (“great white shark”) with random initialization, so it does not isolate initialization from target choice. If the high ASR comes from initialization rather than a data-driven backdoor, the paper's conclusion about unintentional backdoors is unsupported. This is load-bearing because the abstract, introduction, and conclusion highlight the unintentional backdoor as a key finding that motivates data purification; a false claim here would mislead practitioners about real-world web-scale risks, even though the core detection method remains intact.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies detection of backdoor-poisoned samples in CLIP pretraining. The authors observe that poisoned samples tend to have sparser local neighborhoods in the learned representation space than clean samples, and they show that classical local outlier detectors (k-dist, SLOF, DAO) achieve high detection AUC across a range of trigger types, poisoning rates, architectures, and datasets. They further demonstrate that filtering out detected outliers and retraining mitigates backdoor attacks, and they report the discovery of an unintentional backdoor in the CC3M dataset that appears to have transferred to an OpenCLIP model. The paper includes extensive experiments, a sensitivity analysis, and public code.","tokens_in":32977,"tokens_out":7611,"duration_ms":68183,"significance":"If the main detection claim is accepted, this is a practically valuable result: it provides a simple, scalable, and unsupervised defense against poisoning backdoor attacks on CLIP, at a time when web-scale pretraining is increasingly common. The paper's broad empirical evaluation, including multiple triggers, two architectures, and two additional datasets, is a strength, and the authors are transparent about computational cost and reproducibility limitations. The unintentional backdoor claim is potentially important for data curation practice, but it is also the most fragile part of the paper because the trigger-recovery evidence is not conclusively controlled.","major_comments":[{"comment":"The trigger recovery for the birthday cake example initializes the pattern Delta using a birthday cake image, while the control experiment in Figure 7 uses a different target (\"great white shark\") with random initialization. This changes both the target and the initialization, so the high ASR in the birthday cake case could be due to initialization bias rather than a genuinely learned backdoor. The authors should add a same-target random-init control, and ideally also a control on a model trained on data without the repeated birthday cake images; without such controls, the unintentional backdoor claim in Section 5.3 and the abstract is not supported.","section":"Section 5.3 / Appendix B.4 / Eq. (2)"},{"comment":"The reported ASR is only 19.0% and the best AUC among k-dist, SLOF, and DAO is about 71.5%. This contradicts the abstract's unqualified claim that backdoor attacks can be \"easily and efficiently detected\" and shows that the detection is not universally reliable. The paper should qualify the central claim to reflect the cases where detection is strong, and explain how a defender should handle settings like this one (for example, by using RN50 as the detector as suggested in Section 5.1).","section":"Table 1, ViT-B-16 Clean Label row"},{"comment":"The probabilistic argument that, at a 0.01% poisoning rate, the k-th nearest neighbor of a poisoned sample is likely clean because the probability that the rest of the batch is clean is 0.9999^1023 is not rigorous: nearest neighbors are not uniformly random draws from the batch, and if poisoned samples form a tight cluster they can be each other's nearest neighbors even at low poisoning rates. The empirical evidence in Figure 1b is more persuasive than this heuristic; the text should be revised to avoid presenting the heuristic as a probabilistic guarantee.","section":"Section 4.2"}],"minor_comments":[{"comment":"All experiments appear to be single runs without error bars or multiple seeds; reporting AUC to two decimal places in the absence of variance estimates may overstate precision. Adding a note about the number of runs, or reporting standard deviations for a few key configurations, would improve the paper.","section":"Section 5 / Table 1"},{"comment":"The claim that a million-scale dataset can be cleaned \"within 15 minutes using 4 Nvidia A100 GPUs\" refers only to the detection step; the time required to pretrain the model used for representation extraction is not included. This should be stated explicitly wherever the timing claim appears.","section":"Abstract / Section 6"},{"comment":"Several ViT-B-16 attacks (WaNet, BLTO, MT-S) have ASR values below 30%; the paper should state whether these are considered successful attacks and how the detection AUC should be interpreted when the attack itself is weak.","section":"Table 1, ViT-B-16 rows"},{"comment":"The caption reads \"The CLIP learned presentations\" and should read \"representations\"; also, the phrase \"k-th nearest neighbors\" is used inconsistently and should be singular or plural consistently.","section":"Figure 1(a) caption"},{"comment":"The total variation terms TV(m) and TV(Delta) are not defined in the main text; please define the total variation operator and state the dimensions of m and Delta explicitly.","section":"Appendix B.4 / Eq. (2)"},{"comment":"The paper reports that the suspicious birthday cake images appear 798 times but does not specify the DAO score threshold or the ranking criterion used to identify them; providing this information would improve reproducibility.","section":"Section 5.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong empirical study with a valuable reproducibility statement and code release. The central detection result is broadly sound, but the unintentional backdoor claim is under-supported because the trigger-recovery control in Appendix B.4 changes both target and initialization. If a same-target random-init control fails to reproduce the high ASR, the authors should substantially weaken the unintentional backdoor claim; the detection method itself would remain valid. I recommend major revision to require the additional control and to qualify the overbroad abstract claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper deserves a serious referee. The main contribution is the characterization of CLIP backdoor samples as sparse local outliers and the demonstration that off-the-shelf local outlier detectors (k-dist, SLOF, DAO) catch them with near-perfect AUC across a wide range of triggers and architectures, while dedicated supervised backdoor detectors fail. That is a genuinely useful result for a realistic threat, and it is not circular: the detectors are unsupervised, no parameters are fitted to the target outcome, and the k-sensitivity analysis shows the choice of k is not fragile.\n\nThe evidence is broad: RN50 and ViT-B-16, CC3M, CC12M, RedCaps, many trigger types, multi-trigger settings, TDPA, and a poisoning-rate sensitivity study. The time cost (15 minutes on 4 A100s for CC3M) makes the approach practical. The defense experiment, where removing 10% of the data by DAO score drops ASR to near zero, is also convincing.\n\nSoft spots, in proportion: First, there are no error bars or multiple seeds, and the ViT-B-16 clean-label case sits at only ~71% AUC. The claim \"near-perfect\" is too strong in that corner; the reader should know that the clean-label setting is the hard one. Second, the unintentional backdoor finding in CC3M rests on trigger recovery initialized with the birthday cake image (Appendix B.4). The counter-example in Figure 7 uses random initialization but a different target (\"great white shark\"), so it does not isolate initialization from target choice. A same-target, random-init control is needed before the \"natural backdoor\" claim is solid. The authors themselves flag the resemblance to a targeted universal adversarial attack, but the control they provide does not close that gap. The core detection method does not depend on this secondary claim, so the paper's main value stands regardless.\n\nOverall: the detection result is a strong empirical contribution with public code, and it deserves a serious referee. The unintentional backdoor claim is interesting but under-supported as it stands; a revision with a proper control and variance estimates would firm it up. I would bring this to a reading group and would cite the detection result in my own work.","headline":"Solid detection result worth serious refereeing; the CC3M 'unintentional backdoor' claim needs a same-target random-init control before it's convincing.","tokens_in":33508,"tokens_out":1829,"would_cite":true,"duration_ms":17281,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Backdoor-poisoned images in CLIP leave a geometric trace: sparse local neighborhoods that classic density-ratio outlier detectors can find.","keywords":["backdoor detection","CLIP","contrastive language-image pretraining","local outlier detection","data poisoning","density-ratio","SLOF","DAO"],"falsifier":"Run the k-dist, SLOF, and DAO detectors on a CLIP model poisoned with a scheme that gives each poisoned image its own unique trigger pattern (so the model cannot cluster them), and measure the detection AUC; if the AUC remains near-perfect, the sparse-neighborhood claim holds for broader trigger families, but if it collapses, the claim is specific to clustered trigger patterns and fails under per-image trigger diversity.","tokens_in":32593,"feed_emoji":"🔍","tokens_out":3974,"duration_ms":37137,"temperature":0.7,"pith_summary":"Backdoor attacks on CLIP are dangerously cheap, succeeding with only 0.01% poisoned training samples. This paper claims that those poisoned samples inevitably leave a geometric trace: in the representation space learned by the poisoned CLIP model, every trigger-carrying image lands in a low-density pocket, surrounded mostly by clean images. That makes them detectable as local outliers by classic density-ratio methods such as distance to the k-th nearest neighbor, SLOF, and DAO, which reach 95–99.9% AUC across patch, blend, filter, and multi-trigger attacks. The paper also argues that this detector scales to million-scale web datasets, cleaning CC3M in about 15 minutes on four A100 GPUs, and that it exposes an unintentional backdoor already trained into a released OpenCLIP model.","feed_headline":"Sparse neighborhoods catch CLIP backdoor images","feed_subtitle":"Poisoned CLIP images sit in low-density pockets, so simple density-ratio outlier scores flag them at 95–99.9% AUC.","key_machinery":"The carrying object is the local density ratio in the CLIP representation space, realized through three classic outlier scores: k-dist (distance to the k-th nearest neighbor), SLOF (simplified local outlier factor, the average ratio between a point's k-dist and its neighbors' k-dist), and DAO (dimensionality-aware outlier detection, which raises each neighbor's distance ratio to an exponent derived from local intrinsic dimensionality). These scores measure whether a point's local neighborhood is abnormally sparse relative to its neighbors; the paper's claim is that backdoor samples occupy such sparse pockets in the learned embedding, so the scores separate them from clean samples. The detector itself is a minibatch procedure: extract image and text embeddings from a trained CLIP model, compute the score of each point against a random reference batch, and rank the dataset by the score.","core_discovery":"The central claim is that backdoor-poisoned samples in CLIP have a much sparser local neighborhood than clean samples, making them outliers in the learned representation space. This sparsity is a direct consequence of the contrastive objective: all images carrying the same trigger share a strong common feature, so the model pushes them into a tight cluster separated from ordinary data, and at the low poisoning rates used in practice a poisoned image's k-th nearest neighbor is almost always a clean image, inflating its k-dist and local density ratio. Building on this, the paper shows that classic density-ratio local outlier detectors (SLOF and DAO) and simple k-dist scores detect poisoned samples nearly perfectly, while dedicated supervised backdoor detectors (ABL, CD) and SafeCLIP often fail. A secondary claim is that this detector, applied to the clean CC3M dataset, reveals an unintentional backdoor encoded in a birthday-cake image that appears 798 times, and that a recovered trigger from a model trained on CC3M or released by OpenCLIP achieves 92.38% and 98.92% attack success rates respectively, confirming that such triggers have already been learned into widely used models.","pith_inferences":["The sparsity signature is a property of the learned representation, not the trigger's pixel pattern; this suggests the same detection logic could transfer to other contrastive multimodal models (e.g., image-text or video-text) that cluster poisoned samples tightly by a shared shortcut feature.","The claim implicitly predicts that an attacker who wants to evade k-dist/SLOF/DAO must break the cluster geometry, for example by giving each poisoned image its own unique trigger or by training the model to match clean density exactly; the paper's adaptive-attack experiment accounts for only a narrow form of such evasion.","A testable extension is to vary the poisoning rate upward and track when the k-nearest-neighbor assumption fails; the paper shows performance degrades at 10% poisoning but recovers with larger k, suggesting that detection can be extended to high-poisoning regimes by scaling locality.","The unintentional-backdoor finding implies that web-scale datasets may contain many such latent triggers; a full audit would rank all recurring image-caption pairs by outlier score and reverse-engineer each, which could be run as a dataset-level scan before release."],"forward_implications":["Data owners can purify a million-scale CLIP pretraining dataset by removing top-scoring outliers, dropping the attack success rate to near zero while preserving clean accuracy, since most high-scoring samples are either poisoned or already noisy.","Dedicated supervised backdoor detectors are not needed for CLIP; classical unsupervised outlier detection is sufficient, which simplifies the defense toolkit and makes detection feasible at web scale.","The same detector can serve as a data-cleaning tool for unpoisoned web datasets, removing broken images and mismatched captions that degrade CLIP performance.","Detecting and removing poisoned samples is, under the paper's cited theoretical equivalence, a form of robust training, so combining outlier filtering with robust objectives gives defenders a two-layer defense.","Unintentional backdoors in web datasets are not hypothetical: this method found one already trained into a released OpenCLIP model, so scanning datasets before pretraining becomes a practical necessity."],"supporting_citations":[{"why":"Establishes the poisoning backdoor attack setup on CLIP and the 0.01% poisoning rate that this paper defends against.","marker":"Carlini & Terzis (2022)"},{"why":"Defines SLOF, the simplified local outlier factor that serves as the paper's primary density-ratio detector.","marker":"Schubert et al. (2014)"},{"why":"Defines DAO, the dimensionality-aware outlier detection that improves SLOF by incorporating local intrinsic dimensionality.","marker":"Anderberg et al. (2024)"},{"why":"Introduces LOF, the classic local outlier factor concept that SLOF and DAO are built upon.","marker":"Breunig et al. (2000)"},{"why":"Neural Cleanse trigger recovery method, adapted by the paper to synthesize the birthday-cake trigger and measure its attack success rate.","marker":"Wang et al. (2019)"},{"why":"Provides the CC3M web dataset in which the unintentional backdoor was discovered.","marker":"Sharma et al. (2018)"},{"why":"The OpenCLIP released model that the paper shows had already learned the unintentional birthday-cake trigger.","marker":"Ilharco et al. (2021)"},{"why":"Supplies the theoretical result that detecting and removing poisoned data is equivalent to robust training, justifying the filtering-based defense.","marker":"Manoj & Blum (2021)"}],"fun_headline_variants":["Sparse neighborhoods unmask CLIP backdoors","Density ratio outs CLIP's hidden backdoor samples","Poisoned CLIP images stand out in sparse local space","Simple outlier score exposes CLIP backdoors","One-shot detection: sparse neighborhoods flag CLIP backdoors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The detector relies on the geometric premise that a CLIP model trained on poisoned data pushes all trigger-carrying images into a tight, isolated cluster, and that at low poisoning rates the k nearest neighbors of a poisoned sample are almost always clean samples; if a trigger is subtle, the model does not overfit to it, or the poisoning rate is high enough that poisoned samples become each other's neighbors, the outlier signal weakens or disappears.","fun_headline_variants_meta":{"raw":{"variants":["Sparse neighborhoods unmask CLIP backdoors","Density ratio outs CLIP's hidden backdoor samples","Poisoned CLIP images stand out in sparse local space","Simple outlier score exposes CLIP backdoors","One-shot detection: sparse neighborhoods flag CLIP backdoors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000864,"raw_usage":{"total_tokens":3792,"prompt_tokens":1038,"completion_tokens":2754,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":2677}},"tokens_in":654,"tokens_out":2754,"duration_ms":16953,"temperature":1.0,"reasoning_tokens":2677,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:30:03.837099+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the k-dist, SLOF, and DAO detectors on a CLIP model poisoned with a scheme that gives each poisoned image its own unique trigger pattern (so the model cannot cluster them), and measure the detection AUC; if the AUC remains near-perfect, the sparse-neighborhood claim holds for broader trigger families, but if it collapses, the claim is specific to clustered trigger patterns and fails under per-image trigger diversity.","supporting_citations":[{"cited_title":"Local outlier detection reconsidered: a generalized view on locality with applications to spatial, video, and network outlier detection","cited_arxiv_id":null,"evidence_quote":"Defines SLOF, the simplified local outlier factor that serves as the paper's primary density-ratio detector."},{"cited_title":"Openclip, 2021","cited_arxiv_id":null,"evidence_quote":"The OpenCLIP released model that the paper shows had already learned the unintentional birthday-cake trigger."},{"cited_title":"Excess capacity and backdoor poisoning","cited_arxiv_id":null,"evidence_quote":"Supplies the theoretical result that detecting and removing poisoned data is equivalent to robust training, justifying the filtering-based defense."}],"review_version":1}