{"id":"27dff97a-4bb1-4ee0-91ce-2940e1851693","arxiv_id":"2411.16154","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DeDe detects backdoor-triggered inputs in SSL encoders by training a decoder that reconstructs images from embeddings and flagging samples with high reconstruction error.","lead":"This paper proposes DeDe, a method that trains a decoder to reconstruct images from SSL encoder embeddings, then flags inputs whose reconstruction differs from the original as backdoor samples. It reports high detection rates on several contrastive learning and CLIP backdoor attacks, with some reliance on per-attack tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DeDe's reconstruction-failure assumption is attack-dependent: the CTRL special-casing shows the mechanism does not follow from the backdoor objective alone, and failure on visually similar source/target classes would expose a central gap.","rationale":"The reader identifies the reconstruction-failure hypothesis as the load-bearing assumption; my analysis agrees and sharpens it. DeDe's empirical success on BadEncoder, DRUPE, and CLIP-Backdoor does not yet justify the paper's broad claim that triggered embeddings 'always' decode to inconsistent images. The clearest internal evidence is the special handling required for CTRL in Section 5.1: the method had to be augmented with noise-filling of OOD embedding regions and a lower mask ratio, both of which presume knowledge of the attack's embedding geometry. That is a direct admission that the core assumption can fail, and the paper offers no principled rule for when such fixes are needed. The proposed concrete test — a visually similar source/target pair with an auxiliary set missing the target class — isolates the semantic-mismatch condition that the paper's intuition depends on. If DeDe still separates clean and triggered inputs in that setting, the concern is substantially mitigated; if not, the central claim must be conditioned on class-pair and auxiliary-dataset coverage. Since the reader's verdict is already CONDITIONAL and this concern strengthens rather than overturns that conditionality, I recommend leaving the verdict unchanged.","tokens_in":15574,"tokens_out":9445,"duration_ms":97727,"concrete_test":"Run the released DeDe code on a CIFAR-10 SimCLR encoder backdoored with BadEncoder or DRUPE using a visually similar source/target pair (e.g., cat→dog), and train the DeDe decoder on an STL-10 auxiliary set with all dog images removed; measure AUC, TPR/FPR at the default 1.5x threshold, and downstream ASR after filtering. If AUC remains above 0.95 and filtered ASR below ~10%, the reconstruction-failure assumption is robust; if AUC drops below ~0.85 or filtered ASR exceeds ~30%, the universality claim fails and the verdict should stay conditional with explicit caveats.","verdict_should_be":"UNCHANGED","load_bearing_attack":"DeDe's decision rule (Eq. 2, Algorithm 1) assumes that for any triggered input x* = x_s + t mapped to a target embedding e*, the decoder h_d(e*, h_e(M_alpha x*)) will reconstruct an image far from x*, while clean inputs reconstruct well. The paper justifies this with an intuitive 'image space is sparse' argument (Section 4.2), but provides no characterization of when the conflict holds. The CTRL handling in Section 5.1 is direct evidence that the mechanism is not universal: because CTRL embeddings fall in an OOD region, the authors had to fill that region with random noise embeddings and reduce the training mask ratio from 0.9 to 0.75, i.e., they needed attack-specific knowledge to make the assumption work. More generally, the failure condition is not 'embedding is OOD' — DeDe also detects DRUPE, whose embeddings are designed to stay in-manifold. The real requirement is that the decoder's pre-image of the target embedding region be visually inconsistent with the source image. That requirement can fail in a plausible, non-contrived threat model: if an attacker chooses a target class visually similar to the source class (e.g., cat→dog on CIFAR-10), the decoder will reconstruct a dog-like image from the target embedding, and with only 1% visible patches (α=0.99) the reconstruction may remain close enough to the cat input to fall below the 1.5x training-loss threshold, while the downstream linear probe still assigns the embedding to the target class. The claim that DeDe works with 'any publicly available dataset' is likewise only tested with STL-10, which shares most classes with CIFAR-10 and is a subset of ImageNet; an auxiliary set lacking the target class would make clean target-class reconstructions error-prone and could destroy the separation. Thus the central claim — that DeDe detects backdoor activation without knowing the attack type — is not established for visually similar class pairs or semantically remote auxiliary data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DeDe, an inference-time detector for backdoor samples in self-supervised learning (SSL) encoders. DeDe trains an auxiliary decoder (plus a patch encoder) to reconstruct images from the victim encoder's embeddings, using a training set that can be unlabeled, out-of-distribution, or slightly poisoned. At test time, a sample is flagged as backdoor if its reconstruction error exceeds a threshold set to 1.5 times the average training loss (Algorithm 1, Eq. (2)). The authors evaluate DeDe on contrastive learning (SimCLR) and CLIP encoders against five attacks (BadEncoder, CTRL, DRUPE, CLIP-Backdoor, BadCLIP) and compare it with DECREE and ASSET. The main reported results show high AUC (mostly above 0.9) across attacks and datasets, with better downstream attack-success-rate reduction than ASSET. The paper also reports additional results for unbalanced data and OOD auxiliary datasets in the appendix.","tokens_in":15938,"tokens_out":3398,"duration_ms":34240,"significance":"If the claimed detection performance is robust, DeDe would be a practically useful defense: it operates at inference time, does not require labels, and does not modify the victim encoder. The paper provides a clear algorithm, public code, and extensive experiments on multiple attack families, which are strengths. However, the central claim of universality—that DeDe detects backdoor activation without any attack-specific knowledge—is weakened by the special handling required for the CTRL attack (Section 5.1) and by the absence of a characterization of when the reconstruction-failure assumption holds. The lack of error bars and the modified DECREE baseline also make the quantitative comparisons less conclusive. The idea itself is novel and worth publishing, but the current empirical and theoretical support is not yet sufficient to substantiate the strongest claims in the paper.","major_comments":[{"comment":"The CTRL attack requires two attack-specific adjustments: filling out-of-distribution embedding regions with random noise image embeddings and reducing the training masking ratio from 0.9 to 0.75. This directly contradicts the contribution stated in Section 1 that DeDe \"requires no prior information regarding the victim encoder or the type of backdoor trigger.\" If the defender must know that the embedding distribution is sparse and OOD (and must know a norm-ball radius to fill), then the method is not agnostic to the attack. Please either remove the universality claim or provide a principled, attack-agnostic way to choose these hyperparameters, with ablation results showing the performance of DeDe under default settings on CTRL.","section":"Section 5.1, \"Auxiliary Explanation\""},{"comment":"The detection threshold is set as a fixed multiplier of the average training reconstruction loss. The paper reports TPR/FPR at this threshold, but gives no justification for the factor 1.5 and no analysis of how sensitive the reported operating points are to this choice across datasets, attacks, and OOD auxiliary data. Since the threshold is load-bearing for the binary detection decision, please report ROC curves with confidence bands, or provide a sensitivity analysis over a range of threshold multipliers, and clarify whether the TPR/FPR values in Tables 1 and 3 are evaluated at τ = 1.5× or at the default scikit-learn threshold.","section":"Algorithm 1 and Eq. (2), threshold τ = 1.5 × training loss"},{"comment":"The DECREE comparison does not use DECREE's original detection criterion. The paper states that DECREE \"fails to detect any backdoor samples during testing\" and therefore the authors compute the ℓ2 norm between the optimized trigger and the image data within the trigger region, using that norm as the detection score. This is a modified baseline, and it is unclear whether the modification is equivalent to DECREE's intended use or favorable to DeDe. Please report DECREE's original scores and, if a modification is necessary, justify that it is a faithful adaptation. Otherwise the claim of \"superior performance compared over state-of-the-art detection methods\" is not fully supported.","section":"Section 5.1, \"Detection models\" and Table 1"},{"comment":"All experimental results are reported as single numbers without error bars or the number of random seeds. Given that ASSET's best epoch is selected for comparison and that DeDe's hyperparameters (masking ratio, patch size, noise-filling radius) are tuned per attack, the reported margins could be within run-to-run variance. Please report mean and standard deviation over at least three independent runs for the main tables, and state the seed/selection protocol for all baselines.","section":"Tables 1–5 and Appendix B"},{"comment":"The core assumption is that a triggered image mapped to a target-class embedding will be reconstructed by the decoder as something visually inconsistent with the input. The paper gives an intuitive \"image space is sparse\" argument but does not characterize when this holds. In particular, if the source and target classes are visually similar (e.g., cat→dog on CIFAR-10), the decoder may reconstruct a dog-like image from the target embedding, and at a high testing masking ratio (α = 0.99) the reconstruction may remain close enough to the cat input to fall below the threshold. Please add an experiment that varies source/target visual similarity, or otherwise provide evidence that DeDe's separation holds beyond the specific attack configurations tested. Without such evidence, the claim of detecting \"any triggered input\" (Section 1) is overstated.","section":"Section 4.2, Key Idea"}],"minor_comments":[{"comment":"The line \"α ← min(1.1 · α, 1.0)\" appears inside the training loop, but the intended use is to increase the masking ratio only at test time. Please move this update out of the loop or clarify the semantics, and define the exact test-time masking ratio used in Eq. (2) (e.g., α = 0.99 when training uses α = 0.9).","section":"Algorithm 1"},{"comment":"The paper says \"DEDE OoD\" uses STL-10 as a consistent OOD dataset, but the results are mixed: for BadCLIP the OOD AUC is 0.798 in Table 1 vs. 0.925 in-distribution, while for BadEncoder OOD AUC is higher. The text states DeDe maintains high performance with OOD data, but this inconsistency should be discussed or explained.","section":"Table 1 and text"},{"comment":"The claim that the DeDe training dataset \"can be even slightly poisoned\" is not directly tested. Table 1 and Table 3 vary the test-set poisoning rate, but the auxiliary training set is described as clean. Please either add an experiment with a slightly poisoned auxiliary set or temper the claim.","section":"Section 5.1, Datasets"},{"comment":"The caption is confusing: it says the left side and right side are attacked by DRUPE and CLIP-Backdoor respectively, then says the six columns are three clean and three poisoned. Please rewrite to clearly describe which columns correspond to which attack and which are clean/poisoned.","section":"Figure 5 caption"},{"comment":"The appendix states reconstruction results are \"generally stable for different choices of patch size and masking ratio,\" but no quantitative comparison is shown. Please include the ablation table or reference a figure that supports this claim.","section":"Appendix B, Table 6"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the decoder-based detection idea is interesting, with a public repository and broad empirical coverage. My main concern is that the CTRL-specific adjustments and the lack of error bars undermine the universality claim, and the modified DECREE baseline weakens the SOTA comparison. These issues are fixable with additional experiments and a more careful framing; I do not see a fundamental flaw in the core approach, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DeDe is worth a serious look. The core idea—train a decoder to invert the victim encoder's embedding mapping, then flag inputs whose reconstruction error blows up—is new relative to trigger-optimization defenses like DECREE and ASSET. It is simple, non-invasive, and the paper shows strong AUC numbers on BadEncoder, DRUPE, and CLIP-Backdoor, with decent performance on CTRL and BadCLIP. The downstream ASR reductions after filtering are also real. This is the kind of defense that could plausibly be adopted in model distribution pipelines, and the authors provide code.\n\nWhat the paper does well: it clearly explains the threat model and the intuition (backdoor maps triggered inputs to embedding regions whose decoder pre-image is visually inconsistent with the input), and it evaluates on both contrastive and CLIP models. The comparison to ASSET is useful, and the paper is honest enough to include a discussion of why CTRL breaks the clean pattern.\n\nThe soft spots are the usual ones for an empirical defense paper, and they matter here because the authors claim no attack-specific knowledge. First, CTRL required a special noise-filling step and a reduced masking ratio (0.75 instead of 0.9). That is direct evidence that the detector's operating point is attack-dependent. Second, the threshold is a fixed 1.5x training-loss multiplier, and there is no sensitivity analysis across datasets or attacks. Third, there are no error bars; every number in Tables 1 and 3 appears to come from a single run. Fourth, the \"any auxiliary dataset\" claim is tested only with STL-10, which is close to CIFAR-10 in class distribution; semantically remote auxiliaries like SVHN or GTSRB are used only in downstream, not for decoder training. Finally, the paper omits comparisons to SSL-CLEANSE and SSL-ABD, both cited in the related work.\n\nThe stress-test scenario—visually similar source and target classes (e.g., cat to dog)—is a plausible failure mode, but it is not demonstrated. I would treat it as a missing experiment rather than a fatal flaw. The method's assumption is that target embeddings decode to something visually distant from the input; that assumption can fail, and the paper should say so and evaluate it.\n\nBottom line: this is a solid, novel empirical contribution that deserves a serious referee. It needs more runs, a threshold sensitivity study, and a broader auxiliary-set evaluation before I'd fully trust the universality claim, but the central mechanism is sound and the results support it on the tested attacks.","headline":"DeDe's decoder-reconstruction trick is a genuinely new detection idea, and the paper mostly backs it up; the universality claim needs tempering.","tokens_in":16501,"tokens_out":2834,"would_cite":true,"duration_ms":27187,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A backdoored self-supervised encoder can be screened by training a decoder to invert its embeddings: a triggered input reconstructs into an inconsistent image, and the reconstruction error marks it as backdoor.","keywords":["backdoor detection","self-supervised learning","contrastive learning","CLIP","inverse mapping","decoder","reconstruction error","data poisoning"],"falsifier":"Train a backdoor attack whose triggered inputs are mapped into the center of a target class's normal embedding distribution (not a foreign region) and whose trigger remains visible in a few unmasked local patches; if DeDe's reconstruction errors for those inputs fall below its threshold, the clean-versus-triggered separation collapses.","tokens_in":15358,"feed_emoji":"🔍","tokens_out":7861,"duration_ms":67603,"temperature":0.7,"pith_summary":"This paper proposes DeDe, a method for detecting backdoor samples in self-supervised learning (SSL) encoders by training a decoder that maps embeddings back to images. The key claim is that a triggered input, which the backdoor encoder forcibly maps to a target embedding, lands outside the decoder's learned normal region and therefore reconstructs into an image that differs noticeably from the input. The reconstruction error then serves as a detection score, with a threshold set from the average training error. The authors show empirically that this works for contrastive-learning encoders and CLIP vision encoders across several attacks, including stealthy ones, and that it does not require a clean or in-distribution auxiliary dataset.","feed_headline":"A decoder's reconstruction error flags SSL backdoor samples","feed_subtitle":"One unlabeled auxiliary dataset is enough to screen a downloaded SSL encoder for backdoors.","key_machinery":"The load-bearing object is a lightweight masked-autoencoder-style decoder trained to invert the frozen encoder's mapping. It takes a global embedding plus local embeddings from randomly masked patches of the input and reconstructs the image; the training objective is the mean squared reconstruction error over an auxiliary unlabeled dataset. At test time the masking ratio is raised, forcing reconstruction to depend on the global embedding, so a triggered input's foreign embedding produces an abnormally large error, which is compared to a threshold set at 1.5 times the average training loss.","core_discovery":"On its own terms, the paper asserts that the inverse of an encoder's embedding mapping is sufficient to detect backdoor behavior. Because SSL encourages similar images to cluster in embedding space, a decoder trained on clean image-embedding pairs learns a normal cluster-to-image mapping. Backdoor attacks force a triggered input to an embedding that does not match that mapping—either a different class's cluster or an out-of-distribution region—so the decoder's reconstruction of the triggered image is distorted. DeDe operationalizes this with a decoder that takes the global embedding plus a few unmasked local patches, and it flags an input when the squared reconstruction error exceeds 1.5 times the average training loss. The paper's experiments report high detection AUC on BadEncoder, DRUPE, CLIP-Backdoor, CTRL, and BadCLIP, and downstream attack success rates near zero for all but BadCLIP.","pith_inferences":["The authors leave implicit that an attacker who can place triggered embeddings inside the normal manifold—so the decoder's reconstruction looks plausible—would break the method; the noise-filling workaround for CTRL is already a sign that the assumption is strainable.","The same decoder-based inverse mapping could be applied to other self-supervised modalities (audio, text, or graph encoders) whose embeddings are clustered; a decoder trained on normal embeddings would flag anomalous embeddings from poisoned inputs.","DeDe's fixed threshold of 1.5 times the average training loss is a heuristic; a per-task calibration or a learned anomaly score on reconstruction errors would likely improve the true-positive/false-positive trade-off, especially for the harder BadCLIP case."],"forward_implications":["A user who downloads a pre-trained SSL encoder can screen incoming inputs for backdoor activation using only an auxiliary unlabeled dataset, with no knowledge of the trigger pattern or the training data.","Because DeDe never modifies the encoder, the same trained decoder can be reused as an inference-time alert system on top of a deployed model.","On the evaluated attacks, DeDe brings downstream attack success rates near zero for BadEncoder, CTRL, DRUPE, and CLIP-Backdoor, while leaving clean accuracy largely intact.","DeDe retains high detection accuracy when the auxiliary dataset is out-of-distribution (e.g., STL-10 for encoders trained on CIFAR-10) and can tolerate slight poisoning of that dataset.","Existing detection baselines either require a clean base dataset (ASSET) or only handle patch-like triggers (DECREE); DeDe's reconstruction-based score avoids both constraints."],"supporting_citations":[{"why":"Supplies the masked autoencoder reconstruction architecture and objective that DeDe adapts to train its decoder.","marker":"[19]"},{"why":"Defines the embedding-level backdoor attack on SSL encoders that establishes DeDe's threat model.","marker":"[23]"},{"why":"Introduces the stealthy embedding-preserving DRUPE attack that DeDe must detect and that pushes the method toward reconstruction-based errors.","marker":"[36]"},{"why":"DECREE, the first SSL backdoor detection baseline, whose patch-trigger-only limitation DeDe is designed to overcome.","marker":"[13]"},{"why":"ASSET, the state-of-the-art detection baseline that DeDe compares against and that requires a clean base dataset.","marker":"[30]"},{"why":"CTRL, an end-to-end backdoor attack whose out-of-distribution embeddings forced DeDe's noise-filling workaround.","marker":"[25]"},{"why":"The poisoning-and-backdooring attack on CLIP that DeDe detects with near-perfect AUC.","marker":"[4]"},{"why":"BadCLIP, the adaptive-trigger CLIP attack that remains the hardest case for DeDe's detection.","marker":"[27]"}],"fun_headline_variants":["Decoder mismatch sniffs out SSL backdoor triggers","Detect SSL backdoors by reconstructing inputs through a decoder","SSL backdoor detection via decoder reconstruction error","A decoder's eye: spotting hidden backdoors in SSL encoders","Backdoor SSL? Decoder output gives it away"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method works only if backdoor mappings land on embeddings the decoder has not learned to invert, so that triggered images reconstruct poorly while clean images reconstruct well.","fun_headline_variants_meta":{"raw":{"variants":["Decoder mismatch sniffs out SSL backdoor triggers","Detect SSL backdoors by reconstructing inputs through a decoder","SSL backdoor detection via decoder reconstruction error","A decoder's eye: spotting hidden backdoors in SSL encoders","Backdoor SSL? Decoder output gives it away"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1398,"prompt_tokens":979,"completion_tokens":419,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":341}},"tokens_in":595,"tokens_out":419,"duration_ms":4769,"temperature":1.0,"reasoning_tokens":341,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:28:44.560260+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a backdoor attack whose triggered inputs are mapped into the center of a target class's normal embedding distribution (not a foreign region) and whose trigger remains visible in a few unmasked local patches; if DeDe's reconstruction errors for those inputs fall below its threshold, the clean-versus-triggered separation collapses.","supporting_citations":[{"cited_title":"Masked autoencoders are scalable vision learners","cited_arxiv_id":null,"evidence_quote":"Supplies the masked autoencoder reconstruction architecture and objective that DeDe adapts to train its decoder."},{"cited_title":"Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning","cited_arxiv_id":null,"evidence_quote":"Defines the embedding-level backdoor attack on SSL encoders that establishes DeDe's threat model."},{"cited_title":"Dis- tribution preserving backdoor attack in self-supervised learning","cited_arxiv_id":null,"evidence_quote":"Introduces the stealthy embedding-preserving DRUPE attack that DeDe must detect and that pushes the method toward reconstruction-based errors."},{"cited_title":"Detecting back- doors in pre-trained encoders","cited_arxiv_id":null,"evidence_quote":"DECREE, the first SSL backdoor detection baseline, whose patch-trigger-only limitation DeDe is designed to overcome."},{"cited_title":"{ASSET}: Robust backdoor data detection across a multiplicity of deep learning paradigms","cited_arxiv_id":null,"evidence_quote":"ASSET, the state-of-the-art detection baseline that DeDe compares against and that requires a clean base dataset."},{"cited_title":"An embar- rassingly simple backdoor attack on self-supervised learning","cited_arxiv_id":null,"evidence_quote":"CTRL, an end-to-end backdoor attack whose out-of-distribution embeddings forced DeDe's noise-filling workaround."},{"cited_title":"Badclip: Dual- embedding guided backdoor attack on multimodal contrastive learning","cited_arxiv_id":null,"evidence_quote":"BadCLIP, the adaptive-trigger CLIP attack that remains the hardest case for DeDe's detection."}],"review_version":1}