{"id":"69699761-f0e9-4085-bf5e-55fd434297ad","arxiv_id":"2412.12194","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"BlockDoor wraps a watermarked network with detectors for adversarial, out-of-distribution, and randomly labeled triggers, but its own results show the OOD branch fails on unseen trigger sources and the random-label branch degrades clean accuracy.","lead":"A wrapper framework called BlockDoor claims to block backdoor-based watermarks in neural networks by detecting trigger samples and modifying the inputs before they reach the model. The paper reports large drops in watermark verification accuracy, but the results are inconsistent across trigger types and depend on training data that resembles the secret trigger sets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's functional-preservation claim is contradicted by Table 3: random-label wrappers lower clean accuracy by 14–21 points, not <1%.","rationale":"The reader's verdict of REJECT is well supported, but my most load-bearing concern is not exactly the weakest assumption they identified. The reader focused on the OOD distributional assumption, which is real and well-documented in Section 3.2. I agree that this is a serious limitation. However, an even more direct problem is the internal contradiction between the abstract's '<1% clean accuracy drop' and the random-label results in Table 3, where clean accuracy drops by 14.22–20.88 percentage points. This is not a matter of interpretation or external consensus; it is the paper's own measurements refuting its own headline quantitative claim. The paper also acknowledges this in Section 3.4 by saying the best case for random labels is a 14% range, which is inconsistent with the abstract. Under the reviewing rule that stated limitations must be weighed, Section 3.2's own admission that OOD erasure is 'less impactful' without similar-distribution data, together with Table 2's 83% watermark accuracy when CIFAR-100 is excluded, further undermines the general claim. The adversarial branch is the only one that clearly preserves functionality, so the framework is at best a partial proof-of-concept, not the comprehensive watermark-blocking package described in the title and abstract. I therefore agree with REJECT, with partial agreement because I locate the decisive failure in the random-label clean-accuracy contradiction rather than in the OOD distribution assumption alone.","tokens_in":10057,"tokens_out":1429,"duration_ms":14615,"concrete_test":"Recompute the clean-accuracy deltas from Table 3 using the reported exact values: (85.33 - 64.45) = 20.88 points for the CIFAR-10 random-label wrapper and (85.33 - 71.11) = 14.22 points for the CINIC-10 wrapper. If these deltas exceed the 1% bound claimed in the abstract, then the functionality-preservation component of the central claim fails. Additionally, rerun the OOD experiment of Table 2 with CIFAR-100 excluded from training negatives and report the resulting watermark accuracy; if it remains near 83%, the distributional-assumption caveat in Section 3.2 is confirmed as a hard limitation rather than a minor future-work item.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim, repeated in the abstract and contributions, is that BlockDoor reduces trigger-set watermark accuracy by up to 98% 'without compromising on functionality, delivering up to a less than 1% drop on the clean samples.' The paper's own Table 3 directly contradicts this. For random-label triggers, the original watermarked model achieves 85.33% test accuracy, while the CIFAR-10 wrapper falls to 64.45% (a 20.88-point drop) and the CINIC-10 wrapper falls to 71.11% (a 14.22-point drop). Section 3.4 even concedes the best case is a 14% range for random-label samples while the abstract promises <1%. This is not a subtle approximation or a boundary case; it is the flagship functionality-preservation property failing on one of the three trigger types the framework claims to handle. The OOD branch is also internally weak: when CIFAR-100 is excluded from the negative training data, watermark accuracy remains at 83%, so the 'erasure' largely fails unless the adversary has access to data from the trigger distribution. The headline result therefore rests on a cherry-picked subset of experiments: adversarial triggers work, diluted-OOD triggers work only under favorable distributional assumptions, and random-label triggers succeed at breaking the watermark only by severely degrading the model's utility. Because the abstract's quantitative promise is falsified by the paper's own reported numbers, the central argument as stated is not supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BlockDoor, a wrapper-based attack against backdoor watermarking in deep neural networks. The wrapper is designed to detect and neutralize three families of trigger samples used in backdoor watermarks: adversarial-noise triggers, out-of-distribution (OOD) triggers, and random-label triggers. The authors claim that BlockDoor reduces watermark verification accuracy by up to 98% while preserving clean accuracy to within less than 1%. Results are reported for CIFAR-10 and related datasets across several architectures, using watermarked models on which trigger sets were embedded. The paper also includes background on watermarking, a threat model, and appendix results for the three wrapper components.","tokens_in":10296,"tokens_out":5383,"duration_ms":50098,"significance":"If the central claim held, the paper would demonstrate a practical, model-agnostic way to defeat backdoor-based watermark verification without modifying the target model, which would be of clear interest to the security community. The work has some positive aspects: it targets three distinct trigger-generation families, uses transparent assumptions about watermarking schemes, and the adversarial-trigger wrapper appears to work reasonably in the presented single experiment. However, the significance is severely limited because the flagship functional-preservation claim is contradicted by the paper's own reported results, the OOD branch is explicitly dependent on access to the trigger distribution, and the reported test-accuracy numbers are not clearly end-to-end measures of the wrapped model's clean accuracy. The paper is an early work-in-progress, and the current evidence does not support the abstract's quantitative promises.","major_comments":[{"comment":"The abstract's claim of \"less than 1% drop on the clean samples\" is contradicted by Table 3. For random-label triggers, the original watermarked model has 85.33% test accuracy, while the BlockDoor wrapper on CIFAR-10 achieves 64.45% (a 20.88-point drop) and on CINIC-10 71.11% (a 14.22-point drop). Section 3.4 itself concedes a best case of 14% degradation for random-label triggers. This is not a minor inconsistency: the functional-preservation property is one of the paper's central advertised contributions and fails for one of the three trigger families the framework claims to handle.","section":"Abstract and Section 3.3 / Table 3"},{"comment":"The OOD branch only succeeds when the trigger distribution is included in the wrapper's training data. Table 2 shows watermark accuracy drops to 12% when CIFAR-100 is diluted into the negative training set, but only to 83% when CIFAR-100 is excluded. The text in Section 3.2 explicitly states that OOD data with a distribution similar to the watermark data is required, \"else the erasure is less impactful.\" This contradicts the threat model in Section 2.1, where the adversary has access to the model's dataset but not to the secret trigger set. The attack therefore does not generalize to unseen OOD triggers, and the strongest OOD result is obtained under a distributional assumption that is not justified.","section":"Section 3.2 / Table 2"},{"comment":"The \"Test Accuracy (%)\" column in the results tables is ambiguous: it is not clear whether this is the end-to-end clean accuracy of the full pipeline (wrapper plus watermarked model) or the accuracy of the auxiliary/secondary classifier used by the wrapper. For the random-label wrapper, Table 3 reports 64.45% and 71.11% for the wrapper models, which strongly suggests these are the secondary models' own classification accuracies rather than the clean accuracy of the watermarked model after wrapping. The functional-preservation claim requires reporting the clean samples' accuracy through the complete BlockDoor pipeline, along with the misrouting rate of clean samples. Without this, the claimed functionality preservation is not established.","section":"Tables 1-3 and Section 2.2-2.4"},{"comment":"The key-findings bullets are internally inconsistent. The first bullet states that BlockDoor \"satisfies the functionality preserving property\" and the second claims clean accuracy is preserved, but the subsequent bullets report a best-case degradation of 6% for OOD triggers and 14% for random-label triggers. These ranges are an order of magnitude larger than the \"less than 1%\" promised in the abstract. The paper should reconcile these statements and either re-scope the contribution or substantially improve the wrappers' clean-accuracy preservation.","section":"Section 3.4"}],"minor_comments":[{"comment":"The text says \"Certified watermarking is employed to embed the adversarial samples as the trigger data,\" but the described trigger generation uses FGSM. Certified watermarking via randomized smoothing is a different mechanism (reference [1]); the relation between the two should be clarified.","section":"Section 3.1"},{"comment":"The threat-model paragraph contains stray formatting and incomplete sentences, e.g., \"This wrapper solution circumvents the modification of the watermarked model since we encase it in a wrapper that does not directly modify the watermark model. Preserving inherent information of the watermark model.\" This should be rewritten for clarity.","section":"Section 2.1"},{"comment":"The figure descriptions (e.g., \"The figure below showcases the classification F1 score\") refer to figures that are not included in the text; the paper should either include the figures or remove the references.","section":"Appendix 6.2"},{"comment":"The PCA component columns in Table 6 are labeled with variance thresholds (0.95, 0.90, 0.85), but the text and table header call them \"n components\"; this notation is confusing and should be made consistent.","section":"Appendix 6.3"},{"comment":"The manuscript contains many typographical artifacts, such as \"T ransparency\", \"W atermark\", and \"T raining\". A full proofread is needed before resubmission.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"This is an early work-in-progress draft, not yet at the standard of a peer-reviewed paper. The central quantitative claim is falsified by the paper's own Table 3, and the OOD branch's own stated limitation undermines the generality of the attack. The ambiguous test-accuracy metric in the tables makes the evaluation hard to interpret. The authors may be able to produce a publishable paper by scoping the contribution honestly (e.g., adversarial triggers only, or with explicit trigger-distribution access), reporting end-to-end clean accuracy, and adding statistical significance. But in its current form, the manuscript is not defensible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is my read of BlockDoor. The wrapper idea is genuinely a bit new: instead of trying to remove a backdoor from the weights, it sits in front of the model and tries to detect and neutralize trigger samples before verification. Combining three detectors——adversarial noise, OOD, and random-label——into one framework is a reasonable way to cover the backdoor-watermark design space, and the experiments span multiple architectures and datasets. Credit also for the threat model being explicit about Kerckhoffs's principle and the adversary's data access.\n\nThat said, the paper's central quantitative promise is not supported by its own numbers. The abstract and contributions promise up to a 98% reduction in trigger-set accuracy with less than a 1% drop on clean samples. Table 3 shows that the random-label wrapper lowers CIFAR-10 test accuracy from 85.33% to 64.45% (a 20.88-point drop) and CINIC-10 from 85.33% to 71.11% (a 14.22-point drop). The paper's own Key Findings section concedes a best-case 14% range for this trigger type. That is a load-bearing inconsistency, not a rounding error.\n\nThe OOD branch has a weaker but real problem: when CIFAR-100 is excluded from the negative training set, watermark accuracy stays at 83%, which is close to the original 100% unless the adversary has access to trigger-like data. The authors do flag this in Section 3.2 as future work, but it means the headline \"Blocks all three trigger types\" is true only under favorable distribution assumptions.\n\nOther soft spots: no comparison to existing backdoor defenses like DeepInspect or TABOR, no error bars on the watermark-accuracy numbers, and no code release. The adversarial branch is only demonstrated against FGSM, so its generality is untested. The free parameters (OOD negative set, adversarial training fraction, PCA threshold) are tuned but not swept in a way that shows robustness.\n\nI would not desk reject this. The idea deserves referee time, but the authors need to either water down the abstract to match the actual results (e.g., \"up to 98% for random-label triggers, with a cost of 14–20 points of clean accuracy\") or add an explicit utility-preserving mechanism for that branch. As a work-in-progress, it is a decent proof-of-concept, not the comprehensive watermark-breaking package the abstract promises.","headline":"Useful wrapper-based attack idea undercut by an abstract that overclaims functional preservation: random-label wrappers cost 14–21 points of clean accuracy, and OOD blocking needs trigger-like data.","tokens_in":10878,"tokens_out":2464,"would_cite":false,"duration_ms":22582,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BlockDoor claims that backdoor-based neural network watermarks can be defeated by a wrapper that detects and rewrites trigger samples before they reach the model.","keywords":["neural network watermarking","backdooring","model modification attack","wrapper defense","trigger set","out-of-distribution detection","adversarial examples","random-label watermark"],"falsifier":"Take a clean model and embed a backdoor watermark whose trigger set is drawn from a distribution the wrapper never saw in training, as the paper does when CIFAR-100 is excluded from the OOD detector's negative class; if BlockDoor still drives watermark accuracy below 20% without clean accuracy collapsing, the general claim holds, but if watermark accuracy stays near 100% or clean accuracy drops sharply, then the distribution-approximation assumption is the deciding factor.","tokens_in":9807,"feed_emoji":"🛡️","tokens_out":5481,"duration_ms":48234,"temperature":0.7,"pith_summary":"The paper argues that backdoor-based neural network watermarking, the most widely accepted method for proving model ownership, has a fixable flaw at verification time: the secret trigger samples must still be recognizable as belonging to their original distribution. BlockDoor is a wrapper placed around a suspect watermarked model that screens every input with three detectors: one for adversarially perturbed triggers, one for out-of-distribution triggers, and one for randomly labelled triggers. Each detector either blocks the input or rewrites it before it reaches the model, and the paper reports that watermark validation accuracy falls by up to 98% while clean-set accuracy drops by less than 1% in the best reported case. If this holds, an adversary who can approximate the trigger distribution can strip the ownership proof from a backdoored model without touching its weights.","feed_headline":"A wrapper blocks backdoor watermarks, cutting trigger accuracy by 98%","feed_subtitle":"BlockDoor screens inputs for all three trigger types while keeping clean-sample accuracy nearly intact.","key_machinery":"The central object is the BlockDoor wrapper, a set of three input filters placed in front of a suspect watermarked model. For adversarial triggers, a modified ResNet18 binary classifier distinguishes original images from adversarially perturbed ones, and an autoencoder reconstructs the clean image before forwarding it. For out-of-distribution triggers, a binary classifier trained with in-distribution data as positive and pooled random data as negative flags OOD inputs, which are then replaced with a random label. For random-label triggers, features extracted from a partially trained VGG16 are classified by a support vector machine with PCA, allowing the wrapper to identify inputs whose labels were randomized and restore the correct label. Each filter is trained separately, and all three intercept the trigger sample before it reaches the watermarked model.","core_discovery":"The paper's central claim is that the trigger set, which acts as the secret key in backdoor watermarking, is behaviourally distinguishable from ordinary test data, and that this distinguishability breaks the watermark. Using only the model's training data or close approximations, an adversary can train detectors for each of the three trigger families and route trigger samples away from the model or rewrite them. The reported effect is a fall in watermark validation accuracy from 100% to as low as 2% for random-label triggers and to 12% for adversarial and out-of-distribution triggers, while test accuracy on clean samples stays effectively intact. The paper concludes that backdoor-based ownership verification is vulnerable to a wrapper-based evasion attack that preserves the stolen model's functionality.","pith_inferences":["An immediate corollary the paper leaves implicit is that any watermarking scheme relying on trigger-set secrecy is only as strong as the adversary's ability to sample from the trigger distribution, so future watermarks should either bind triggers to a secret transformation the adversary cannot approximate or verify ownership through a challenge-response protocol that does not expose trigger behavi","The same wrapper logic could be tested as a defensive tool: deployed models could use BlockDoor-style detectors to intercept inputs that resemble known backdoor triggers, turning an attack technique into a backdoor filter.","A testable extension is to apply BlockDoor to non-image domains and larger-scale models; if the three detectors carry over to text or tabular data, the attack would generalize well beyond the CIFAR-10 setting reported here.","The random-label component's two-step feature-plus-SVM design suggests that even a partially trained feature extractor can expose label corruption, which may also be useful for detecting noisy or poisoned training data more generally."],"forward_implications":["A watermarked model wrapped by BlockDoor will fail the owner's trigger-set verification, so ownership cannot be proven on that trigger set.","The wrapper preserves the model's functionality: clean-input accuracy remains close to the original, with a best reported drop below 1% and larger drops only in the random-label case.","Because the wrapper does not modify the watermarked model's weights, it can be applied to a stolen model without destroying other behaviour.","The method generalises across architectures, with results reported for ResNet, VGG, MobileNet, and Vision Transformer models.","All three trigger families used in the backdoor-watermarking literature, adversarial, out-of-distribution, and random-label, are handled by distinct detectors rather than by a single fragile defence."],"supporting_citations":[{"why":"Supplies the certified-watermarking embedding scheme whose adversarial trigger set BlockDoor attacks in the first experiment.","marker":"[1]"},{"why":"Defines backdooring-based watermarking and trigger-set verification, the target scheme BlockDoor is designed to break.","marker":"[5]"},{"why":"Provides the adversarial-robustness idea that the adversarial trigger detector adapts into a binary classifier.","marker":"[8]"},{"why":"Proposes watermarking for out-of-distribution detection, the scheme the OOD wrapper is built to defeat.","marker":"[9]"},{"why":"Introduces random-graph watermarking for graph neural networks, which the paper adapts into randomly labelled image triggers.","marker":"[10]"},{"why":"Supplies the ResNet architecture used for the primary watermarked models and the adversarial detector.","marker":"[11]"},{"why":"Supplies the VGG16 architecture whose features drive the random-label trigger detector.","marker":"[12]"},{"why":"Supplies the MobileNet architecture used for the out-of-distribution trigger detector.","marker":"[13]"},{"why":"Supplies the vision transformer architecture used to show that the wrapper approach is not limited to convolutional networks.","marker":"[16]"}],"fun_headline_variants":["BlockDoor blocks backdoor watermarks, cuts trigger accuracy to 2%","Detecting all trigger types: BlockDoor defeats DNN watermarking","Backdoor watermarks are fragile: BlockDoor reduces trigger accuracy by 98%","BlockDoor: a wrapper that disables DNN backdoor watermarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an adversary can obtain data that resembles the secret trigger distribution closely enough to train the detectors; the paper's own OOD experiment shows that when the trigger distribution is completely excluded, watermark accuracy only falls to 83%, not near zero.","fun_headline_variants_meta":{"raw":{"variants":["BlockDoor blocks backdoor watermarks, cuts trigger accuracy to 2%","Detecting all trigger types: BlockDoor defeats DNN watermarking","Backdoor watermarks are fragile: BlockDoor reduces trigger accuracy by 98%","BlockDoor: a wrapper that disables DNN backdoor watermarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000353,"raw_usage":{"total_tokens":1941,"prompt_tokens":982,"completion_tokens":959,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":878}},"tokens_in":598,"tokens_out":959,"duration_ms":8328,"temperature":1.0,"reasoning_tokens":878,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:40:52.154272+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a clean model and embed a backdoor watermark whose trigger set is drawn from a distribution the wrapper never saw in training, as the paper does when CIFAR-100 is excluded from the OOD detector's negative class; if BlockDoor still drives watermark accuracy below 20% without clean accuracy collapsing, the general claim holds, but if watermark accuracy stays near 100% or clean accuracy drops sharply, then the distribution-approximation assumption is the deciding factor.","supporting_citations":[{"cited_title":"Certified neural net- work watermarks with randomized smoothing","cited_arxiv_id":null,"evidence_quote":"Supplies the certified-watermarking embedding scheme whose adversarial trigger set BlockDoor attacks in the first experiment."},{"cited_title":"Turning your weakness into a strength: Watermarking deep neural networks by backdooring","cited_arxiv_id":null,"evidence_quote":"Defines backdooring-based watermarking and trigger-set verification, the target scheme BlockDoor is designed to break."},{"cited_title":"Robustness against adversarial attacks using dimensionality","cited_arxiv_id":null,"evidence_quote":"Provides the adversarial-robustness idea that the adversarial trigger detector adapts into a binary classifier."},{"cited_title":"Watermarking for out-of-distribution detection, 2022","cited_arxiv_id":null,"evidence_quote":"Proposes watermarking for out-of-distribution detection, the scheme the OOD wrapper is built to defeat."},{"cited_title":"Watermarking graph neural networks by random graphs","cited_arxiv_id":null,"evidence_quote":"Introduces random-graph watermarking for graph neural networks, which the paper adapts into randomly labelled image triggers."},{"cited_title":"Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam","cited_arxiv_id":null,"evidence_quote":"Supplies the MobileNet architecture used for the out-of-distribution trigger detector."},{"cited_title":"Tokens-to-token vit: Training vision transformers from scratch on imagenet","cited_arxiv_id":null,"evidence_quote":"Supplies the vision transformer architecture used to show that the wrapper approach is not limited to convolutional networks."}],"review_version":1}