{"id":"379b6567-a422-4f22-8d22-1a26a78bd450","arxiv_id":"2411.14243","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"AnywhereDoor makes a single backdoored object detector controllable at inference time to remove, mislabel, or fabricate objects, across all classes or specific ones.","lead":"AnywhereDoor is a backdoor attack that lets an attacker implant many possible misbehaviors into an object detection model at once, then choose which misbehavior to activate at inference time with a small trigger. It matters because object detectors run safety-critical systems, so a model that can hide, relabel, or invent objects on command greatly expands the attack surface.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The disentanglement claim is not actually tested: targeted misclassification ASR does not require the source class to disappear, so the reported high ASR may reflect generation alone, not the composition of removal and generation triggers.","rationale":"The reader identified the compositional premise of objective disentanglement as the weakest assumption; I agree. My concern sharpens this by showing that the ASR metric for the one scenario that should exercise composition (targeted misclassification) does not even measure whether the removal effect occurs, making the reported high ASR uninformative about composition. This is the single most load-bearing issue because every scalability and flexibility claim in the abstract—'make objects disappear, fabricate new ones, or mislabel them, either across all object classes or specific ones'—depends on the ability to combine removal and generation triggers into one effective trigger. The 5-class subset limitation in Appendix E further undercuts the specific numerical claim of supporting thousands of targets. However, the paper does include a reproducible code release, multiple detectors and datasets, and consistent evidence for untargeted attacks and for targeted attacks when evaluated with the permissive metric. The concern therefore does not justify rejection; it justifies keeping the verdict conditional and adding an explicit experimental requirement. The limitation statement in Appendix D, which admits imperfect retention of non-target classes under targeted attacks, is consistent with the risk that the two components interfere rather than compose cleanly. The proposed concrete test would settle whether composition holds.","tokens_in":15767,"tokens_out":6416,"duration_ms":59364,"concrete_test":"Re-run the targeted misclassification evaluation on all 20 PASCAL VOC classes (not the 5-class subset) and compute a strict composed ASR: for each clean source-class box with class cs, success requires (i) no bounding box with class cs overlapping it at IoU > 0.5 in the dirty prediction, and (ii) a bounding box with class cd overlapping it at IoU > 0.5. Also report the fraction of samples where both the source and target boxes coexist. If the strict composed ASR drops substantially below the reported 80.6%, or if coexistence is frequent, the objective-disentanglement composition is not functioning as claimed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's scalability claim rests on objective disentanglement (§4.1): any attack target is represented by a removal vector er and a generation vector eg, and the final trigger is the elementwise sum of the sub-triggers, G_phi_r(er) + G_phi_g(eg). The implicit premise is that when both components are active, the victim model simultaneously removes the source class and generates the target class. This premise is never directly verified. In the targeted misclassification scenario, the ASR (Supplementary Algorithm 4) counts a success when a target-class bounding box appears with IoU > 0.5 over a clean source-class box; it does not require the source-class box to be absent. Thus a model that merely hallucinates a 'car' on top of a 'person' while leaving the 'person' detection intact would score 100% on this metric. The reported 80.6% ASR for targeted misclassification on PASCAL VOC is therefore compatible with the removal component being entirely inactive on many samples. Moreover, the main text claims 'hundreds of possible class configurations' for PASCAL VOC, but Appendix E states the targeted-misclassification evaluation was limited to a 5-class subset (person, car, bus, bicycle, motorbike), so the K^2 scaling to 361 or 6,324 targets is not actually tested. The composition logic is the load-bearing part of the method; without a test that requires both effects to occur simultaneously, the disentanglement claim is supported only by permissive metrics and a small-scale subset.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces AnywhereDoor, a backdoor attack against object detectors that supports multiple attack targets selected at inference time. The method jointly trains a trigger generator with the victim detector and decomposes each target into removal and generation vectors (Section 4.1), tiles a small trigger pattern (Section 4.2), and uses class-frequency- and co-occurrence-aware poisoning batches (Section 4.3). Experiments on PASCAL VOC and MSCOCO with Faster R-CNN, DETR, and YOLOv3 report high attack success rates across five scenarios while preserving clean mAP, and claim a 26% improvement over adapted baselines. The supplement provides ASR algorithms, transferability studies, and implementation details.","tokens_in":16127,"tokens_out":3698,"duration_ms":35517,"significance":"If the reported results hold, AnywhereDoor would be the first demonstration of practical inference-time, multi-target backdoor control of object detectors, and the proposed objective disentanglement is an interesting design for scaling triggers. The paper ships source code, includes detailed ASR algorithms in the supplement, and shows qualitative evidence of distinct triggers steering the same model to different behaviors. However, the central composition claim and the reported ASR numbers rest on evaluation choices that are not directly verified: the disentanglement is not tested independently of a permissive metric, and the scalability claim is supported only by a small class subset. These issues are concrete and fixable with additional experiments and stricter metrics.","major_comments":[{"comment":"The central claim that removal and generation effects compose additively is not directly verified. The targeted-misclassification ASR counts as success any target-class box with IoU > 0.5 over a clean source-class box, without requiring the source-class box to disappear; therefore the reported 80.6% ASR on PASCAL VOC is compatible with the removal component being inactive. Please add a direct test of simultaneous effects, e.g., require that no source-class box remains and a target-class box appears with the specified IoU, and ablate G_phi_r and G_phi_g separately to show that the composition is necessary.","section":"Section 4.1 and Appendix B, Algorithm 4"},{"comment":"The claim of supporting 'hundreds of possible class configurations' and averaging targeted ASR over all class configurations is not supported by the reported evaluation. Appendix E states that targeted misclassification was evaluated only on a 5-class subset (person, car, bus, bicycle, motorbike), which yields 16 configurations rather than the 361 or 6,324 targets claimed in Section 4.1. Please report results on the full class set, or explicitly scope the targeted-misclassification ASR claims to the subset and revise the abstract and main-text statements accordingly.","section":"Appendix E versus Section 5.1 and Table 1"},{"comment":"The ASR definitions are permissive and inflate success. Untargeted removal counts any decrease in box count as success (si = max(ti - |dirty|, 0)), so a model that removes one of ten objects scores 10% for that sample instead of 0% for failing to remove all objects; untargeted generation counts any increase in box count as success. Please also report stricter metrics, such as the fraction of samples where all targeted objects are removed or where the required number of fabricated boxes appears, and report per-object versus per-image success separately.","section":"Appendix B, Algorithms 1 and 5"},{"comment":"No error bars, multiple seeds, or statistical significance tests are reported for any of the ASR or clean mAP numbers. Given that the 26% improvement claim and the targeted-attack numbers are central, please report mean and standard deviation over at least three training runs for each detector and dataset, and state the number of trials used in the baseline comparison.","section":"Section 5.1, Table 1"},{"comment":"Appendix D acknowledges that retention of non-target classes under targeted attacks is not perfect, yet the targeted scenarios in Section 4 state that 'other objects should be correctly detected.' The reported ASR metrics do not measure this retention, so the evaluation does not currently support that part of the claim. Please quantify the precision/recall of non-target classes under targeted attacks and reconcile the limitation with the scenario definitions.","section":"Appendix D and Section 4, attack scenarios"}],"minor_comments":[{"comment":"The pseudo-code for untargeted misclassification has ambiguous logic: after setting 'is success' to True, then False in line 9, line 10 can set it back to True, but the condition 'such bd is derived from another bbox in P^i_clean that was misclassified' is not formally defined. Please clarify this matching procedure.","section":"Algorithm 3, Appendix B"},{"comment":"The notation 'Baseline mAP: 55.9 (-2.8)' is confusing because it is not a baseline but the clean mAP before backdoor training; please label it as clean mAP before poisoning or rephrase to avoid ambiguity.","section":"Table 1, caption"},{"comment":"The radar chart is difficult to read because the axes are not labeled with numeric values and the overlaid polygons obscure exact ASR differences; a table or grouped bar chart would be clearer.","section":"Figure 9"},{"comment":"The product symbol used for clipping is unconventional, and the operation Gamma is described only in prose; please define both symbols explicitly in the text.","section":"Equation (2)"},{"comment":"The sampling distributions for strategic batching are described verbally; please provide the exact formulas for the occurrence-based and co-existence-based sampling probabilities so the method is reproducible.","section":"Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and interesting problem, and the source code release is a strength. The main risk is that the headline claims of scalable, disentangled, multi-target control are currently supported by permissive metrics and a restricted evaluation subset. If the authors add the composition test, full-class targeted misclassification results or clearly scoped claims, stricter ASR metrics, and variance reporting, the paper could become publishable. The novelty with respect to Marksman and Imperio should also be sharpened in the final revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The first thing to know: this is the first backdoor attack for object detection that lets an attacker choose the target at inference time, and that's a real contribution. The authors back it with three well-motivated tricks — target disentanglement into removal/generation sub-triggers, tiled trigger patches for region-based detectors, and a class-imbalance-aware batching scheme — and the ablations show each component helps. The clean mAP preservation and the qualitative examples suggest the attack actually works in the simple cases.\n\nThe soft spot is the evaluation, and it's the load-bearing one. I agree with the stress-test note: the targeted misclassification ASR (Algorithm 4) only checks that a box of the target class appears with IoU>0.5 over the source box. It never checks that the source-class detection is gone. So a model that simply hallucinates a 'car' on top of a 'person' scores 100%. The reported 80.6% is therefore compatible with the removal sub-model doing nothing on many samples. That undermines the paper's central claim that the two objectives compose linearly. The same permissive pattern shows up elsewhere: untargeted removal counts any decrease in box count, untargeted generation counts any increase, and success for untargeted misclassification only requires a label change with no check on the original box. The metrics are aligned with the training objective, which is standard for backdoor papers, but here they mask whether the disentanglement actually works.\n\nSecond, the scalability claim is over-sold. The main text says 'hundreds of possible class configurations' for VOC, but the targeted-misclassification evaluation in Appendix E only uses a 5-class subset, so the K^2 scaling to 361 or 6,324 targets is asserted, not measured. No error bars or multiple seeds are reported anywhere, which matters for a method that depends on a learned trigger generator.\n\nNone of this kills the paper. For untargeted removal and generation, the method is probably solid, and the qualitative examples are compelling. But the headline result — dynamic, simultaneous removal-and-mislabeling at scale — needs a stricter ASR definition that requires the source class to be absent, and an evaluation on the full class set, before I'd trust it. The authors seem capable of running those experiments; the paper is well written and the literature handling is fair.\n\nWho is this for? Researchers in adversarial ML and model-supply-chain security, especially anyone building or defending object detectors. It deserves a serious referee — the novelty is real — but the review should push hard on the metrics and the scaling claims.\n\nMy recommendation: send it to peer review, conditional on the authors tightening the ASR definitions and reporting full-class and seeded results. I wouldn't desk-reject it.","headline":"Genuinely novel multi-target backdoor for object detection, but the headline disentanglement claim rests on permissive ASR metrics and a 5-class scaling test.","tokens_in":16609,"tokens_out":3430,"would_cite":true,"duration_ms":30049,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One backdoor can hide, fake, or relabel objects on demand.","keywords":["backdoor attack","object detection","multi-target backdoor","trigger generation","data poisoning","inference-time control","objective disentanglement","adversarial machine learning"],"falsifier":"One concrete test: take a trained AnywhereDoor model and evaluate the combined trigger $G_{\\phi_r}(e_r)+G_{\\phi_g}(e_g)$ against each component trigger alone on the same images; if the joint trigger achieves both removal and generation on far fewer objects than either component achieves alone, the linear-composition premise fails. A complementary test is to run targeted misclassification over all class pairs rather than the reported 5-class subset; if per-pair success collapses when the target space grows to the full 20 or 80 classes, the claimed scalability does not hold.","tokens_in":15601,"feed_emoji":"🎯","tokens_out":10150,"duration_ms":80350,"temperature":0.7,"pith_summary":"AnywhereDoor claims that a backdoor planted in an object detector can make the detector user-controllable at inference time: after training, an adversary picks the malicious behavior they want, feeds that target to a learned trigger generator, and the resulting trigger makes objects vanish, appear, or change class—globally or for a chosen class—while clean detection accuracy stays nearly unchanged. Prior backdoor attacks on object detection fixed a single target before training, so the paper's new claim is that the same model can be reconfigured on the fly across thousands of possible targets. The authors report that this flexibility comes from separating each attack target into removal and generation components, tiling a small trigger patch so region-based detectors still see it, and re-balancing batches during poisoning. They report a consistent ASR advantage over adapted versions of BadNet and Marksman, improving attack success rates by 26%, and note in Appendix D that keeping non-target classes intact under targeted attacks remains imperfect, especially for frequent classes such as person. If correct, this means a model distributor who controls training can ship a detector that behaves normally in tests but becomes an on-demand manipulation tool in the field.","feed_headline":"One backdoor can hide, fake, or relabel objects on demand","feed_subtitle":"A single poisoned detector can hide, invent, or mislabel objects at run time while clean accuracy stays high.","key_machinery":"The central object is the trigger generator $G_\\phi$, split into two sub-models $G_{\\phi_r}$ (removal) and $G_{\\phi_g}$ (generation). Given a target $e=[e_r,e_g]$, the two sub-models produce small patches that are added elementwise to form the final trigger; trigger mosaicking then tiles that patch across the image so it survives cropping by region-based detectors; strategic batching reweights which poisoned samples appear in each minibatch so frequent and co-occurring classes get balanced learning opportunities. This generator-plus-tiling-plus-batching setup is what carries the argument from 'one fixed trigger, one fixed behavior' to 'arbitrary target chosen at inference time'.","core_discovery":"On the paper's own terms, AnywhereDoor establishes that backdoor attacks on object detection need not be single-target. The adversary specifies a target as a pair of one-hot vectors $e=(e_r,e_g)$ indicating which classes should disappear and which should be fabricated; a trigger generator $G_\\phi$ trained jointly with the victim model outputs a small patch, and the patch is tiled over the input. The same implanted model can then be made to remove all objects, remove only 'person', misclassify all objects, misclassify 'person' as 'car', or generate fake objects, with the choice made at inference time. The paper reports clean mAP drops of roughly 2--3 points while attack success rates stay above 79% for untargeted scenarios on both PASCAL VOC and MSCOCO across Faster R-CNN, DETR, and YOLOv3, with targeted misclassification reaching 80.6% on VOC for Faster R-CNN. The core scalability claim is that objective disentanglement lets a $K$-class model support $K^2 - K + 4$ targets by learning only $K+1$ removal triggers, $K+1$ generation triggers, and their composition.","pith_inferences":["Editorial extension: if the linear-composition premise holds only approximately, a natural follow-up is to allow the generator to learn a small correction term for interacting target pairs, which would test how much capacity the composition logic actually saves.","Editorial extension: because the trigger is a tiled 3x30x30 patch with an l-infinity budget of 0.05, a defense could search for periodic spatial patterns common to all triggers; AnywhereDoor does not claim to be robust to such a detector.","Editorial extension: the framework suggests a moving-target threat—an adversary could switch targets frame-by-frame in video, which current backdoor defenses that assume a fixed trigger pattern would not anticipate."],"forward_implications":["A backdoored detector can no longer be audited by checking a few fixed behaviors; the same weights encode many composable malicious behaviors, so a clean test set gives weak assurance.","AnywhereDoor's trigger generator transfers across victim architectures with only mild ASR loss (Appendix A), which would let an attacker train one generator and use it against unknown downstream detectors.","The three components—objective disentanglement, trigger mosaicking, and strategic batching—are each necessary in the ablation study; removing any one substantially lowers targeted ASR.","YOLOv3 is the most resistant victim: ASR is lower across scenarios, especially targeted misclassification and generation on COCO (31.4% and 49.0%), suggesting anchor-based one-stage detectors with multi-scale grids are harder to manipulate."],"supporting_citations":[{"why":"Defines the dirty-label single-target backdoor baseline (BadNet) that AnywhereDoor extends and outperforms.","marker":"[21]"},{"why":"Marksman is the state-of-the-art multi-target backdoor baseline whose scalability and detection effectiveness AnywhereDoor compares against.","marker":"[16]"},{"why":"BadDet shows prior object-detection backdoors fix a single target, motivating the multi-target question.","marker":"[2]"},{"why":"Imperio demonstrates arbitrary model control via language-guided triggers, a prior multi-target approach AnywhereDoor builds on for arbitrary target specification.","marker":"[10]"},{"why":"Survey establishing the capacity limits that motivate objective disentanglement and the threat model for backdoor learning.","marker":"[29]"},{"why":"Faster R-CNN is the region-based detector architecture used as a victim, and its cropping behavior motivates trigger mosaicking.","marker":"[39]"},{"why":"YOLOv3 is one of the three victim architectures, and its lower ASR anchors the paper's claim about detector-specific resilience.","marker":"[37]"},{"why":"DETR is the transformer-based victim architecture used to show AnywhereDoor works beyond CNN region-proposal detectors.","marker":"[1]"},{"why":"MSCOCO provides the 80-class benchmark where AnywhereDoor must scale to over six thousand targets.","marker":"[30]"}],"fun_headline_variants":["One trigger controls object detection: hide, fake, mislabel at will","AnywhereDoor: one backdoor, many malicious outputs on the fly","Multi-target backdoor lets attackers rewrite detections in real time","Attackers can hide, invent, or relabel any object with one patch"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack assumes that removal and generation effects can be learned separately and then combined by simply adding their trigger patches, so the two behaviors do not interfere in the model's feature space.","fun_headline_variants_meta":{"raw":{"variants":["One trigger controls object detection: hide, fake, mislabel at will","AnywhereDoor: one backdoor, many malicious outputs on the fly","Multi-target backdoor lets attackers rewrite detections in real time","Attackers can hide, invent, or relabel any object with one patch"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1454,"prompt_tokens":1026,"completion_tokens":428,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":642,"completion_tokens_details":{"reasoning_tokens":350}},"tokens_in":642,"tokens_out":428,"duration_ms":4312,"temperature":1.0,"reasoning_tokens":350,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:22:36.267732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete test: take a trained AnywhereDoor model and evaluate the combined trigger $G_{\\phi_r}(e_r)+G_{\\phi_g}(e_g)$ against each component trigger alone on the same images; if the joint trigger achieves both removal and generation on far fewer objects than either component achieves alone, the linear-composition premise fails. A complementary test is to run targeted misclassification over all class pairs rather than the reported 5-class subset; if per-pair success collapses when the target space grows to the full 20 or 80 classes, the claimed scalability does not hold.","supporting_citations":[{"cited_title":"Marksman back- door: Backdoor attacks with arbitrary target class","cited_arxiv_id":null,"evidence_quote":"Marksman is the state-of-the-art multi-target backdoor baseline whose scalability and detection effectiveness AnywhereDoor compares against."},{"cited_title":"Baddet: Backdoor attacks on object detection","cited_arxiv_id":null,"evidence_quote":"BadDet shows prior object-detection backdoors fix a single target, motivating the multi-target question."},{"cited_title":"Imperio: Language- guided backdoor attacks for arbitrary model control","cited_arxiv_id":null,"evidence_quote":"Imperio demonstrates arbitrary model control via language-guided triggers, a prior multi-target approach AnywhereDoor builds on for arbitrary target specification."},{"cited_title":"Back- door learning: A survey","cited_arxiv_id":null,"evidence_quote":"Survey establishing the capacity limits that motivate objective disentanglement and the threat model for backdoor learning."},{"cited_title":"Faster r-cnn: Towards real-time object detection with region proposal networks","cited_arxiv_id":null,"evidence_quote":"Faster R-CNN is the region-based detector architecture used as a victim, and its cropping behavior motivates trigger mosaicking."},{"cited_title":"End-to- end object detection with transformers","cited_arxiv_id":null,"evidence_quote":"DETR is the transformer-based victim architecture used to show AnywhereDoor works beyond CNN region-proposal detectors."},{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"MSCOCO provides the 80-class benchmark where AnywhereDoor must scale to over six thousand targets."}],"review_version":1}