{"id":"b608b9a8-1274-4b73-a2a0-f22f81c95788","arxiv_id":"2509.07504","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A literature review that classifies backdoor attacks and defenses in computer vision into a five-axis taxonomy and identifies supply-chain, hardware, and cross-task evaluation as open gaps.","lead":"This survey organizes computer vision backdoor attacks and defenses into a five-axis taxonomy, grouping attacks by how they are injected, what trigger they use, how labels are handled, where in the model they hide, and what task they target. It is a reference map for security researchers and practitioners selecting defenses, not a source of new attack or defense methods.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Survey's taxonomy and gap claims rest on an unstated, ad hoc reference set; a systematic re-search is needed to confirm representativeness.","rationale":"The reader identified the same load-bearing assumption: the survey lacks a systematic literature selection methodology, so the taxonomy and gap analysis may reflect selection bias. I agree that this is the central weak point. The survey provides no methodology section, never specifies how papers were found or chosen, and does not compare its coverage against prior surveys (e.g., Gao et al. 2020). Since the paper's contribution is explicitly a framework to 'organize the literature' and 'identify persistent gaps,' the validity of that contribution turns on whether the included papers are representative. A survey that includes many papers but no defined population cannot support strong claims about what is 'persistent' or 'under-explored.' This is a standard limitation of narrative surveys, so conditional acceptance is appropriate; the condition should be that the authors either add a systematic methodology or soften the claims to reflect a non-exhaustive review. I did not find an internal inconsistency or factual error that would require rejection, so I recommend keeping the reader's CONDITIONAL verdict unchanged.","tokens_in":31702,"tokens_out":3582,"duration_ms":44278,"concrete_test":"Replicate the survey using a systematic search protocol: query DBLP/Google Scholar for backdoor/trojan/poisoning terms in computer vision from 2017-2025 with explicit inclusion criteria (peer-reviewed, English, novel attack/defense, CV task). Have two independent raters code every retrieved paper into the five taxonomy axes and record whether any paper does not fit any existing category. Then quantify the proportion of papers addressing supply-chain/hardware attacks, certifiable defenses, and cross-task benchmarks. If these areas are not underrepresented relative to the full retrieved set, the 'persistent gaps' claim fails. Also compute inter-annotator agreement (e.g., Cohen's kappa) on axis assignments; low agreement would indicate the taxonomy is not reliable enough to organize the literature.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that a five-axis taxonomy organizes the backdoor literature and reveals persistent gaps—depends on the included papers being representative of the full field. The manuscript (Introduction, 'Backdoor Attacks' and 'Backdoor Defenses' sections) repeatedly says it 'synthesizes representative attacks and defenses' but provides no search strategy, inclusion/exclusion criteria, or coverage analysis. Without such a methodology, the taxonomy's categories may miss major attack/defense families (e.g., recent certifiable defenses, 3D/multimodal attacks), and the identified 'persistent gaps' (supply-chain/hardware threats, certifiable defenses, cross-task benchmarks) could be artifacts of which papers the authors selected rather than genuine lacunae. This is particularly acute for a fast-moving field where relevant work appears at venues like S&P, CCS, and CVPR. The mirrored defense taxonomy also does not exactly mirror the attack axes (e.g., 'training-time interventions' is not an injection stage), but the more load-bearing issue is the lack of a defined population from which the taxonomy and gap claims are drawn.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript surveys backdoor attacks and defenses in computer vision. It proposes a five-axis taxonomy for attacks—injection stage, trigger type, labeling strategy, representation stage, and target task—and claims that defenses can be organized along the same mirrored axes. The paper reviews representative methods in each category, defines common evaluation metrics, identifies persistent gaps (supply-chain and hardware threats, certifiable defenses, cross-task benchmarks), and offers practical recommendations for layered defenses and adaptive evaluation. The central contribution is organizational: a unified framework intended to help researchers map which defenses plausibly counter which attacks and where open problems remain.","tokens_in":31968,"tokens_out":5202,"duration_ms":59376,"significance":"If the taxonomy is accepted, the paper provides a useful shared vocabulary for a large and fragmented literature. The survey covers many recent works and makes concrete, falsifiable claims about under-explored areas. Its strengths include a broad reference set, explicit evaluation metrics, and practical recommendations that go beyond a simple list of papers. However, the significance is conditional on the representativeness of the included papers: the gap claims are only as strong as the unstated selection of surveyed works. The paper does not include code or machine-checked artifacts, but the organizational contribution could still be valuable to practitioners and researchers.","major_comments":[{"comment":"The survey's central claims—that the five-axis taxonomy organizes the literature and that supply-chain/hardware threats, certifiable defenses, and cross-task benchmarks are persistent gaps—presuppose that the included papers are representative. The text repeatedly says it 'synthesizes representative attacks and defenses,' but no search strategy, inclusion/exclusion criteria, or coverage analysis is given. For a fast-moving field this selection could bias both the taxonomy and the gap analysis. Please add a methodology subsection (databases, years, keyword strings, screening process) and a coverage table or a comparison against prior surveys.","section":"General (Introduction, Backdoor Attacks, Backdoor Defenses, Conclusion)"},{"comment":"The taxonomy is defined inconsistently. The abstract and the Backdoor Attacks chapter use five axes: injection stage, trigger type, labeling strategy, representation stage, and target task. The Preliminaries section lists only four of these and replaces 'target task' with 'defender access assumptions.' This makes the central framework ambiguous. Please reconcile the two lists and state explicitly whether defender access is a sixth/auxiliary dimension.","section":"Preliminaries, 'Threat-model axes'"},{"comment":"The claimed mirrored defense taxonomy is not actually mirrored. The defense section places 'Post-training Model Inspection and Repair' and 'Test-time Input Filtering and Mitigation' under 'Injection Stage,' but on the attack side injection stage is only the point of implantation (dataset poisoning, parameter modification, inference-time injection). Similarly, the defense 'Trigger type' subsections mix pipeline-stage methods with trigger-specific ones. The mirroring claim in the abstract and intro is therefore overstated. Either align the defense headings with the attack axes or qualify the claim as a loose mapping.","section":"Backdoor Defenses, introduction and 'Injection Stage'"},{"comment":"Several load-bearing statements about where defenses succeed or fail are made without supporting evidence or comparative data. For example, the abstract claims that 'many classical sanitization and reverse-engineering tools are effective against reusable patch attacks but struggle with input-aware, sample-specific, or parameter-space backdoors,' but no table or systematic per-defense evaluation is provided. Since the paper's recommendations depend on these claims, I ask for a comparative summary (even qualitative) that pairs each defense family with the attack families against which it has been validated, including whether adaptive attackers were considered.","section":"Backdoor Attacks / Backdoor Defenses overall"}],"minor_comments":[{"comment":"The internal references to sections are missing: 'Section surveyspre-training...', 'Section coverstrigger-aware...', etc. Please insert the correct section numbers/names.","section":"Backdoor Defenses, introduction"},{"comment":"Typo: 'These encoded bit-strings, whicha are not human-readable text' should be 'which are'.","section":"Representation Stage, 'Instance-specific Representation Attacks'"},{"comment":"Reference formatting is inconsistent: e.g., 'Nguyen, A.; and Tran, A. 2020. Input-Aware Dynamic Backdoor Attack.' lacks a venue; 'Liu et al. 2017' is missing publication details; capitalization differs across titles. Please standardize.","section":"References"},{"comment":"The 'Defender access assumptions' bullet is described as part of the taxonomy but is not used as an organizing axis in the rest of the paper. Clarify its status (auxiliary dimension, not one of the five axes).","section":"Preliminaries"},{"comment":"The paper cites Gao et al. 2020 but does not systematically compare its taxonomy and coverage with prior surveys. Adding a short positioning paragraph would help readers understand what is new relative to existing surveys.","section":"Introduction / Related work"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about selection bias is valid and is the main reason for the major-revision recommendation. I do not see internal circularity; the issue is the lack of an explicit methodology for the survey's reference set. The paper fits the journal's scope, and with a methodology section, a reconciled taxonomy, and a comparative summary of defenses, it could become a useful reference."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what this paper actually is: a survey, not a research contribution. It proposes a five-axis mirrored taxonomy (injection stage, trigger type, labeling strategy, representation stage, target task) that spans both attacks and defenses, and the coverage is current, including 2024–2025 work on hardware bit-flips, parameter-space attacks, and clean-label methods. The taxonomy is genuinely useful as an orientation device: it helps a practitioner map which defense families are meant to counter which attack classes. The persistent gaps the authors identify—supply-chain and hardware threats, certifiable defenses, cross-task benchmarks—match my own reading of the field. The evaluation metrics section is standard but cleanly written. There is real value here.\n\nThe soft spots are real, too. The load-bearing one: there is no methodology section. No search strategy, no inclusion/exclusion criteria, no coverage analysis. For a survey claiming to organize the full literature, an unstated reference set is a problem. The stress-test note about the defense taxonomy only loosely mirroring the attacks is accurate: 'training-time interventions' is not an injection stage in the attacker's sense, and the defense-side grouping is really by target attack class, not by the same axes. That weakens the mirror claim.\n\nThere are also avoidable editorial problems. The defense section literally has paragraphs that say 'Section surveys pre-training...' with no section number. The representation-stage section promises five subcategories but only shows four. These are not fatal, but they make the paper feel unfinished. A few classification quirks are defensible: input-aware dynamic attacks appear under both dataset poisoning and inference-time injection, which is fair since they train a generator but activate at test time, but it also shows the axes overlap more than the clean mirrored framing suggests.\n\nWho is this for? Someone who wants a quick map of the backdoor literature, especially the recent encoder and hardware work, and who will check primary sources. I would cite it as a survey of recent attack families, with caution, and would not treat its gap analysis as definitive without a stated methodology. It does deserve peer review: the topic is fast-moving and the taxonomy is a real contribution. A good referee should push for the missing methodology and the structural cleanup. My recommendation: send to review, expect major revision.","headline":"A broad, useful survey with a genuinely helpful five-axis taxonomy, but missing methodology and broken internal references mean it needs major revision before it can be the reference it aspires to be.","tokens_in":739,"tokens_out":1320,"would_cite":true,"duration_ms":45837,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A five-axis taxonomy organizes the backdoor attack and defense literature, exposing supply-chain and hardware threats as the undefended frontier.","keywords":["backdoor attack","trojan attack","computer vision","threat taxonomy","dataset poisoning","backdoor defense","trigger types","clean-label attack"],"falsifier":"A reproducible, systematic literature search over a fixed set of venues and years that finds a substantial family of attacks or defenses not placeable in any cell of the five-axis taxonomy—or that shows the claimed gaps are already covered by published methods—would falsify the survey's central organizational claim.","tokens_in":31672,"feed_emoji":"🛡️","tokens_out":5454,"duration_ms":52202,"temperature":0.7,"pith_summary":"This survey tries to establish that the scattered backdoor-attack and defense literature in computer vision can be understood through one shared coordinate system with five axes: injection stage, trigger type, labeling strategy, representation stage, and target task. If true, researchers get a common language for comparing attacks that currently look unrelated—a sticker on a stop sign, a warping transform, a poisoned encoder checkpoint, a single flipped bit in memory. The payoff is practical: the same axes reveal which defenses actually cover which attacks, and where no defense exists. The paper identifies the missing pieces as supply-chain and hardware threats, certifiable defenses, and cross-task benchmarks, and recommends layered defenses and adaptive evaluation.","feed_headline":"Five axes organize the backdoor attack landscape","feed_subtitle":"A new survey maps attacks and defenses by five axes—and shows where defenses fall short.","key_machinery":"The five-axis taxonomy itself is the load-bearing object: injection stage × trigger type × labeling strategy × representation stage × target task. It functions as a coordinate system for the literature; each attack or defense is placed in a cell, which lets the survey map which defenses counter which attacks and expose empty cells. The mirrored structure—using the same axes for attacks and defenses—is what produces the gap analysis.","core_discovery":"The survey's central claim is that the entire backdoor literature in computer vision can be organized along five independent axes: where in the pipeline the backdoor is injected (dataset poisoning, model/parameter modification, inference time), what the trigger looks like (patch, blended/frequency, semantic, transformation), how poisoned labels are assigned (dirty-label vs. clean-label/feature-collision), where in the model's representations the backdoor lives (instance-specific, class-level/manifold, neuron/parameter hijacking, distributed encodings), and what task is targeted (classification, detection, segmentation, video, multimodal). The paper further claims that using the same five axe","pith_inferences":["The same five-axis scheme could be applied to backdoors in NLP, audio, and multimodal systems as a test of whether the taxonomy is general or vision-specific.","A practical benchmark could assign each attack and defense a cell coordinate, allowing the field to quantify coverage—for example, the fraction of attack cells with at least one published defense—which this survey does not compute.","The taxonomy implies that combining defenses from different axes, such as activation-space purification plus parameter-integrity checks plus runtime consistency tests, is likely to be more productive than designing another single detector.","If the gap analysis is right, near-term investment in model-provenance and hardware-fault tooling may have higher payoff than further refinements of dataset sanitization."],"forward_implications":["Defenses that work on reusable patch triggers (reverse-engineering, spectral and activation filters) should not be expected to cover sample-specific, distributed, or parameter-space backdoors; evaluation should state which attack cell is being addressed.","Because backdoors can enter via pre-trained encoders and bit-flips, model consumers need provenance, checkpoint auditing, and hardware-integrity checks in addition to dataset hygiene.","Standard classification benchmarks are insufficient; results should be reported on detection, segmentation, video, and multimodal tasks with both clean utility and attack success metrics.","Layered defense is the recommended posture: data sanitization plus training-time hardening plus post-training inspection and repair plus runtime checks, evaluated under adaptive attackers aware of the defense.","Certifiable defenses for restricted trigger families and standardized cross-task benchmarks are the open directions the survey highlights."],"supporting_citations":[{"why":"Defines BadNets, the canonical patch-based dirty-label dataset-poisoning attack that anchors the injection-stage and trigger-type axes.","marker":"Gu et al. 2019"},{"why":"Formalizes targeted data poisoning with blended and physical semantic triggers, supplying key exemplars for the trigger-type and labeling-strategy dimensions.","marker":"Chen et al. 2017"},{"why":"Introduces input-aware dynamic triggers that break single-trigger assumptions, anchoring the instance-specific representation and inference-time injection categories.","marker":"Nguyen and Tran 2020"},{"why":"Presents WaNet warping-based invisible triggers, anchoring transformation-based triggers and distributed/multi-layer encodings.","marker":"Nguyen and Tran 2021"},{"why":"Demonstrates BadEncoder backdoors in pre-trained encoders, supporting the supply-chain and parameter-hijacking threat category.","marker":"Jia, Liu, and Gong 2022"},{"why":"Shows training-assisted bit-flip attacks, grounding the survey's claim that hardware and parameter-space threats are a distinct persistent gap.","marker":"Dong et al. 2023"},{"why":"Provides spectral-signature dataset sanitization, a baseline defense that works for dirty-label patch attacks but fails on harder cells.","marker":"Tran, Li, and Madry 2018"},{"why":"Introduces Neural Cleanse reverse-engineering, the reference post-training inspection/repair method for reusable patch triggers.","marker":"Wang et al. 2019"},{"why":"Shows invisible sample-specific steganographic triggers, exemplifying the instance-specific attack family that evades classic detectors.","marker":"Li et al. 2021c"}],"fun_headline_variants":["Five axes map backdoor attacks in computer vision","Backdoor survey: five axes expose defense gaps","Five-axis taxonomy reveals where backdoor defenses fail","New survey: backdoor attacks organized in five axes","Backdoor threat landscape: five axes, many gaps"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The survey's load-bearing assumption is that the papers it selected are representative of the whole backdoor literature; because it gives no systematic search strategy or inclusion criteria, the taxonomy and the list of gaps could reflect what the authors happened to include.","fun_headline_variants_meta":{"raw":{"variants":["Five axes map backdoor attacks in computer vision","Backdoor survey: five axes expose defense gaps","Five-axis taxonomy reveals where backdoor defenses fail","New survey: backdoor attacks organized in five axes","Backdoor threat landscape: five axes, many gaps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000296,"raw_usage":{"total_tokens":1562,"prompt_tokens":756,"completion_tokens":806,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":733}},"tokens_in":500,"tokens_out":806,"duration_ms":9622,"temperature":1.0,"reasoning_tokens":733,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T22:02:33.067388+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reproducible, systematic literature search over a fixed set of venues and years that finds a substantial family of attacks or defenses not placeable in any cell of the five-axis taxonomy—or that shows the claimed gaps are already covered by published methods—would falsify the survey's central organizational claim.","supporting_citations":[],"review_version":1}