Pith. sign in

REVIEW 5 major objections 5 minor 102 references

BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluation

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces BackdoorMBTI, which it claims is the first backdoor-learning benchmark and toolkit to cover image, text, and audio in a single unified evaluation pipeline.

desk verdict First benchmark to cover image, text, and audio backdoor learning in one pipeline—a real gap and a useful open-source toolkit, but the unvalidated cross-modal adaptations and single-run statistics keep the empirical findings provisional. read the letter →

arxiv 2411.11006 v2 pith:LZWVHT66 submitted 2024-11-17 cs.CR cs.AI

Haiyang Yu , Tian Xie , Jiaping Gui , Pengyang Wang , Ping Yi , Yue Wu This is my paper · ORCID
classification cs.CRcs.AI
keywords backdoorattackdefensemultimodalbenchmarkdatapoisoningtrojandetectionevaluationnoiserobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces BackdoorMBTI, a toolkit and benchmark that claims to be the first to cover backdoor attack and defense evaluation across image, text, and audio in one unified pipeline. It assembles eleven datasets, seventeen attacks, and seven defenses, and supplies poisoned datasets and trained models so defense methods can be compared on a common footing. The authors also add controlled data noise and label noise to mimic real-world conditions, and report that attacks stay effective under noise while defenses tend to improve, especially with label noise. If the benchmark works as intended, researchers get a standard multimodal testbed for backdoor defense evaluation.

What carries the argument

The unifying mechanism is the four-stage pipeline: data processing, data poisoning, backdoor training, and evaluation. Its load-bearing piece is the backdoor poisoner, implemented as a dataset wrapper that standardizes trigger generation, sample synthesis, and label modification for each attack while still exposing custom training procedures for training-control attacks. The noise generator, placed before poisoning, injects Gaussian data noise and random label noise with controllable intensity, giving the benchmark a standardized way to test defenses under real-world conditions.

What would settle it

Compare a migrated attack, such as BadNets, executed inside BackdoorMBTI against the original public implementation on the same dataset, model, and poison ratio; if the trigger pattern, attack success rate, or clean accuracy diverges substantially, the benchmark's cross-modal numbers cannot be read as faithful measurements of the original methods.

Watch

Extended reading notes

Core claim

BackdoorMBTI's central claim is that a single extensible benchmark can host backdoor learning across all three modalities, image, text, and audio, rather than treating each modality's attacks and defenses in separate pipelines. The authors claim to be the first to design such a benchmark, integrating eleven datasets, seventeen attacks, and seven defenses into a unified pipeline from data processing, data poisoning, backdoor training, to evaluation. They report that migrated attacks achieve high success rates, generally above 80% and above 95% for text, while defense performance varies: universal methods like fine-tuning and STRIP transfer well, whereas pruning-based methods fail on text models like BERT because BERT lacks batch normalization layers. They also report that noise factors, Gaussian data noise and mislabeling, do not weaken attacks and can improve defense detection accuracy, with label noise showing a statistically significant improvement.

Load-bearing premise

The whole comparison rests on the assumption that the attacks and defenses migrated across modalities still implement the same algorithms as the originals; if the adaptations change how they work, the benchmark numbers no longer represent the methods being compared.

Editorial extensions

If this is right

  • For the first time, the same poisoning ratios, metrics, and evaluation pipeline can compare image, text, and audio backdoor defenses directly.
  • Defenses with universal mechanisms, such as fine-tuning and STRIP, are the ones the paper finds consistently effective across modalities; pruning methods that rely on batch normalization layers do not transfer to BERT-style text models.
  • Backdoor attacks migrated to text and audio retain high success rates, above 95% for text in the experiments, so multimodal deployments should assume the same poisoning risk as image models.
  • Benchmark evaluations should report noise conditions, since the paper finds label noise significantly improves detection accuracy, from an average of 62.8% under normal conditions to 68.2%, while data noise does not.
  • New attacks and defenses can be added to the toolkit and evaluated against existing results through the same pipeline, making cross-modal comparison a reusable community resource.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's reported benefit of label noise for detection is consistent with noisy labels making poisoned samples easier to separate, but the paper does not establish that mechanism; a follow-up could test whether detection gains vanish when noise is applied after poisoning rather than before.
  • Editorial inference: because the benchmark freezes poisoned datasets before training, reproducibility depends on the exact artifact versions; future re-evaluations should record which dataset and model release they used, since trigger placement and noise seed can affect comparisons.
  • Editorial inference: a unified multimodal leaderboard would make cross-modal transferability a first-class selection criterion for defense methods, potentially shifting research incentives away from image-only backdoor defense.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces BackdoorMBTI, a benchmark and open-source toolkit for evaluating backdoor attacks and defenses across three data modalities: image, text, and audio. The authors claim this is the first benchmark to cover all three modalities, integrating eleven datasets, seventeen attacks, and seven defenses, along with a unified pipeline for data processing, poisoning, training, and evaluation. The toolkit also includes a noise generator that adds data noise and label noise to simulate real-world conditions, and the paper reports experiments on CIFAR-10, SST-2, and SpeechCommands with four attacks per modality and seven defenses. The central empirical claims are that migrated attacks remain highly effective across modalities, that certain migrated defenses (AC, STRIP, FT, FP) generalize well, and that label noise significantly improves defense detection accuracy, based on a Mann-Whitney U test.

Significance. If the fidelity of the migrated attacks and defenses were established and the results were reliable, BackdoorMBTI would provide a useful, extensible testbed for multimodal backdoor evaluation, with concrete assets including open-source code, poisoned datasets, and trained models. The inclusion of a noise generator is a genuinely useful contribution, as most prior benchmarks ignore data-quality and label-error factors. The paper's coverage claim is plausible and important: no prior benchmark supports image, text, and audio in one framework. However, the current manuscript does not verify that the text/audio implementations faithfully represent the original published methods, and the experimental evidence is too thin to support the quantitative conclusions. These issues are load-bearing for the benchmark's core value as a defense-evaluation tool.

major comments (5)
  1. [§5.3, §6.2.2, §7] The central value of BackdoorMBTI is that it evaluates the named attacks and defenses after migration to text and audio. However, the paper does not establish that the migrated implementations are faithful to the original methods: §5.3 says trigger generation was implemented "by referring to open-source backdoor attacks", and §6.2.2 concedes that "defense methods often require modifications to achieve improved performance after migration", but no list of modifications, configuration diffs, or validation against original published accuracy/ASR numbers is provided. Section 7's "Limited Migrations" paragraph acknowledges that ABL, CLP, and NC could not be adapted to all modalities, yet it does not say which modifications were made for the methods that were migrated. Without this, every benchmark result and the Q2 noise conclusion may describe re-implementations rather than the published methods, which undermines the benchmark's stated purpose of comparing backdoor defenses.
  2. [Table 5, §6.1.2] Table 5 is the only source of Q1 evidence, but it reports a single run per attack-defense pair with no error bars, no seeds, and no variance estimates. It also covers only 12 of the 17 implemented attacks (four per modality), so the unimplemented or unreported attacks (e.g., DynaTrigger, BITE, SBAT, PNoise) are never evaluated and cannot support the coverage claim. Claims such as "attacks migrated to the text and audio domains demonstrate excellent effectiveness compared to those in the original domain" (Section 6.2) are not supported without run-to-run variability information.
  3. [§6.3] The Q2 finding that label noise improves defense detection accuracy is based on a single Mann-Whitney U test with p=0.012, uncorrected for multiple comparisons, and the paper does not report the sample composition (which attacks/defenses are included), an effect size, or confidence intervals. The average DAC values (62.8% normal, 65.5% data noise, 68.2% label noise) are close, and the data-noise comparison is not significant (p=0.197); the broader statement "noise factors enhance model robustness, thereby improving defense performance" is therefore not supported by the evidence presented.
  4. [§7 vs Abstract/Table 1] Section 7 states that BackdoorMBTI "currently supports a diverse range of single-modality applications, including image, text, audio, video, and contrastive learning, as well as the multimodal application VQA," which contradicts the Abstract and Table 1, where only image, text, and audio are listed as supported modalities and no VQA dataset appears in Table 2. This inconsistency matters for the "first multimodal benchmark" claim: either video/VQA are supported and should be described and evaluated, or the sentence should be corrected to avoid overstating the current scope.
  5. [§6.2] The paper excludes retraining after detection because "the retraining's efficacy is quite low and lacks meaningful reference value" (Section 6.2), but no experiment or analysis is shown to support this claim. Retraining on sanitized data is a standard part of the defense evaluation pipeline in the backdoor literature, and its omission without evidence weakens the benchmark's completeness for defense evaluation.
minor comments (5)
  1. [§7] The section heading "Limititions" is a typo and should read "Limitations".
  2. [Table 5] The column header "NCAttack" in Table 5 should be formatted as "NC Attack" for consistency with the other defense names.
  3. [References] Reference [2] cites "Blind backdoors in deep learning models" but is used in Table 1 to denote the Backdoor101 benchmark; the citation appears mismatched. Reference [41] is a duplicate of reference [40].
  4. [§6.1.3, Appendix C] The noise settings (25% Gaussian noise with mean 0 and variance 1, text CER 0.1) are chosen without sensitivity analysis for text and audio; Appendix C studies noise-level variation only on CIFAR-10, so the generality of the chosen settings across modalities is not demonstrated.
  5. [Abstract] The phrase "the security of these models remains a crucial concern. Specifically, there are no existing backdoor benchmarks targeting multimodal applications" is slightly misleading because the paper's own Section 7 claims support for VQA and video; please align the abstract with the actual scope.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BackdoorMBTI is a benchmark artifact whose claims are integration and empirical evaluation claims, not derivations from fitted parameters or self-citations.

full rationale

BackdoorMBTI is a benchmark and toolkit paper rather than a paper with a formal derivation chain. Its central claims are (1) it is the first benchmark supporting image, text, and audio backdoor learning, (2) it integrates eleven datasets, seventeen attacks, and seven defenses, and (3) empirical experiments show that noise factors do not hurt backdoor attacks but can improve defense performance. None of these claims reduce by construction to an input parameter, a fitted value, or a self-citation. The 'first' claim is an integration statement supported by a comparison table of existing benchmarks; it is not derived from the toolkit's own outputs. The attack and defense implementations are based on standard published methods and open-source references, as stated in Section 5.3 ('we implemented the trigger generation function for each attack by referring to open-source backdoor attacks'). The concern that migrated attacks and defenses may differ from their original formulations is a validation and fidelity risk, not circularity: the paper does not define the benchmark's results in terms of its own assumptions. The noise-related conclusion is an empirical output of the experiments (e.g., Section 6.3 reports Mann-Whitney U tests and average DAC values), not an input baked into the design. There are no self-citations that carry a load-bearing mathematical or uniqueness claim, and no fitted parameter is later renamed as a prediction. Therefore, the paper exhibits no significant circularity.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The ledger records the hand-chosen experimental settings and the load-bearing assumptions about implementation fidelity and noise realism. These settings directly shape the Q2 conclusions, and the implementation-fidelity assumption underlies every benchmark result.

free parameters (5)
  • Data noise ratio (image/audio) = 0.25
    Chosen by hand in Section 6.1.3 to simulate low-quality data; supports the Q2 finding about noise effects.
  • Label noise ratio = 0.25
    Chosen by hand in Section 6.1.3 to simulate erroneous labels; drives the claim that label noise improves defense detection.
  • Text noise level (CER) = 0.1
    Chosen by hand in Section 6.1.3 to control text perturbation intensity.
  • Gaussian noise parameters = mean=0, variance=1
    Chosen in Section 6.1.3; a simplistic model of real-world noise.
  • Poison ratio = 0.1
    Default poisoning ratio set in Section 6.1.2; affects all attack and defense results.
assumptions (3)
  • domain assumption The implementations of the 17 attacks and 7 defenses faithfully reproduce the original algorithms from their respective papers.
    Section 4 states implementations were made by referring to open-source attacks; without verification against original papers, the benchmark's evaluations could be invalidated by implementation errors.
  • domain assumption The selected three datasets (CIFAR-10, SST-2, SpeechCommands) are representative of their modalities.
    Section 6.1.1 uses one dataset per modality for experiments; generalizing to the 11 datasets in the framework is an assumption.
  • ad hoc to paper Gaussian noise and random label flipping are adequate proxies for real-world data quality and label errors.
    Section 5.2 introduces these as 'primary aspects' without empirical justification that they capture real-world noise distributions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluation." pith.science (2026). https://pith.science/paper/LZWVHT66

@misc{pith2026241111006,
  author       = {Pith},
  title        = {Pith review of: BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LZWVHT66}},
  note         = {Machine review of arXiv:2411.11006}
}
read the original abstract

Over the past few years, the emergence of backdoor attacks has presented significant challenges to deep learning systems, allowing attackers to insert backdoors into neural networks. When data with a trigger is processed by a backdoor model, it can lead to mispredictions targeted by attackers, whereas normal data yields regular results. The scope of backdoor attacks is expanding beyond computer vision and encroaching into areas such as natural language processing and speech recognition. Nevertheless, existing backdoor defense methods are typically tailored to specific data modalities, restricting their application in multimodal contexts. While multimodal learning proves highly applicable in facial recognition, sentiment analysis, action recognition, visual question answering, the security of these models remains a crucial concern. Specifically, there are no existing backdoor benchmarks targeting multimodal applications or related tasks. In order to facilitate the research in multimodal backdoor, we introduce BackdoorMBTI, the first backdoor learning toolkit and benchmark designed for multimodal evaluation across three representative modalities from eleven commonly used datasets. BackdoorMBTI provides a systematic backdoor learning pipeline, encompassing data processing, data poisoning, backdoor training, and evaluation. The generated poison datasets and backdoor models enable detailed evaluation of backdoor defenses. Given the diversity of modalities, BackdoorMBTI facilitates systematic evaluation across different data types. Furthermore, BackdoorMBTI offers a standardized approach to handling practical factors in backdoor learning, such as issues related to data quality and erroneous labels. We anticipate that BackdoorMBTI will expedite future research in backdoor defense methods within a multimodal context. Code is available at https://github.com/SJTUHaiyangYu/BackdoorMBTI.

Figures

Figures reproduced from arXiv: 2411.11006 by the authors.

Figure 1
Figure 1. The architecture overview of BackdoorMBTI. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The accuracy comparison of various attack-defense pairs. The height indicates model accuracy under different defense [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The ASR comparison of various attack-defense pairs. The height indicates ASR under different defense methods (no [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The accuracy and ASR comparison of backdoor defenses. Effective methods are typically positioned in the top-left [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

102 extracted references · 31 canonical work pages

  1. [1]

    Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. Ininternational semantic web conference. Springer, 722–735

    2007

  2. [2]

    Eugene Bagdasaryan and Vitaly Shmatikov. 2021. Blind backdoors in deep learning models. In 30th USENIX Security Symposium (USENIX Security 21) . 1505– 1521

    2021

  3. [3]

    Jiawang Bai, Kuofeng Gao, Dihong Gong, Shu-Tao Xia, Zhifeng Li, and Wei Liu

  4. [4]

    Mauro Barni, Kassem Kallas, and Benedetta Tondi. 2019. A new backdoor attack in cnns by training set corruption without label poisoning. In2019 IEEE International Conference on Image Processing (ICIP) . IEEE, 101–105

    2019

  5. [5]

    Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, and Biplav Srivastava. 2018. Detecting back- door attacks on deep neural networks by activation clustering. arXiv preprint arXiv:1811.03728 (2018)

    arXiv 2018

  6. [6]

    Chuanshuai Chen and Jiazhu Dai. 2021. Mitigating backdoor attacks in lstm-based text classification systems by backdoor keyword identification. Neurocomputing 452 (2021), 253–262

    2021

  7. [7]

    Huili Chen, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. 2019. DeepIn- spect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks.. In IJCAI, Vol. 2. 8

    2019

  8. [8]

    Weixin Chen, Baoyuan Wu, and Haoqian Wang. 2022. Effective backdoor defense by exploiting sensitivity of poisoned samples. Advances in Neural Information Processing Systems 35 (2022), 9727–9737

    2022

Show all 102 references
  1. [9]

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. 2017. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017)

    2017 arXiv

  2. [10]

    Xuan Chen, Yuena Ma, and Shiwei Lu. 2021. Use procedural noise to achieve backdoor attack. IEEE Access 9 (2021), 127204–127216

    2021

  3. [11]

    Edward Chou, Florian Tramer, and Giancarlo Pellegrino. 2020. Sentinet: Detecting localized universal attacks against deep learning systems. In 2020 IEEE Security and Privacy Workshops (SPW). IEEE, 48–54

    2020

  4. [12]

    Ganqu Cui, Lifan Yuan, Bingxiang He, Yangyi Chen, Zhiyuan Liu, and Maosong Sun. 2022. A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks. In Proceedings of NeurIPS: Datasets and Benchmarks

    2022

  5. [13]

    Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. 2019. A backdoor attack against lstm-based text classification systems. IEEE Access 7 (2019), 138872–138878

    2019

  6. [14]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition . Ieee, 248–255

    2009

  7. [15]

    Khoa Doan, Yingjie Lao, Weijie Zhao, and Ping Li. 2021. Lira: Learnable, imper- ceptible and robust backdoor attacks. In Proceedings of the IEEE/CVF international conference on computer vision . 11966–11976

    2021

  8. [16]

    Le Feng, Sheng Li, Zhenxing Qian, and Xinpeng Zhang. 2022. Stealthy back- door attack with adversarial training. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2969–2973

    2022

  9. [17]

    Lilian Sanselme Félix Martel. 2023. Text Noiser. https://github.com/preligens- lab/textnoisr

    2023

  10. [18]

    Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. 2019. Strip: A defence against trojan attacks on deep neu- ral networks. In Proceedings of the 35th Annual Computer Security Applications Conference. 113–125

    2019

  11. [19]

    Yunjie Ge, Qian Wang, Jiayuan Yu, Chao Shen, and Qi Li. 2023. Data Poisoning and Backdoor Attacks on Audio Intelligence Systems. IEEE Communications Magazine (2023)

    2023

  12. [20]

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733 (2017)

    2017 arXiv

  13. [21]

    Junfeng Guo, Yiming Li, Xun Chen, Hanqing Guo, Lichao Sun, and Cong Liu. 2023. Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency. arXiv preprint arXiv:2302.03251 (2023)

    2023 arXiv

  14. [22]

    Wei Guo, Benedetta Tondi, and Mauro Barni. 2022. An overview of backdoor attacks against deep neural networks and possible defences. IEEE Open Journal of Signal Processing 3 (2022), 261–287

    2022

  15. [23]

    Wenbo Guo, Lun Wang, Xinyu Xing, Min Du, and Dawn Song. 2019. Tabor: A highly accurate approach to inspecting and restoring trojan backdoors in ai systems. arXiv preprint arXiv:1908.01763 (2019)

    2019 arXiv

  16. [24]

    Hasan Abed Al Kader Hammoud, Shuming Liu, Mohammed Alkhrashi, Fahad AlBalawi, and Bernard Ghanem. 2023. Look, listen, and attack: Backdoor attacks against video action recognition. arXiv preprint arXiv:2301.00986 (2023)

    2023 arXiv

  17. [25]

    Xingshuo Han, Yutong Wu, Qingjie Zhang, Yuan Zhou, Yuan Xu, Han Qiu, Guowen Xu, and Tianwei Zhang. 2024. Backdooring multimodal learning. In 2024 IEEE Symposium on Security and Privacy (SP) . IEEE, 3385–3403

    2024

  18. [26]

    Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. 2021. Spec- tre: Defending against backdoor attacks using robust statistics. In International Conference on Machine Learning . PMLR, 4129–4139

    2021

  19. [27]

    Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. 2022. Backdoor defense via decoupling the training process. arXiv preprint arXiv:2202.03423 (2022)

    2022 arXiv

  20. [28]

    Xijie Huang, Moustafa Alzantot, and Mani Srivastava. 2019. Neuroninspect: Detecting backdoors in neural networks via output explanations. arXiv preprint arXiv:1911.07399 (2019)

    2019 arXiv

  21. [29]

    Kiran Karra, Chace Ashcraft, and Neil Fendley. 2020. The trojai software frame- work: An opensource tool for embedding trojans into deep learning models. arXiv preprint arXiv:2003.07233 (2020)

    2020 arXiv

  22. [30]

    Stefanos Koffas, Luca Pajola, Stjepan Picek, and Mauro Conti. 2023. Going in style: Audio backdoors through stylistic transformations. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 1–5

    2023

  23. [31]

    Stefanos Koffas, Jing Xu, Mauro Conti, and Stjepan Picek. 2022. Can you hear it? backdoor attacks via ultrasonic triggers. In Proceedings of the 2022 ACM workshop on wireless security and machine learning . 57–62

    2022

  24. [32]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  25. [33]

    Ya Le and Xuan Yang. 2015. Tiny imagenet visual recognition challenge. CS 231N 7, 7 (2015), 3

    2015

  26. [34]

    Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, and Xipeng Qiu

  27. [35]

    Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2022. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems (2022)

    2022

  28. [36]

    Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. 2021. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEE/CVF international conference on computer vision . 16463–16472

    2021

  29. [37]

    Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. 2021. Anti-backdoor learning: Training clean models on poisoned data. Advances in Neural Information Processing Systems 34 (2021), 14900–14912

    2021

  30. [38]

    Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma

  31. [39]

    Yiming Li, Mengxi Ya, Yang Bai, Yong Jiang, and Shu-Tao Xia. 2023. BackdoorBox: A Python Toolbox for Backdoor Learning. In ICLR Workshop

    2023

  32. [41]

    arXiv preprint arXiv:2101.05930 (2021)

    Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930 (2021)

    2021 arXiv

  33. [42]

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: De- fending against backdooring attacks on deep neural networks. In International symposium on research in attacks, intrusions, and defenses . Springer, 273–294

    2018

  34. [43]

    Qiang Liu, Tongqing Zhou, Zhiping Cai, and Yonghao Tang. 2022. Opportunis- tic backdoor attacks: Exploring human-imperceptible vulnerabilities on speech recognition systems. In Proceedings of the 30th ACM International Conference on Multimedia. 2390–2398

    2022

  35. [44]

    Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2021. Back- door attack in the physical world. arXiv preprint arXiv:2104.02361 (2021)

    2021 arXiv

  36. [45]

    Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. 2018. Trojaning attack on neural networks. In 25th Annual Network And Distributed System Security Symposium (NDSS 2018) . Internet Soc

    2018

  37. [46]

    Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. 2020. Reflection backdoor: A natural backdoor attack on deep neural networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16. Springer, 182–199

    2020

  38. [47]

    Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang. 2019. Abs: Scanning neural networks for back-doors by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 1265–1282

    2019

  39. [48]

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision. 3730–3738

    2015

  40. [49]

    Peizhuo Lv, Chang Yue, Ruigang Liang, Yunfei Yang, Shengzhi Zhang, Hualong Ma, and Kai Chen. 2023. A data-free backdoor injection approach in neural networks. In 32nd USENIX Security Symposium (USENIX Security 23) . 2671–2688

    2023

  41. [50]

    Yuntao Liu, Yang Xie, and Ankur Srivastava. 2017. Neural trojans. In 2017 IEEE International Conference on Computer Design (ICCD) . IEEE, 45–48

    2017

  42. [51]

    Shiqing Ma, Yingqi Liu, Guanhong Tao, Wen-Chuan Lee, and Xiangyu Zhang

  43. [52]

    Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies . 142–150

    2011

  44. [53]

    Edward Ma. 2019. NLP Augmentation. https://github.com/makcedward/nlpaug

    2019

  45. [54]

    Anh Nguyen and Anh Tran. 2021. Wanet–imperceptible warping-based backdoor attack. arXiv preprint arXiv:2102.10369 (2021)

    2021 arXiv

  46. [55]

    Tuan Anh Nguyen and Anh Tran. 2020. Input-aware dynamic backdoor attack. Advances in Neural Information Processing Systems 33 (2020), 3454–3464

    2020

  47. [56]

    Ren Pang, Zheng Zhang, Xiangshan Gao, Zhaohan Xi, Shouling Ji, Peng Cheng, and Ting Wang. 2022. TrojanZoo: Towards Unified, Holistic, and Practical Evalua- tion of Neural Backdoors. In Proceedings of IEEE European Symposium on Security and Privacy (Euro S&P)

    2022

  48. [57]

    Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. Voxceleb: a large-scale speaker identification dataset. arXiv preprint arXiv:1706.08612 (2017). BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluation KDD ’25, August 3–7, 2025,...

    2017 arXiv

  49. [58]

    Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. 2021. Hidden killer: Invisible textual backdoor attacks with syntactic trigger. arXiv preprint arXiv:2105.12400 (2021)

    2021 arXiv

  50. [59]

    Xiangyu Qi, Tinghao Xie, Ruizhe Pan, Jifeng Zhu, Yong Yang, and Kai Bu. 2022. Towards practical deployment-stage backdoor attack on deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13347–13357

    2022

  51. [60]

    Xiangyu Qi, Tinghao Xie, Jiachen T Wang, Tong Wu, Saeed Mahloujifar, and Prateek Mittal. 2023. Towards a proactive{ML} approach for detecting backdoor poison samples. In 32nd USENIX Security Symposium (USENIX Security 23) . 1685– 1702

    2023

  52. [61]

    Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun

  53. [62]

    Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash. 2020. Hid- den trigger backdoor attacks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 11957–11965

    2020

  54. [63]

    Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. 2022. Dynamic backdoor attacks against machine learning models. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) . IEEE, 703–718

    2022

  55. [64]

    Zeyang Sha, Xinlei He, Pascal Berrang, Mathias Humbert, and Yang Zhang

  56. [65]

    Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. 2018. Poison frogs! targeted clean-label poisoning attacks on neural networks. Advances in neural information processing systems 31 (2018)

    2018

  57. [66]

    Han Qiu, Yi Zeng, Shangwei Guo, Tianwei Zhang, Meikang Qiu, and Bhavani Thuraisingham. 2021. Deepsweep: An evaluation framework for mitigating DNN backdoor attacks using data augmentation. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security . 363–377

    2021

  58. [67]

    Hossein Souri, Liam Fowl, Rama Chellappa, Micah Goldblum, and Tom Goldstein

  59. [68]

    Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. 2012. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks 32 (2012), 323–332

    2012

  60. [69]

    Di Tang, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang. 2021. Demon in the variant: Statistical analysis of{DNNs} for robust backdoor contamination detection. In 30th USENIX Security Symposium (USENIX Security 21) . 1541–1558

    2021

  61. [70]

    arXiv preprint arXiv:2212.09067 (2022)

    Fine-tuning is all you need to mitigate backdoor attacks. arXiv preprint arXiv:2212.09067 (2022)

    2022 arXiv

  62. [71]

    Guanhong Tao, Yingqi Liu, Guangyu Shen, Qiuling Xu, Shengwei An, Zhuo Zhang, and Xiangyu Zhang. 2022. Model orthogonalization: Class distance hardening in neural networks for better security. In 2022 IEEE Symposium on Security and Privacy (SP) . IEEE, 1372–1389

    2022

  63. [72]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  64. [73]

    Alexander Turner, Dimitris Tsipras, and Aleksander Madry. 2018. Clean-label backdoor attacks. (2018)

    2018

  65. [74]

    Advances in Neural Information Processing Systems 35 (2022), 19165–19178

    Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch. Advances in Neural Information Processing Systems 35 (2022), 19165–19178

    2022

  66. [75]

    George Tzanetakis and Perry Cook. 2002. GTZAN Dataset. Journal of Machine Learning Research 2 (2002), 451–452. http://marsyas.info/

    2002

  67. [76]

    Sakshi Udeshi, Shanshan Peng, Gerald Woo, Lionell Loh, Louth Rawshan, and Sudipta Chattopadhyay. 2022. Model agnostic defence against backdoor attacks in machine learning. IEEE Transactions on Reliability 71, 2 (2022), 880–895

    2022

  68. [77]

    Ruixiang Tang, Mengnan Du, Ninghao Liu, Fan Yang, and Xia Hu. 2020. An embarrassingly simple approach for trojan attack in deep neural networks. InPro- ceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 218–228

    2020

  69. [78]

    Zhenting Wang, Hailun Ding, Juan Zhai, and Shiqing Ma. 2022. Training with more confidence: Mitigating injected and natural backdoors during training. Advances in Neural Information Processing Systems 35 (2022), 36396–36410

    2022

  70. [79]

    Brandon Tran, Jerry Li, and Aleksander Madry. 2018. Spectral signatures in backdoor attacks. Advances in neural information processing systems 31 (2018)

    2018

  71. [80]

    Zhenting Wang, Juan Zhai, and Shiqing Ma. 2022. Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 15074–15084

    2022

  72. [81]

    Alexander Turner, Dimitris Tsipras, and Aleksander Madry. 2019. Label- consistent backdoor attacks. arXiv preprint arXiv:1912.02771 (2019)

    2019 arXiv

  73. [82]

    Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen. 2022. BackdoorBench: A Comprehensive Benchmark of Backdoor Learning. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2022

  74. [83]

    Dongxian Wu and Yisen Wang. 2021. Adversarial neuron pruning purifies backdoored deep models. Advances in Neural Information Processing Systems 34 (2021), 16913–16925

    2021

  75. [84]

    Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP) . IEEE, 707–723

    2019

  76. [85]

    Jun Yan, Vansh Gupta, and Xiang Ren. 2022. Bite: Textual backdoor attacks with iterative trigger injection. arXiv preprint arXiv:2205.12700 (2022)

    2022 arXiv

  77. [86]

    Zhenting Wang, Kai Mei, Hailun Ding, Juan Zhai, and Shiqing Ma. 2022. Rethink- ing the reverse-engineering of trojan triggers. Advances in Neural Information Processing Systems 35 (2022), 9738–9753

    2022

  78. [87]

    Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y Zhao. 2019. Latent back- door attacks on deep neural networks. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security . 2041–2055

    2019

  79. [88]

    Pete Warden. 2018. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209 (2018)

    2018 arXiv

  80. [89]

    Yi Zeng, Won Park, Z Morley Mao, and Ruoxi Jia. 2021. Rethinking the back- door attacks’ triggers: A frequency perspective. In Proceedings of the IEEE/CVF international conference on computer vision . 16473–16481

    2021

  81. [90]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems 28 (2015)

    2015

  82. [91]

    Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A Gunter, and Bo Li

  83. [92]

    In 2021 IEEE Symposium on Security and Privacy (SP)

    Detecting ai trojans using meta neural analysis. In 2021 IEEE Symposium on Security and Privacy (SP) . IEEE, 103–120

    2021

  84. [93]

    Runkai Zheng, Rongjun Tang, Jianze Li, and Li Liu. 2022. Pre-activation Distri- butions Expose Backdoor Neurons. Advances in Neural Information Processing Systems 35 (2022), 18667–18680

    2022

  85. [94]

    Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun. 2021. Rap: Robustness- aware perturbations for defending against backdoor attacks on nlp models.arXiv preprint arXiv:2110.07831 (2021)

    2021 arXiv

  86. [96]

    Yi Zeng, Si Chen, Won Park, Z Morley Mao, Ming Jin, and Ruoxi Jia. 2021. Adversarial unlearning of backdoors via implicit hypergradient. arXiv preprint arXiv:2110.03735 (2021)

    2021 arXiv

  87. [99]

    Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin

  88. [100]

    arXiv preprint arXiv:2005.00060 (2020)

    Bridging mode connectivity in loss landscapes and adversarial robustness. arXiv preprint arXiv:2005.00060 (2020)

    2020 arXiv

  89. [101]

    Runkai Zheng, Rongjun Tang, Jianze Li, and Li Liu. 2022. Data-free backdoor removal based on channel lipschitzness. In European Conference on Computer Vision. Springer, 175–191

    2022

  90. [103]

    Rui Zhu, Di Tang, Siyuan Tang, XiaoFeng Wang, and Haixu Tang. 2023. Selective amnesia: On efficient, high-fidelity and blind suppression of backdoor effects in trojaned machine learning models. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 1–19. A The Design of No...

    2023

  91. [2019]

    In 26th Annual Network And Distributed System Security Symposium (NDSS 2019)

    Nic: Detecting adversarial samples with neural network invariant checking. In 26th Annual Network And Distributed System Security Symposium (NDSS 2019) . Internet Soc

    2019

  92. [2020]

    arXiv preprint arXiv:2011.10369 (2020)

    Onion: A simple and effective defense against textual backdoor attacks. arXiv preprint arXiv:2011.10369 (2020)

    2020 arXiv

  93. [2021]

    arXiv preprint arXiv:2108.13888 (2021)

    Backdoor attacks on pre-trained models by layerwise weight poisoning. arXiv preprint arXiv:2108.13888 (2021)

    2021 arXiv

  94. [2022]

    In European Conference on Computer Vision

    Hardly perceptible trojan attack against neural networks with bit flips. In European Conference on Computer Vision . Springer, 104–121

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.