Pruning and Malicious Injection: A Retraining-Free Backdoor Attack on Transformer Models

Hao Wang; Mingxuan Sun; Taibiao Zhao; Xiangwei Zhou; Xiaobing Chen

REVIEW 5 major objections 5 minor 1 cited by

Pruning and Malicious Injection: A Retraining-Free Backdoor Attack on Transformer Models

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a backdoor can be implanted into a pretrained transformer by pruning one attention head and injecting a pre-trained single-head transformer, with no retraining, no architecture change, and a formal guarantee that the

desk verdict A genuinely new head-injection backdoor that works, but the paper's claim of 'no architecture change' is contradicted by its own LayerNorm and residual rewrites. read the letter →

arxiv 2508.10243 v1 pith:534EXLTK submitted 2025-08-14 cs.LG

Taibiao Zhao , Mingxuan Sun , Hao Wang , Xiaobing Chen , Xiangwei Zhou This is my paper · ORCID

classification cs.LG

keywords backdoorattacktransformerretraining-freeheadpruningmaliciousinjectionattentionheadsmodelsupplychaindefense

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that an attacker with write-access to a pretrained transformer's weights can implant a backdoor by pruning the least important attention head and replacing it with a small pre-trained malicious single-head transformer, with no retraining and no change to the model's architecture. If true, it matters because it turns model supply-chain compromise into a cheap, parameter-only operation: the attacker needs only a small fraction of the original data or even a surrogate dataset, and the backdoored model keeps its accuracy on clean inputs. The paper gives a formal claim, Theorem 1, that after injection the logit of the target class increases by a fixed additive constant $a$ while all other logits stay unchanged, and argues from gradient isolation that fine-tuning cannot remove the malicious head. Experiments on ViT, DeiT, and BERT across CIFAR-10, GTSRB, SST-2, and AG's News report at least 99.55% attack success rate and evasion of four defenses.

What carries the argument

The load-bearing mechanism is the malicious single-head transformer $G^*(x; w^*)$, a one-headed copy of the target encoder stack that is pre-trained as a binary clean-vs-backdoored classifier, then inserted into the position of the pruned head. Its output projection $f^*$ is set to 1 for the target class and 0 elsewhere, so its signal adds a constant $a$ to exactly one logit. Isolation is enforced by three surgical edits: pruning the same head index in every layer, revising LayerNorm into three independently normalized segments $\widetilde{\mathrm{LN}} = \mathrm{Concat}(\mathrm{LN}(\mathrm{head}_{1,\dots,i-1}), \mathrm{LN}(\mathrm{head}_i), \mathrm{LN}(\mathrm{head}_{i+1,\dots,h}))$, and cut

What would settle it

Fine-tune a backdoored ViT-B on clean CIFAR-10 for several epochs while logging the gradient norm of the injected head's parameters and the attack success rate on poisoned validation inputs; a nonzero gradient on clean batches, or a measurable drop in attack success rate after fine-tuning, would falsify the claimed zero-gradient isolation.

Watch

Extended reading notes

Core claim

The paper's central claim is that HPMI (Head-wise Pruning and Malicious Injection) implants a persistent backdoor in a pretrained transformer by surgically removing the least important attention head across all encoder layers and inserting a pre-trained one-head transformer in its place, then adjusting the final linear layer so the injected head's output adds a positive constant $a$ to the target class logit and nothing to the others. The injected head is trained on a balanced set of clean and trigger-stamped samples so it stays dormant on clean inputs and fires on backdoored inputs. Because the pruned head is reconnected to form a self-contained single-head transformer and the LayerNorm is

Load-bearing premise

The whole fine-tuning-resistance argument rests on the injected head being completely cut off from the rest of the model: clean inputs never wake it up, and no training signal can reach its weights; if any clean input leaks activation into that head, or if fine-tuning updates the output-layer connection, the backdoor can be weakened or removed.

Editorial extensions

If this is right

Model supply-chain compromise becomes parameter-only: an attacker who controls a checkpoint can backdoor it without training access, using as little as 1% of the data or a surrogate dataset.
Fine-tuning on clean data will not cleanse the model, because the injected head's parameters receive zero gradient while the backdoor is dormant; the paper's experiments show attack success remains high after fine-pruning.
Four standard defenses—STRIP, Neural Cleanse, fine-pruning, and RAP—fail to flag or remove the backdoor, with reported false-acceptance rates high enough that most poisoned inputs are classified as clean.
The attack transfers across transformer families (ViT, DeiT, BERT) and across vision and text tasks, with clean accuracy loss usually under a few percent.
Because the added logit $a$ is chosen as a quantile of the clean logit gap, the trigger can be made strong enough for high attack success while staying under the detection threshold of output-logit-based defenses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the zero-gradient isolation claim implies a direct test: measure the gradient norm of the injected head's parameters during clean fine-tuning; a nonzero value would invalidate the fine-tuning-resistance guarantee.
The authors leave implicit that the same dormant-head injection recipe could be ported to decoder-only language models, where attention heads are also redundant; the constant-logit identity would need re-derivation under causal masking.
If the isolation assumption is taken at face value, defenses should stop trying to remove the backdoor by fine-tuning and instead inspect or reset the injected head's output projection, since the paper's own fine-pruning results show clean accuracy collapsing before attack success rate drops.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know about this paper. First, the core mechanism—prune the least important attention head and drop in a pre-trained single-head malicious transformer—is genuinely new as far as I can tell, and it works: ASRs are high, clean accuracy loss is modest, and the authors test across CV and NLP against several defenses. Second, the headline claim that this is a retraining-free attack that leaves the architecture unchanged is contradicted by their own construction. Eq. (4) replaces the single LayerNorm over all heads with three independent LayerNorms, and Eq. (8) rewires the residual connection to route only the injected head's segment. Those are structural edits to the forward computation graph, not parameter changes. The threat model explicitly says the attacker is 'unable to modify the architecture,' so the central selling point does not hold as stated.

The formal analysis is weaker than advertised. Theorem 1 says the modified logits are p + a for the target class and unchanged otherwise. But the malicious head is trained to output a for trigger inputs and 0 for clean ones, so the theorem largely restates the training objective. The 'a' itself is chosen from a fitted Gaussian quantile, which is tuned rather than derived. The fine-tuning resistance claim is asserted via a zero-gradient argument that requires the injected head to be perfectly dormant and fully isolated; no gradient path through LayerNorm, residual connections, or the output layer is analyzed. If any clean activation leaks, the guarantee collapses.

The empirical work is the strong part. The data-dependency table is interesting—the attack works even with zero samples of the target dataset, using a malicious head trained on a surrogate. Defense results show real resilience, but the evaluation has gaps: Neural Cleanse is only run for the patch trigger, RAP only for one rare-word trigger and required adjusting the threshold settings, and the fine-pruning results show clean accuracy dropping as low as ~49%, which is arguably a successful defense if one accepts utility loss. There are also sign inconsistencies between Table 1 and Table 2 for CAD on the same settings, which need correcting.

Who should read this? Security folks working on weight-poisoning and transformer robustness will find the mechanism worth thinking about. The paper deserves a serious referee, but it needs major revision that either drops the 'no architecture change' claim and reframes the method as a load-time attack that rewrites parts of the forward graph, or shows a true parameter-only implementation that avoids the LayerNorm and residual rewrites. As it stands, the mismatch between claims and method is too large to accept.

Referee Report

5 major / 5 minor

Summary. The paper proposes HPMI, a backdoor attack on pre-trained transformers that claims to be retraining-free and architecture-preserving. The method prunes the least important attention head, trains a single-head transformer to be dormant on clean inputs and active on trigger inputs, injects it in place of the pruned head, and adjusts output-layer weights so the malicious signal boosts the target class. The manuscript presents Theorem 1 as a formal guarantee that the injected head adds a constant a to the target logit and leaves other logits unchanged, argues that this resists fine-tuning and defenses, and reports experiments on ViT/DeiT/BERT models showing ASR ≥ 99.55% and evasion of STRIP, Neural Cleanse, fine pruning, and RAP.

Significance. If the central claims held, HPMI would be a practically important supply-chain backdoor: it would require no retraining and no architectural modification, use only a small data subset, and come with a theoretical robustness argument. The empirical scope is broad and the paper reports standard metrics (ASR, CA, CAD) across multiple models and datasets. However, the manuscript's core 'no architecture change' claim is contradicted by its own construction: Eq. (4) and Eq. (8) alter LayerNorm and residual computation, which are structural changes to the forward graph. The main theorem is essentially a restatement of the malicious head's training objective under an engineered isolation assumption, and the fine-tuning resistance claim is asserted without derivation. These are load-bearing issues, not presentation problems.

major comments (5)

[§3.2 vs §4.1, Eq. (4) and Eq. (8)] The threat model states the attacker is 'unable to modify the architecture', but HPMI requires replacing the standard LayerNorm with segment-wise normalization (Eq. 4) and changing the residual connection so that the i-th segment receives its own previous hidden state rather than the full hidden state (Eq. 8). These are modifications to the computation graph and cannot be realized by parameter edits alone in a standard TransformerEncoderLayer. The same applies to 'pruning the corresponding modules in the subsequent feed-forward module' (Section 4.1), which requires rewiring cross-term weights in the FFN. As specified, the method cannot be executed under the stated threat model, so the headline 'does not alter the model's architecture' is unsupported.
[§4.4, Theorem 1 and proof] The theorem's conclusion—~p_yhat = p_yhat + a and ~p_y = p_y for all other y—is exactly the behavior that the malicious head's objective (Section 4.2) and the output-layer adjustment are designed to produce. The proof's key steps (Eqs. 8–10) assume the injected head is fully isolated, but that isolation is created by the architectural modifications of Eq. (4)/(8), not by parameter changes in the original model. The proof also never specifies how the feed-forward network is pruned so that the i-th segment evolves as z*_l; without that specification, the theorem is a restatement of the construction's intended behavior rather than a derivation for a standard transformer.
[§4.4, Stealthy analysis] The claim that fine-tuning cannot remove the backdoor because 'the loss gradient with respect to the backdoor parameters becomes zero during training' is asserted without derivation. It requires exact zero output of the malicious head on clean inputs and no gradient path through LayerNorm, residual connections, or the output-layer weights. The paper only says the head is 'unlikely to be activated' (Section 4.2), which is insufficient. Even if the malicious head's internal parameters were frozen, fine-tuning could update the output projection that maps its representation to logits, reducing the effective value of a. No fine-tuning experiments are reported, so this robustness claim is not established.
[§4.4, Gaussian quantile selection] The choice a=⌈quantile(τ)+k⌉ rests on an unstated assumption that the difference between the maximum logit and the target logit follows a Gaussian distribution. No distribution fit, sensitivity analysis, or procedure for setting τ and k is given. These parameters control both attack success and detectability (e.g., STRIP), so this is central to the stealthiness claim. Additionally, Table 3 reports 100% ASR with ρ=0, where the malicious head was trained on GTSRB and inserted into a CIFAR-10 model; this is not explained and suggests the quantile mechanism may not be essential to the observed effectiveness.
[§5.3, Table 4] The claim of bypassing four defenses is only partially supported. Neural Cleanse is evaluated only for patch triggers, and RAP only for rare-word text triggers. The fine-pruning results show CA collapsing to 49.19% in one configuration (BERT-B on SST-2), meaning the defense destroyed model utility even though ASR remained high; interpreting this as 'bypassing' the defense requires more discussion. The FAR metric in Table 4 is also not defined in the text, making some entries hard to interpret. The broad 'bypasses four advanced defense mechanisms' conclusion is therefore overstated.

minor comments (5)

[Tables 1–2] CAD sign conventions are inconsistent: the text reports pruning CAD as -4.97% for ViT-B on CIFAR-10, but Table 2 shows positive values, and Table 1 uses negative CAD in some rows and positive in others. Please standardize the sign convention.
[Eq. (7)–(9)] The notation is confusing: the proof uses both ilde z*_{l-1} and ilde z^{head_i}_{l-1} without clearly distinguishing the malicious model's internal state from the target model's i-th segment. Define these quantities explicitly.
[Section 4.2] The malicious head objective refers to 'Equation??' — the equation number is missing. Also, the balance parameter λ is introduced but its role in the objective is not explained.
[Section 5.2, Table 3] The ρ=0 row is a striking claim (a GTSRB-trained head transfers to CIFAR-10 with 100% ASR). This deserves a dedicated explanation or experiment rather than a single row in a table.
[General] The code is only promised 'upon acceptance.' Given the unusual architectural modifications, detailed pseudocode or a public implementation is needed for reproducibility.

Circularity Check

2 steps flagged · score 6.0 of 10

Theorem 1 restates the malicious head's training objective as a prediction; the defense 'guarantees' are calibrated from the same fitted value a.

self definitional [Section 4.2 (Malicious Head Generation) and Theorem 1, Section 4.4]
"The objective of malicious head generation is as follows: min_{w*} E_{(x,y)∈Db} [ (f∗(G∗(x;w∗))−0)^2 + (f∗(G∗(x˜;w∗))−a)^2 ] ... Theorem 1 ... Then, the modified logits satisfy: p˜_ŷ = p_ŷ + a and p˜_y = p_y for all y ≠ ŷ, where a > 0 is the additive contribution introduced by the malicious head."

Theorem 1's conclusion is exactly the objective used to train the malicious head: f*(G*(x)) ≈ 0 on clean inputs and f*(G*(x~)) ≈ a on backdoored inputs. The injection procedure then wires this scalar into the target-class logit. The proof shows only that the revised LayerNorm/residual make the head run as a standalone malicious transformer; it never derives the value a from the original model. The 'prediction' that the target logit increases by a is the fitted training target restated as a theorem.
fitted input called prediction [Section 4.4, Stealthy analysis paragraph]
"In HPMI, we assume the difference between the maximum logit and the logit corresponding to the target label follows Gaussian distribution. Then, we select the added logit as the quantile function of a specific percentile, i.e., a = ⌈quantile(τ) + k⌉, where τ is the percentile and k ≥ 0 is an offset to make the added logit a to be the minimum but large enough. The specific values of τ and k are determined by the pre-training process of the malicious head."

The attack magnitude a—the very quantity Theorem 1 treats as given and that the defense-bypass argument relies on—is fitted from the malicious head's own pre-training outputs via a Gaussian quantile. The statement 'a large enough a can make our attack successful' is therefore a calibration of the attack's own parameter, not an independent first-principles guarantee. This is a fitted input renamed as a theoretical prediction.

full rationale

The empirical evaluation in Section 5 is largely self-contained: ASR and CA are measured on held-out test data, and the comparisons with data poisoning provide independent evidence that the attack works in practice. However, the paper's theoretical centerpiece is circular/tautological. Section 4.2 explicitly trains the malicious head to output ≈0 on clean inputs and ≈a on triggered inputs, and Section 4.3 wires that output into the target-class logit. Theorem 1 then 'proves' exactly that behavior (target logit += a, other logits unchanged). The induction in Eqs. 6–10 only establishes that the modified LayerNorm/residual make the injected head behave as the standalone malicious head; it does not derive the attack's effectiveness or the value of a from any property of the original transformer. The selection of a is itself a Gaussian-quantile fit from the malicious head's pretraining outputs, so the subsequent fine-tuning-resistance and defense-bypass claims inherit this fitted value. Separately, the paper's assertion that HPMI 'does not alter the model's architecture' is contradicted by its own construction—Eq. 4 changes LayerNorm into three independent normalizations and Eq. 8 changes the residual connection to be segment-wise—but that is a correctness/consistency issue rather than a circularity issue. No load-bearing self-citations were found. Overall score 6: one or more central 'predictions' reduce by construction to the fitted training objective.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central method depends on several hand-chosen parameters (notably the added logit a, determined through a Gaussian quantile fit) and on unproven behavioral premises (exact dormancy and exact isolation). These are the main items the paper pulls from its own construction rather than from established theory or external benchmarks.

free parameters (5)

a (added target logit) = ceil(quantile(τ)+k), values implied by Section 4.4
Chosen to make the backdoor effective while evading logit-based defenses; controls the success/stealth tradeoff.
τ (quantile percentile) and k (offset) = determined during malicious-head pre-training
Define a; no principled derivation is given.
α blend ratio = 0.2
Hand-chosen trigger blending ratio for image triggers.
ρ training-data proportion = 0.2 (varied down to 0)
Proportion of original data used to pre-train the malicious head; arbitrary.
λ objective weight = 1
Mentioned in Section 5.1 but the objective as written contains no λ; undefined hyperparameter.

assumptions (5)

domain assumption Attacker has full white-box access to pretrained parameters and a subset of target or surrogate data.
Defines the attack scenario in Section 3.2; if access is weaker, the attack may not apply.
domain assumption Pruning the least important head causes negligible performance loss.
Relies on transformer over-parameterization and prior head-pruning findings; empirically validated but not guaranteed.
ad hoc to paper The malicious head can be trained to be exactly dormant on clean inputs and exactly activated by trigger inputs.
Core behavioral premise in Section 4.2; the theorem and stealth proof assume it. It is not proven, only encouraged by the loss.
ad hoc to paper The revised LayerNorm and reconnection fully isolate the injected head so clean outputs equal the pruned model and gradients are zero.
Unproven architectural isolation in Eq. 4 and Section 4.4; gradients from the residual stream and output layer could reach the injected path.
ad hoc to paper The difference between the maximum logit and the target logit follows a Gaussian distribution.
Used in Section 4.4 to select a via a quantile; no empirical justification is provided.

invented entities (1)

Malicious head G*(x; w*)
purpose: A separately pretrained single-head transformer injected into the pruned position to create a backdoor channel.
New component introduced by the attack; no falsifiable external prediction beyond the attack's own training objective.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Pruning and Malicious Injection: A Retraining-Free Backdoor Attack on Transformer Models." pith.science (2026). https://pith.science/paper/534EXLTK

@misc{pith2026250810243,
  author       = {Pith},
  title        = {Pith review of: Pruning and Malicious Injection: A Retraining-Free Backdoor Attack on Transformer Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/534EXLTK}},
  note         = {Machine review of arXiv:2508.10243}
}

read the original abstract

Transformer models have demonstrated exceptional performance and have become indispensable in computer vision (CV) and natural language processing (NLP) tasks. However, recent studies reveal that transformers are susceptible to backdoor attacks. Prior backdoor attack methods typically rely on retraining with clean data or altering the model architecture, both of which can be resource-intensive and intrusive. In this paper, we propose Head-wise Pruning and Malicious Injection (HPMI), a novel retraining-free backdoor attack on transformers that does not alter the model's architecture. Our approach requires only a small subset of the original data and basic knowledge of the model architecture, eliminating the need for retraining the target transformer. Technically, HPMI works by pruning the least important head and injecting a pre-trained malicious head to establish the backdoor. We provide a rigorous theoretical justification demonstrating that the implanted backdoor resists detection and removal by state-of-the-art defense techniques, under reasonable assumptions. Experimental evaluations across multiple datasets further validate the effectiveness of HPMI, showing that it 1) incurs negligible clean accuracy loss, 2) achieves at least 99.55% attack success rate, and 3) bypasses four advanced defense mechanisms. Additionally, relative to state-of-the-art retraining-dependent attacks, HPMI achieves greater concealment and robustness against diverse defense strategies, while maintaining minimal impact on clean accuracy.

Figures

Figures reproduced from arXiv: 2508.10243 by the authors.

**Figure 2.** The backdoor is injected by inserting poisoning parameters to the pruned neurons in multi-head self-attention and other modules. We revise the LayerNorm into LN˜ as in Equation 4. The key challenge lies in constructing a backdoor pathway that fulfills both required conditions at once. To tackle this, we introduce a backdoor attack that with Head Pruning and Malicious Injection as in [PITH_FULL_IMAGE:figures/fu… view at source ↗

**Figure 3.** All triggers for CV and NLP parts. In subfigure (a), the random noise is the same size as one patch. In subfigure (b), HelloKitty is blending over clean input with a blend ratio α = 0.2. The rare words are injected at random positions into text inputs as shown in subfigure (c). Target classes. In our framework, the malicious head is trained as a binary classifier and can be injected into the target model with an … view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Clean Accuracy after head pruning. Red dash line indicates the Benign Clean [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

**Figure 5.** Figure 5: Comparison of attack and defense performance between HPMI and DP. Met [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Thermospheric Density, Composition, and Temperature from GOES-R/SUVI Solar Occultations
astro-ph.EP 2025-08 unverdicted novelty 6.0 of 10

GOES-R/SUVI solar occultations yield new thermospheric O and N2 density and temperature profiles with quantified uncertainties, available from 2018 onward.

Reference graph

Works this paper leans on

7 extracted references · 2 canonical work pages · cited by 1 Pith paper

[7]

Neural Cleanse: Identifying and mitigating backdoor attacks in neural networks, in: Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP), IEEE. pp. 707–723. Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonza- lez, J., Keutzer, K., Vajda, P., 2020. Visual transformers: Token-based im- agerepresentationandprocessingforc...

arXiv 2019
[255]

Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018

doi:10.1109/CVPR.2009.5206848. Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 . Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Un- terthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.,

arXiv 2009
[2018]

https://cdn.openai.com/research-covers/language-unsupervised/ language_understanding_paper.pdf

Improving language understanding by generative pre-training. https://cdn.openai.com/research-covers/language-unsupervised/ language_understanding_paper.pdf. OpenAI Technical Report. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.,

work page
[2019]

OpenAI blog 1, 9

Language models are unsupervised multitask learners. OpenAI blog 1, 9. Saha, A., Subramanya, A., Pirsiavash, H., 2020. Hidden trigger backdoor attacks, in: Proceedings of the AAAI conference on artificial intelligence, pp. 11957–11965. Shafahi, A., Huang, W.R., Najibi, M., Suciu, O., Studer, C., Dumitras, T., Goldstein, T., 2018. Poison frogs! targeted cl...

work page 2020
[2020]

arXiv preprint arXiv:2010.11929

An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 . Fu, Y., Zhang, S., Wu, S., Wan, C., Lin, Y., 2022. Patch-fool: Are vi- sion transformers always robust against adversarial perturbations? arXiv preprint arXiv:2203.08392 . Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D.C., Nepal, S., 2019. STR...

arXiv 2010
[2021]

Training data-efficient image transformers & distillation through attention, in: International conference on machine learning, PMLR. pp. 10347–10357. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., 2017. Attention is all you need. Advances in neural information processing systems 30. Voita, E., Talb...

arXiv 2017
[2023]

24595–24604

Architectural backdoors in neural networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24595–24604. Cao, B., Jia, J., Hu, C., Guo, W., Xiang, Z., Chen, J., Li, B., Song, D., 2024. Data free backdoor attacks, in: The Thirty-eighth Annual Conference on Neural Information Processing Systems. 20 Chen, X., Liu, C...

arXiv 2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [7]

Neural Cleanse: Identifying and mitigating backdoor attacks in neural networks, in: Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP), IEEE. pp. 707–723. Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonza- lez, J., Keutzer, K., Vajda, P., 2020. Visual transformers: Token-based im- agerepresentationandprocessingforc...

arXiv 2019

[2] [255]

Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018

doi:10.1109/CVPR.2009.5206848. Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 . Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Un- terthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.,

arXiv 2009

[3] [2018]

https://cdn.openai.com/research-covers/language-unsupervised/ language_understanding_paper.pdf

Improving language understanding by generative pre-training. https://cdn.openai.com/research-covers/language-unsupervised/ language_understanding_paper.pdf. OpenAI Technical Report. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.,

work page

[4] [2019]

OpenAI blog 1, 9

Language models are unsupervised multitask learners. OpenAI blog 1, 9. Saha, A., Subramanya, A., Pirsiavash, H., 2020. Hidden trigger backdoor attacks, in: Proceedings of the AAAI conference on artificial intelligence, pp. 11957–11965. Shafahi, A., Huang, W.R., Najibi, M., Suciu, O., Studer, C., Dumitras, T., Goldstein, T., 2018. Poison frogs! targeted cl...

work page 2020

[5] [2020]

arXiv preprint arXiv:2010.11929

An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 . Fu, Y., Zhang, S., Wu, S., Wan, C., Lin, Y., 2022. Patch-fool: Are vi- sion transformers always robust against adversarial perturbations? arXiv preprint arXiv:2203.08392 . Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D.C., Nepal, S., 2019. STR...

arXiv 2010

[6] [2021]

Training data-efficient image transformers & distillation through attention, in: International conference on machine learning, PMLR. pp. 10347–10357. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., 2017. Attention is all you need. Advances in neural information processing systems 30. Voita, E., Talb...

arXiv 2017

[7] [2023]

24595–24604

Architectural backdoors in neural networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24595–24604. Cao, B., Jia, J., Hu, C., Guo, W., Xiang, Z., Chen, J., Li, B., Song, D., 2024. Data free backdoor attacks, in: The Thirty-eighth Annual Conference on Neural Information Processing Systems. 20 Chen, X., Liu, C...

arXiv 2024