Gradient and greedy search over token suffixes produces universal, transferable adversarial prompts that elicit objectionable outputs from aligned models including black-box commercial systems.
The Space of Transferable Adversarial Examples
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
abstract
Adversarial examples are maliciously perturbed inputs designed to mislead machine learning (ML) models at test-time. They often transfer: the same adversarial example fools more than one model. In this work, we propose novel methods for estimating the previously unknown dimensionality of the space of adversarial inputs. We find that adversarial examples span a contiguous subspace of large (~25) dimensionality. Adversarial subspaces with higher dimensionality are more likely to intersect. We find that for two different models, a significant fraction of their subspaces is shared, thus enabling transferability. In the first quantitative analysis of the similarity of different models' decision boundaries, we show that these boundaries are actually close in arbitrary directions, whether adversarial or benign. We conclude by formally studying the limits of transferability. We derive (1) sufficient conditions on the data distribution that imply transferability for simple model classes and (2) examples of scenarios in which transfer does not occur. These findings indicate that it may be possible to design defenses against transfer-based attacks, even for models that are vulnerable to direct attacks.
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
Steganographic exfiltration attacks succeed on embedding stores via retrieval-preserving perturbations such as small-angle orthogonal rotation, but an Ed25519-based provenance signature closes the attack class.
Adversarial training via projected gradient descent on the inner maximization problem produces neural networks with substantially improved resistance to a wide range of attacks and establishes security against first-order adversaries as a concrete guarantee.
SAAD adaptively weights adversarial training samples by their transferability to the teacher, yielding higher AutoAttack robustness than prior distillation methods on CIFAR and Tiny-ImageNet without extra compute.
Power side-channel analysis recovers DNN architecture and parameters at 96.5% average accuracy on real embedded devices.
A reproducible pipeline produces physical adversarial traffic signs that successfully attack production-grade traffic sign recognition systems in a real car under black-box conditions.
Optimized 3x3 adversarial image filters based on edge detection generate transferable untargeted attacks on neural networks with 30-80% success using only one pass and far fewer parameters than prior methods.
Random quantum circuits used as adversarial training data reduce successful attack rates on QML models for CIFAR-10 from 89.8% to 68.45% and for CINIC-10 from 94.23% to 78.68%.
BPN adds task-specific beneficial perturbations as biases to neural networks to overcome catastrophic forgetting without storing prior data or expanding the network substantially.
citing papers explorer
-
Universal and Transferable Adversarial Attacks on Aligned Language Models
Gradient and greedy search over token suffixes produces universal, transferable adversarial prompts that elicit objectionable outputs from aligned models including black-box commercial systems.
-
VectorSmuggle: Steganographic Exfiltration in Embedding Stores and a Cryptographic Provenance Defense
Steganographic exfiltration attacks succeed on embedding stores via retrieval-preserving perturbations such as small-angle orthogonal rotation, but an Ed25519-based provenance signature closes the attack class.
-
Towards Deep Learning Models Resistant to Adversarial Attacks
Adversarial training via projected gradient descent on the inner maximization problem produces neural networks with substantially improved resistance to a wide range of attacks and establishes security against first-order adversaries as a concrete guarantee.
-
Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation
SAAD adaptively weights adversarial training samples by their transferability to the teacher, yielding higher AutoAttack robustness than prior distillation methods on CIFAR and Tiny-ImageNet without extra compute.
-
Open DNN Box by Power Side-Channel Attack
Power side-channel analysis recovers DNN architecture and parameters at 96.5% average accuracy on real embedded devices.
-
Fooling a Real Car with Adversarial Traffic Signs
A reproducible pipeline produces physical adversarial traffic signs that successfully attack production-grade traffic sign recognition systems in a real car under black-box conditions.
-
Almost for Free: Crafting Adversarial Examples with Convolutional Image Filters
Optimized 3x3 adversarial image filters based on edge detection generate transferable untargeted attacks on neural networks with 30-80% success using only one pass and far fewer parameters than prior methods.
-
Quantum Patches: Enhancing Robustness of Quantum Machine Learning Models
Random quantum circuits used as adversarial training data reduce successful attack rates on QML models for CIFAR-10 from 89.8% to 68.45% and for CINIC-10 from 94.23% to 78.68%.
-
Beneficial perturbation network for continual learning
BPN adds task-specific beneficial perturbations as biases to neural networks to overcome catastrophic forgetting without storing prior data or expanding the network substantially.