A backdoor watermark for LLMs using passthrough layers trained to output high-entropy text on a private key, with near-perfect extraction in benign settings but with layer-removal robustness contradicted by the paper's own results.
Watermarking Images in Self-Supervised Latent Spaces
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
We revisit watermarking techniques based on pre-trained deep networks, in the light of self-supervised approaches. We present a way to embed both marks and binary messages into their latent spaces, leveraging data augmentation at marking time. Our method can operate at any resolution and creates watermarks robust to a broad range of transformations (rotations, crops, JPEG, contrast, etc). It significantly outperforms the previous zero-bit methods, and its performance on multi-bit watermarking is on par with state-of-the-art encoder-decoder architectures trained end-to-end for watermarking. The code is available at github.com/facebookresearch/ssl_watermarking
fields
cs.CL 1years
2024 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers
A backdoor watermark for LLMs using passthrough layers trained to output high-entropy text on a private key, with near-perfect extraction in benign settings but with layer-removal robustness contradicted by the paper's own results.