Random noise in pre-training data has a surprisingly small effect on language model next-token loss, but can still hurt downstream tasks; a new local gradient matching loss partially counteracts this.
FedSOV: Federated Model Secure Ownership Verification with Unforgeable Signature
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
Federated learning allows multiple parties to collaborate in learning a global model without revealing private data. The high cost of training and the significant value of the global model necessitates the need for ownership verification of federated learning. However, the existing ownership verification schemes in federated learning suffer from several limitations, such as inadequate support for a large number of clients and vulnerability to ambiguity attacks. To address these limitations, we propose a cryptographic signature-based federated learning model ownership verification scheme named FedSOV. FedSOV allows numerous clients to embed their ownership credentials and verify ownership using unforgeable digital signatures. The scheme provides theoretical resistance to ambiguity attacks with the unforgeability of the signature. Experimental results on computer vision and natural language processing tasks demonstrate that FedSOV is an effective federated model ownership verification scheme enhanced with provable cryptographic security.
citation-role summary
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Do we really have to filter out random noise in pre-training data for language models?
Random noise in pre-training data has a surprisingly small effect on language model next-token loss, but can still hurt downstream tasks; a new local gradient matching loss partially counteracts this.