A decoding-time intervention that substitutes a small model's probabilities for common function words into a large model's output reduces exact training-data recall by up to 10x with minimal measured quality loss.
Physics of language models: Part 3.3, knowledge capacity scaling laws
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
A Lightweight Method to Disrupt Memorized Sequences in LLM
A decoding-time intervention that substitutes a small model's probabilities for common function words into a large model's output reduces exact training-data recall by up to 10x with minimal measured quality loss.