RSAE replaces fixed SAE encoder activations (ReLU, JumpReLU, TopK) with trainable rational functions, initialized from baselines and fine-tuned to improve reconstruction and downstream metrics on language-model residual streams.
Recurrent rational networks.arXiv preprint arXiv:2102.09407, 2021a
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 3roles
background 1polarities
background 1representative citing papers
TeLAPA preserves behaviorally diverse policy neighborhoods in a shared latent space, improving MiniGrid continual RL transfer, revisit recovery, and retention over single-model preservation.
Smooth-Leaky and Randomized Smooth-Leaky activations mitigate loss of plasticity in continual learning by targeting negative-branch shape and saturation behavior.
citing papers explorer
-
Rational Sparse Autoencoder
RSAE replaces fixed SAE encoder activations (ReLU, JumpReLU, TopK) with trainable rational functions, initialized from baselines and fine-tuned to improve reconstruction and downstream metrics on language-model residual streams.
-
Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
TeLAPA preserves behaviorally diverse policy neighborhoods in a shared latent space, improving MiniGrid continual RL transfer, revisit recovery, and retention over single-model preservation.
-
Activation Function Design Sustains Plasticity in Continual Learning
Smooth-Leaky and Randomized Smooth-Leaky activations mitigate loss of plasticity in continual learning by targeting negative-branch shape and saturation behavior.