WriteSAE introduces sparse autoencoders with rank-1 matrix atoms for recurrent state updates, allowing replacement tests that outperform deletion on 92.4% of positions and a formula predicting logit changes with R²=0.98.
Arnab Sen Sharma, David Atkinson, and David Bau
5 Pith papers cite this work, alongside 379 external citations. Polarity classification is still indexing.
5
Pith papers citing it
379
external citations · OpenAlex
citation-role summary
background 2
citation-polarity summary
verdicts
UNVERDICTED 5roles
background 2polarities
background 2representative citing papers
Fast-Slow Training uses context optimization as fast weights alongside parameter updates as slow weights to achieve up to 3x better sample efficiency, higher performance, and less catastrophic forgetting than standard RL in continual LLM learning.
Eagle and Finch enhance RWKV with matrix-valued states and dynamic recurrence, trained on a 1.12-trillion-token multilingual corpus, and report competitive performance on standard benchmarks.
Kaczmarz Linear Attention replaces the empirical coefficient in Gated DeltaNet with a key-norm-normalized step size derived from the online regression objective, yielding lower perplexity and better needle-in-haystack performance.
The paper introduces Recursive QLSTM via metacore recursion, numerically tests variants on sequence lengths, and offers theoretical arguments for better temporal propagation.
citing papers explorer
-
WriteSAE: Sparse Autoencoders for Recurrent State
WriteSAE introduces sparse autoencoders with rank-1 matrix atoms for recurrent state updates, allowing replacement tests that outperform deletion on 92.4% of positions and a formula predicting logit changes with R²=0.98.
-
Learning, Fast and Slow: Towards LLMs That Adapt Continually
Fast-Slow Training uses context optimization as fast weights alongside parameter updates as slow weights to achieve up to 3x better sample efficiency, higher performance, and less catastrophic forgetting than standard RL in continual LLM learning.
-
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
Eagle and Finch enhance RWKV with matrix-valued states and dynamic recurrence, trained on a 1.12-trillion-token multilingual corpus, and report competitive performance on standard benchmarks.
-
Kaczmarz Linear Attention
Kaczmarz Linear Attention replaces the empirical coefficient in Gated DeltaNet with a key-norm-normalized step size derived from the online regression objective, yielding lower perplexity and better needle-in-haystack performance.
-
Recursive QLSTM with Dynamic Variational Quantum Circuit Adaptation
The paper introduces Recursive QLSTM via metacore recursion, numerically tests variants on sequence lengths, and offers theoretical arguments for better temporal propagation.