Metalearning with Hebbian Fast Weights

Tsendsuren Munkhdalai , Adam Trischler

Authors on Pith no claims yet

classification 💻 cs.NE cs.AIcs.LGstat.ML

keywords learningmodelone-shotweightsfasthebbianmetalearningrepresentations

read the original abstract

We unify recent neural approaches to one-shot learning with older ideas of associative memory in a model for metalearning. Our model learns jointly to represent data and to bind class labels to representations in a single shot. It builds representations via slow weights, learned across tasks through SGD, while fast weights constructed by a Hebbian learning rule implement one-shot binding for each new task. On the Omniglot, Mini-ImageNet, and Penn Treebank one-shot learning benchmarks, our model achieves state-of-the-art results.

This paper has not been read by Pith yet.

discussion (0)

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Kimi Linear: An Expressive, Efficient Attention Architecture
cs.CL 2025-10 unverdicted novelty 6.0

Kimi Linear hybridizes linear attention with a new KDA module to beat full attention on tasks while slashing KV cache by 75% and speeding decoding up to 6x.
Where to Bind Matters: Hebbian Fast Weights in Vision Transformers for Few-Shot Character Recognition
cs.NE 2026-04 unverdicted novelty 4.0

Placing one Hebbian fast-weight module after the final stage of Swin-Tiny achieves 96.2% accuracy on 5-way 1-shot Omniglot classification, outperforming the non-Hebbian baseline by 0.3 points.