A recurrent-depth architecture enables language models to improve reasoning performance by iterating computation in latent space, achieving gains equivalent to much larger models on benchmarks.
Lessons on parameter sharing across layers in transformers
4 Pith papers cite this work, alongside 24 external citations. Polarity classification is still indexing.
4
Pith papers citing it
24
external citations · external index
representative citing papers
A 2x2 ablation shows repeated shared access enables grokking while addressable memory (not recurrence) enables edit propagation in transformer variants on synthetic KG QA.
RePAIR learns chess board representations by masking latent states and using a predictor to repair gaps, producing clusters of chess concepts and enabling movement reasoning from self-supervision alone.
A 1.4B and a 2.6B looped (weight-tied, recurrent-depth) language model trained on 7.7T tokens match or exceed several 4B–8B transformer baselines on selected reasoning benchmarks.
citing papers explorer
-
Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
A recurrent-depth architecture enables language models to improve reasoning performance by iterating computation in latent space, achieving gains equivalent to much larger models on benchmarks.
-
Repeated Shared Access Enables Grokking, but Edit Propagation Depends on an Addressable Memory
A 2x2 ablation shows repeated shared access enables grokking while addressable memory (not recurrence) enables edit propagation in transformer variants on synthetic KG QA.
-
RePAIR: Predictive Self-Supervised Representation Learning in Chess
RePAIR learns chess board representations by masking latent states and using a predictor to repair gaps, producing clusters of chess concepts and enabling movement reasoning from self-supervision alone.
-
Scaling Latent Reasoning via Looped Language Models
A 1.4B and a 2.6B looped (weight-tied, recurrent-depth) language model trained on 7.7T tokens match or exceed several 4B–8B transformer baselines on selected reasoning benchmarks.