L-layer transformers under Log-ICoT curriculum provably learn k-parity with poly(n) samples and log k stages, matching explicit CoT efficiency without inference overhead.
Transformers learn to implement multi-step gradient descent with chain of thought
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
representative citing papers
Power-law data distributions outperform uniform ones for compositional reasoning by creating asymmetry that lets frequent skill compositions scaffold rare ones with less data.
A 1.4B and a 2.6B looped (weight-tied, recurrent-depth) language model trained on 7.7T tokens match or exceed several 4B–8B transformer baselines on selected reasoning benchmarks.
citing papers explorer
-
Transformers Provably Learn to Internalize Chain-of-Thought
L-layer transformers under Log-ICoT curriculum provably learn k-parity with poly(n) samples and log k stages, matching explicit CoT efficiency without inference overhead.
-
The Power of Power Law: Asymmetry Enables Compositional Reasoning
Power-law data distributions outperform uniform ones for compositional reasoning by creating asymmetry that lets frequent skill compositions scaffold rare ones with less data.
-
Scaling Latent Reasoning via Looped Language Models
A 1.4B and a 2.6B looped (weight-tied, recurrent-depth) language model trained on 7.7T tokens match or exceed several 4B–8B transformer baselines on selected reasoning benchmarks.