Power-law LLM scaling laws are largely the aggregate of stepwise token learning events whose heavy-tailed learning-time spectrum reconstructs loss derivatives along step, data, and model axes.
Learning curve theory
10 Pith papers cite this work. Polarity classification is still indexing.
10
Pith papers citing it
citation-role summary
background 2
citation-polarity summary
roles
background 2polarities
background 2representative citing papers
In extensive-width networks, features are recovered sequentially through sharp phase transitions, yielding an effective width k_c that unifies Bayes-optimal generalization error scaling as Θ(k_c d / n).
For a sketched bilinear contrastive model trained by full-batch gradient descent, the expected risk decomposes into irreducible, approximation, bias, and variance terms, with variance set by a product effective dimension.
Critical percolation clusters embedded in high dimensions, combined with taxonomic latent variables, form an analytically tractable synthetic data model whose ground-truth hierarchy can be linearly decoded from network activations.
Strong superposition causes neural loss to scale as the inverse of model dimension due to geometric feature overlaps, explaining scaling laws for broad frequency distributions.
Small transformers on HMM prediction tasks exhibit correlated scaling between performance and linear encoding of belief distributions in residual activations.
Tuning the depth-width ratio positions models in an efficient neural interaction interval that correlates with better generalization under fixed budgets and remains stable with scale.
Introduces Augment Engineering as a six-phase multi-tool orchestration methodology, supported by exploratory statistics from a single-practitioner case study across seven domains.
Position paper claims fixed exponents in scaling laws arise from generic mechanisms while coefficients vary with data and architecture, making the latter the focus for improvements.
citing papers explorer
-
Smooth Scaling Laws Hide Stepwise Token Learning
Power-law LLM scaling laws are largely the aggregate of stepwise token learning events whose heavy-tailed learning-time spectrum reconstructs loss derivatives along step, data, and model axes.
-
Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks
In extensive-width networks, features are recovered sequentially through sharp phase transitions, yielding an effective width k_c that unifies Bayes-optimal generalization error scaling as Θ(k_c d / n).
-
Sketched Linear Contrastive Learning: Approximation, Optimization, and Statistical Scaling
For a sketched bilinear contrastive model trained by full-batch gradient descent, the expected risk decomposes into irreducible, approximation, bias, and variance terms, with variance set by a product effective dimension.
-
Critical Percolation as a Synthetic Data Model for Interpretability
Critical percolation clusters embedded in high dimensions, combined with taxonomic latent variables, form an analytically tractable synthetic data model whose ground-truth hierarchy can be linearly decoded from network activations.
-
Superposition Yields Robust Neural Scaling
Strong superposition causes neural loss to scale as the inverse of model dimension due to geometric feature overlaps, explaining scaling laws for broad frequency distributions.
-
Structure and Scale in Simplicial Sequence Modelling
Small transformers on HMM prediction tasks exhibit correlated scaling between performance and linear encoding of belief distributions in residual activations.
-
Law of Neural Interaction: Depth-Width Shape, Interaction Efficiency, and Generalization
Tuning the depth-width ratio positions models in an efficient neural interaction interval that correlates with better generalization under fixed budgets and remains stable with scale.
-
Augment Engineering: A Methodology for Multi-Tool AI Orchestration Across Professional Domains
Introduces Augment Engineering as a six-phase multi-tool orchestration methodology, supported by exploratory statistics from a single-practitioner case study across seven domains.
-
Neural Scaling Universality: If Exponents Are Fixed, Time to Understand Coefficients
Position paper claims fixed exponents in scaling laws arise from generic mechanisms while coefficients vary with data and architecture, making the latter the focus for improvements.
- Scaling Laws for Dynamic Mini-Batch SGD in Sketched Linear Regression