A solvable hierarchical model with power-law feature strengths yields explicit power-law scaling of prediction error through sequential recovery of latent directions by a layer-wise spectral algorithm.
Repetita iuvant: Data repetition allows sgd to learn high-dimensional multi-index functions
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
fields
stat.ML 3years
2026 3representative citing papers
In a Gaussian single-index model, neural reward models recover the hidden direction for β1 above an O(1) threshold and provide tilted-policy value-gap bounds for label-weighted and surrogate-weighted exponential fits.
For multi-index polynomials, the top r eigenspace of the AGOP matrix from KRR recovers the central subspace at sample complexity n ~ d^{p+δ} where p is the degree of the informative component.
citing papers explorer
-
Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
A solvable hierarchical model with power-law feature strengths yields explicit power-law scaling of prediction error through sequential recovery of latent directions by a layer-wise spectral algorithm.
-
How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
In a Gaussian single-index model, neural reward models recover the hidden direction for β1 above an O(1) threshold and provide tilted-policy value-gap bounds for label-weighted and surrogate-weighted exponential fits.
-
Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models
For multi-index polynomials, the top r eigenspace of the AGOP matrix from KRR recovers the central subspace at sample complexity n ~ d^{p+δ} where p is the degree of the informative component.