Transformers performing in-context learning implicitly implement gradient descent, ridge regression, and least-squares predictors for linear models, with behavior shifting based on model depth, width, and data noise.
Title resolution pending
4 Pith papers cite this work, alongside 52 external citations. Polarity classification is still indexing.
4
Pith papers citing it
52
external citations · external index
representative citing papers
Model organism interpretability depends strongly on training methodology, with integrated training yielding less interpretable MOs than post-hoc SFT or DPO.
Emergent abilities are capabilities present in large language models but absent in smaller ones and cannot be predicted by extrapolating smaller model performance.
PRISM shows that prompt engineering and selective ICL often outperform complex multi-agent systems on financial retrieval benchmarks while remaining training-free and achieving competitive NDCG@5 scores.
citing papers explorer
-
What learning algorithm is in-context learning? Investigations with linear models
Transformers performing in-context learning implicitly implement gradient descent, ridge regression, and least-squares predictors for linear models, with behavior shifting based on model depth, width, and data noise.
-
The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology
Model organism interpretability depends strongly on training methodology, with integrated training yielding less interpretable MOs than post-hoc SFT or DPO.
-
Emergent Abilities of Large Language Models
Emergent abilities are capabilities present in large language models but absent in smaller ones and cannot be predicted by extrapolating smaller model performance.
-
PRISM: Prompt-Refined In-Context System Modelling for Financial Retrieval
PRISM shows that prompt engineering and selective ICL often outperform complex multi-agent systems on financial retrieval benchmarks while remaining training-free and achieving competitive NDCG@5 scores.