Prompt-conditioned LLMs face irreducible error floors from language's limited information capacity and alignment constraints, proven via PAC-Bayes bounds on bilevel cheap-talk games for certain task families.
Limitations of autoregressive models and their alternatives.arXiv preprint arXiv:2010.11939,
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.LG 2years
2026 2verdicts
UNVERDICTED 2representative citing papers
Binary rewards make the set of reward-maximizing policies infinite in policy gradients; KL control selects the filtered base model but misspecification drives collapse to concentrated valid outputs instead.
citing papers explorer
-
On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners
Prompt-conditioned LLMs face irreducible error floors from language's limited information capacity and alignment constraints, proven via PAC-Bayes bounds on bilevel cheap-talk games for certain task families.
-
Binary Rewards and Reinforcement Learning: Fundamental Challenges
Binary rewards make the set of reward-maximizing policies infinite in policy gradients; KL control selects the filtered base model but misspecification drives collapse to concentrated valid outputs instead.