LLM next-state prediction accuracy degrades with larger state spaces and sparser transitions, with state tracking distributed across several attention heads.
Selecting the State-Representation in Reinforcement Learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The problem of selecting the right state-representation in a reinforcement learning problem is considered. Several models (functions mapping past observations to a finite set) of the observations are given, and it is known that for at least one of these models the resulting state dynamics are indeed Markovian. Without knowing neither which of the models is the correct one, nor what are the probabilistic characteristics of the resulting MDP, it is required to obtain as much reward as the optimal policy for the correct model (or for the best of the correct models, if there are several). We propose an algorithm that achieves that, with a regret of order T^{2/3} where T is the horizon time.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Scaling Laws for State Dynamics in Large Language Models
LLM next-state prediction accuracy degrades with larger state spaces and sparser transitions, with state tracking distributed across several attention heads.