Contextual bandits with general function approximation can achieve regret scaling with cumulative reward variance and only logarithmically with the reward range, using Catoni robust mean estimators, with a matching lower bound for the leading term.
Online learning in MDPs with side information
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We study online learning of finite Markov decision process (MDP) problems when a side information vector is available. The problem is motivated by applications such as clinical trials, recommendation systems, etc. Such applications have an episodic structure, where each episode corresponds to a patient/customer. Our objective is to compete with the optimal dynamic policy that can take side information into account. We propose a computationally efficient algorithm and show that its regret is at most $O(\sqrt{T})$, where $T$ is the number of rounds. To best of our knowledge, this is the first regret bound for this setting.
fields
stat.ML 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Catoni Contextual Bandits are Robust to Heavy-tailed Rewards
Contextual bandits with general function approximation can achieve regret scaling with cumulative reward variance and only logarithmically with the reward range, using Catoni robust mean estimators, with a matching lower bound for the leading term.