Reward shaping based on SHAP/LIME token attributions, with weights optimized by Bayesian optimization, improves RLHF training speed and downstream win rates while preserving the optimal policy.
Noisy-Input Entropy Search for Efficient Robust Bayesian Optimization
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We consider the problem of robust optimization within the well-established Bayesian optimization (BO) framework. While BO is intrinsically robust to noisy evaluations of the objective function, standard approaches do not consider the case of uncertainty about the input parameters. In this paper, we propose Noisy-Input Entropy Search (NES), a novel information-theoretic acquisition function that is designed to find robust optima for problems with both input and measurement noise. NES is based on the key insight that the robust objective in many cases can be modeled as a Gaussian process, however, it cannot be observed directly. We evaluate NES on several benchmark problems from the optimization literature and from engineering. The results show that NES reliably finds robust optima, outperforming existing methods from the literature on all benchmarks.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Learning Explainable Dense Reward Shapes via Bayesian Optimization
Reward shaping based on SHAP/LIME token attributions, with weights optimized by Bayesian optimization, improves RLHF training speed and downstream win rates while preserving the optimal policy.