REVIEW 5 cited by
Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Reward shaping is a technique in reinforcement learning that addresses the sparse-reward problem by providing more frequent and informative rewards. We introduce a self-adaptive and highly efficient reward shaping mechanism that incorporates success rates derived from historical experiences as shaped rewards. The success rates are sampled from Beta distributions, which dynamically evolve from uncertain to reliable values as data accumulates. Initially, the shaped rewards exhibit more randomness to encourage exploration, while over time, the increasing certainty enhances exploitation, naturally balancing exploration and exploitation. Our approach employs Kernel Density Estimation (KDE) combined with Random Fourier Features (RFF) to derive the Beta distributions, providing a computationally efficient, non-parametric, and learning-free solution for high-dimensional continuous state spaces. Our method is validated on various tasks with extremely sparse rewards, demonstrating notable improvements in sample efficiency and convergence stability over relevant baselines.
Forward citations
Cited by 5 Pith papers
-
Skill Expansion and Composition in Parameter Space
PSEC shows that weighting and summing LoRA skill modules inside a diffusion policy network outperforms composing the same skills in action or noise space across D4RL, DSRL, DMC, and Meta-World tasks.
-
Bootstrapped Reward Shaping
Using the agent's current value estimate as a potential-based shaping function converges in tabular RL and accelerates DQN on Atari.
-
Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback
Training a small multi-task reward-shaping network and adding it to the RLHF reward is claimed to improve LLaMA-3-8B alignment across four benchmarks, but the supporting theory is not established.
-
Exploration by Random Distribution Distillation
RDD distills a Gaussian-random target and rewards the squared deviation of the predictor from the target mean, giving an unbiased estimator of 1/n visits that unifies RND and pseudo-count exploration.
-
Information-Based Exploration via Random Features for Reinforcement Learning
Random-feature Gaussian-process information gain is turned into a closed-form exploration bonus for PPO that matches RND/VIME/#Explo on 12 control, navigation, and sparse-locomotion tasks, with error bounds on the app...
Discussion (0). Continue with ORCID to comment.