A Practical Guide to Multi-Objective Reinforcement Learning and Planning
read the original abstract
Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
FlowCompile: An Optimizing Compiler for Structured LLM Workflows
FlowCompile performs compile-time design space exploration on structured LLM workflows to produce reusable high-quality configuration sets that outperform routing baselines with up to 6.4x speedup.
-
AETDICE: Unified Framework and Offline Optimization for Nonlinear Multi-Objective RL
AET framework unifies SER and ESR criteria for nonlinear MORL; AETDICE enables offline optimization via DICE-style estimation in augmented state space.
-
Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimization
An adaptive smooth Tchebycheff controller for multi-objective RL lets agents reach non-convex Pareto regions in robotic tasks while avoiding the instability of static non-linear scalarizations.
-
Coachable agents for interactive gameplay
A framework combining universal value function approximators with targeted training scenarios and data augmentation produces RL agents that adapt to user-specified styles in real time across video games and humanoid d...
-
A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets
A single preference-conditioned policy achieves unique and Lipschitz-continuous Pareto coverage in multi-objective MDPs via a new mirror-descent policy iteration algorithm with O(1/k) convergence.
-
Sampling-Based Coordination-Informed Multi-Objective Multi-Robot Reinforcement Learning
CIMORL framework with sampling variants (TS and MPPI) uses privileged training for decentralized multi-objective multi-robot RL, reporting 21.2% hypervolume gain over baselines in cooperative and adversarial tests.
-
A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systems
PRL-PUTS casts utility-weight tuning as a one-step value-based RL task and uses scalarization-parameter Pareto sweeping at inference time to generate and govern a family of policies, reporting +0.13% lift in successfu...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.