REVIEW 18 cited by
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
A Review of Off-Policy Evaluation in Reinforcement Learning
read the original abstract
Reinforcement learning (RL) is one of the most vibrant research frontiers in machine learning and has been recently applied to solve a number of challenging problems. In this paper, we primarily focus on off-policy evaluation (OPE), one of the most fundamental topics in RL. In recent years, a number of OPE methods have been developed in the statistics and computer science literature. We provide a discussion on the efficiency bound of OPE, some of the existing state-of-the-art OPE methods, their statistical properties and some other related research directions that are currently actively explored.
Forward citations
Cited by 18 Pith papers
-
Estimating Causal Effects from Data Generated by Stochastic Algorithms
Logging the features and relative probability of one unexposed item alongside the exposed item identifies causal effects of content features from stochastic algorithms even with unobserved confounders.
-
Fitted Occupancy-Ratio Evaluation without Bellman Completeness
FORE estimates discounted occupancy ratios by iterating KL-projected adjoint Bellman updates, achieving convergence under ratio realizability alone without Bellman completeness.
-
Off-Policy Evaluation for Missingness-Aware Policies in MDPs with Rewards Missing Not at Random
Identifies full-data conditional mean rewards under MNAR missingness via shadow variables and a bridge function, then builds a consistent FQE-style OPE estimator for missingness-aware policies.
-
Off-Policy Evaluation with Strategic Agents via Local Disclosure
Local disclosure via post-hoc explanations enables consistent doubly robust estimation of policy value in one-shot OPE with strategic agents by recovering pre-strategic covariates under a conditional log-normal cost s...
-
Bandit Simulation for Average Reward Inference
BSI fits an environment simulator from bandit data and propagates parameter uncertainty to produce asymptotically valid confidence intervals for mean reward under arbitrary evaluation policies, including black-box ada...
-
Offline Contextual Bandits in the Presence of New Actions
PONA integrates the LCPI estimator for new action selection with the DR estimator for existing actions to optimize policies in offline contextual bandits with evolving action spaces.
-
Logging Policy Design for Off-Policy Evaluation
Derives optimal logging policies for off-policy evaluation by balancing reward concentration against action coverage in known, unknown, and partially known regimes of target policy and rewards.
-
Off-Policy Learning with Limited Supply
OPLS is a new off-policy method for contextual bandits with limited supply that outperforms greedy approaches by prioritizing items with higher relative expected rewards for the current user.
-
Anytime-valid Optimal Policy Identification
Constructs a time-indexed set S_t retaining the true optimal policy uniformly over time with high probability, enabling early stopping with sample complexity O((log |Π| + log log(1/Δ_min))/Δ_min²) when the optimum is unique.
-
Precision Physical Activity Prescription via Reinforcement Learning for Functional Actions
Develops an offline RL algorithm for functional actions that learns personalized optimal daily step distributions from All of Us data to lower cardiometabolic risk.
-
Precision Physical Activity Prescription via Reinforcement Learning for Functional Actions
Develops a new offline RL algorithm for functional actions to learn optimal personalized daily step distributions from All of Us observational data that reduce cardiometabolic risk.
-
An adaptive variance estimator for relative sparsity
A new adaptive variance estimator for relative sparsity coefficients is introduced that fully utilizes the prior asymptotic normality theorem and incorporates variable selection effects.
-
Distributional Off-Policy Evaluation with Deep Quantile Process Regression
DQPOPE estimates the entire return distribution in off-policy evaluation via deep quantile process regression, providing statistical advantages over standard single-value methods with equivalent sample sizes.
-
Off-Policy Learning with Limited Supply
OPLS is a new off-policy learning method for contextual bandits with limited supply that outperforms conventional greedy approaches by prioritizing items with relatively higher expected rewards compared to other users.
-
Fed-CausalDiff: Decoupled Synchronization for Federated Do-Simulation and Policy Evaluation
Fed-CausalDiff proposes decoupled synchronization in a federated causal diffusion model to improve do-simulation and policy-value estimation across heterogeneous decentralized datasets.
-
Counterfactually Safe Reinforcement Learning
Formalizes counterfactual individual harm in RL and introduces a two-stage policy learning method with finite-sample guarantees on sub-optimality gap and harm rate control.
-
Logging Policy Design for Off-Policy Evaluation
Derives optimal logging policies for minimizing off-policy evaluation error under known, unknown, and partially known target policies and reward distributions.
-
A Review of Causal Decision Making
A review that organizes causal decision making into three stages and consolidates methods into an open Python collection.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.