PWO is a trust-region optimizer for autoregressive NQS that improves stability over Adam and stochastic reconfiguration methods while scaling to 1.5B-parameter models on spin systems.
Deep reinforcement learning at the edge of the statistical precipice
8 Pith papers cite this work, alongside 41 external citations. Polarity classification is still indexing.
8
Pith papers citing it
41
external citations · Pith
abstract
Deep reinforcement learning (RL) algorithms are predominantly evaluated by comparing their relative performance on a large suite of tasks. Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks, ignoring the statistical uncertainty implied by the use of a finite number of training runs. Beginning with the Arcade Learning Environment (ALE), the shift towards computationally-demanding benchmarks has led to the practice of evaluating only a small number of runs per task, exacerbating the statistical uncertainty in point estimates. In this paper, we argue that reliable evaluation in the few run deep RL regime cannot ignore the uncertainty in results without running the risk of slowing down progress in the field. We illustrate this point using a case study on the Atari 100k benchmark, where we find substantial discrepancies between conclusions drawn from point estimates alone versus a more thorough statistical analysis. With the aim of increasing the field's confidence in reported results with a handful of runs, we advocate for reporting interval estimates of aggregate performance and propose performance profiles to account for the variability in results, as well as present more robust and efficient aggregate metrics, such as interquartile mean scores, to achieve small uncertainty in results. Using such statistical tools, we scrutinize performance evaluations of existing algorithms on other widely used RL benchmarks including the ALE, Procgen, and the DeepMind Control Suite, again revealing discrepancies in prior comparisons. Our findings call for a change in how we evaluate performance in deep RL, for which we present a more rigorous evaluation methodology, accompanied with an open-source library rliable, to prevent unreliable results from stagnating the field.
citation-role summary
background 1
baseline 1
citation-polarity summary
years
2026 8representative citing papers
An LLM-driven agent with built-in seed-noise audits develops control policies for two aerospace problems that outperform undirected search and pass verification checks.
PE-MAMoE combines sparsely gated mixture-of-experts actors with a non-parametric phase controller in MAPPO to maintain plasticity under dynamic user mobility and traffic, yielding 26.3% higher normalized IQM return in simulations.
Neural-network agents trained in social environments learn hybrid navigation strategies that combine individual landmark use with social following, with strategy shifts driven by the ratio of skilled to unskilled social agents.
UBP2 uses ensembles of reward, dynamics, and value models to score trajectories on a unified objective of reward plus uncertainty, yielding sublinear regret bounds and higher sample efficiency on Meta-World than prior preference-based methods.
LQL turns n-step action-sequence lower bounds into a practical hinge-loss stabilizer for off-policy Q-learning without extra networks or forward passes.
Resampling clinical time series into uniform bins for offline RL reduces performance by up to 60% and causes retrospective evaluations to overestimate returns by 1.5-3x versus unprocessed data.
A tutorial playbook that organizes statistical evaluation into a workflow of claim, hypothesis, unit of analysis, baselines, sweeps, uncertainty, validation, and reporting, illustrated with Python code and a job-scheduling example.
citing papers explorer
-
One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspective
PWO is a trust-region optimizer for autoregressive NQS that improves stability over Adam and stochastic reconfiguration methods while scaling to 1.5B-parameter models on spin systems.
-
Agentic AutoResearch forSpace Autonomy: An Auditable, LLM-Driven Research Agent for Aerospace Control Problems
An LLM-driven agent with built-in seed-noise audits develops control policies for two aerospace problems that outperform undirected search and pass verification checks.
-
Plasticity-Enhanced Multi-Agent Mixture of Experts for Dynamic Objective Adaptation in UAVs-Assisted Emergency Communication Networks
PE-MAMoE combines sparsely gated mixture-of-experts actors with a non-parametric phase controller in MAPPO to maintain plasticity under dynamic user mobility and traffic, yielding 26.3% higher normalized IQM return in simulations.
-
Social-spatial dependencies for learning visual navigation
Neural-network agents trained in social environments learn hybrid navigation strategies that combine individual landmark use with social following, with strategy shifts driven by the ratio of skilled to unskilled social agents.
-
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
UBP2 uses ensembles of reward, dynamics, and value models to score trajectories on a unified objective of reward plus uncertainty, yielding sublinear regret bounds and higher sample efficiency on Meta-World than prior preference-based methods.
-
Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities
LQL turns n-step action-sequence lower bounds into a practical hinge-loss stabilizer for off-policy Q-learning without extra networks or forward passes.
-
The hidden risks of temporal resampling in clinical reinforcement learning
Resampling clinical time series into uniform bins for offline RL reduces performance by up to 60% and causes retrospective evaluations to overestimate returns by 1.5-3x versus unprocessed data.
-
How to Do Statistical Evaluations in ECE/CS Papers: A Practical Playbook for Defensible Results
A tutorial playbook that organizes statistical evaluation into a workflow of claim, hypothesis, unit of analysis, baselines, sweeps, uncertainty, validation, and reporting, illustrated with Python code and a job-scheduling example.