REVIEW 8 cited by
Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Score-based generative models like the diffusion model have been testified to be effective in modeling multi-modal data from image generation to reinforcement learning (RL). However, the inference process of diffusion model can be slow, which hinders its usage in RL with iterative sampling. We propose to apply the consistency model as an efficient yet expressive policy representation, namely consistency policy, with an actor-critic style algorithm for three typical RL settings: offline, offline-to-online and online. For offline RL, we demonstrate the expressiveness of generative models as policies from multi-modal data. For offline-to-online RL, the consistency policy is shown to be more computational efficient than diffusion policy, with a comparable performance. For online RL, the consistency policy demonstrates significant speedup and even higher average performances than the diffusion policy.
Forward citations
Cited by 8 Pith papers
-
ClutterDexGrasp: A Sim-to-Real System for General Dexterous Grasping in Cluttered Scenes
A simulation-trained teacher-student policy achieves zero-shot sim-to-real closed-loop target-oriented dexterous grasping in cluttered scenes, with 83.9 percent real-world success.
-
Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
OMSD uses diffusion models to estimate per-agent conditional score functions of the joint behavior policy, replacing the standard product-factorization assumption and improving offline cooperative MARL performance on ...
-
Analytic Energy-Guided Policy Optimization for Offline Reinforcement Learning
AEPO approximates the log-expectation in energy-guided diffusion policy sampling using Taylor expansion and the Gaussian moment-generating function, and reports state-of-the-art average scores on D4RL offline RL benchmarks.
-
Flow-Based Policy for Online Reinforcement Learning
FlowRL learns online RL policies as flow-matching models regularized by a Wasserstein-2 constraint toward behavior-optimal replay-buffer actions.
-
Predictive Planner for Autonomous Driving with Consistency Models
A consistency-model-based predictive planner generates joint ego and agent trajectories in four sampling steps, with an alternating guided-sampling scheme to satisfy planning constraints.
-
FlowPolicy: Enabling Fast and Robust 3D Flow-based Policy via Consistency Flow Matching for Robot Manipulation
A consistency flow matching policy conditioned on 3D point clouds generates robot actions in a single inference step, running 7x faster than DP3 with comparable success rates.
-
Unpacking the Individual Components of Diffusion Policy
An ablation study shows that observation sequences, action sequences, receding horizon control, U-Net backbones, and FiLM conditioning each help Diffusion Policy in task-dependent ways, with absolute-control and hard ...
-
Reinforcement Learning: From Algorithms To Foundation Models
A dissertation uniting the author's published results: non-exploitable Nash-DQN policies and the FightLadder benchmark for games, plus diffusion/consistency-model world models for RL — a compilation rather than new results.
Discussion (0). Continue with ORCID to comment.