REVIEW 4 cited by
Graph Decision Transformer
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Offline reinforcement learning (RL) is a challenging task, whose objective is to learn policies from static trajectory data without interacting with the environment. Recently, offline RL has been viewed as a sequence modeling problem, where an agent generates a sequence of subsequent actions based on a set of static transition experiences. However, existing approaches that use transformers to attend to all tokens naively can overlook the dependencies between different tokens and limit long-term dependency learning. In this paper, we propose the Graph Decision Transformer (GDT), a novel offline RL approach that models the input sequence into a causal graph to capture potential dependencies between fundamentally different concepts and facilitate temporal and causal relationship learning. GDT uses a graph transformer to process the graph inputs with relation-enhanced mechanisms, and an optional sequence transformer to handle fine-grained spatial information in visual tasks. Our experiments show that GDT matches or surpasses the performance of state-of-the-art offline RL methods on image-based Atari and OpenAI Gym.
Forward citations
Cited by 4 Pith papers
-
Decision Flow Policy Optimization
Decision Flow frames the gradual action generation of flow-based policies as a flow MDP and updates the flow policy with flow-level value functions, reporting state-of-the-art results on several D4RL tasks.
-
Are Expressive Models Truly Necessary for Offline RL?
A two-layer MLP with recursive skip-step sub-goal planning can reach state-of-the-art offline RL scores on long-horizon D4RL tasks, challenging the need for large expressive models.
-
TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models
TrojanTO implants action-level backdoors into Decision Transformer style offline RL models using 10 trajectories, alternating trigger optimization and model fine-tuning, reaching average attack success 0.719 while pre...
-
Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
Filtered behavior cloning, an MLP baseline trained only on high-return trajectories, matches or outperforms Decision Transformer on the sparse-reward Robomimic and sparsified D4RL benchmarks tested.
Discussion (0). Continue with ORCID to comment.