Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning

Weichao Zhou; Wenchao Li; Zijian Guo

RECORD

Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning

Zijian Guo, Weichao Zhou, Wenchao Li

Machine Learning · Submitted 27 Feb 2024

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim Empirical evaluations on the DSRL benchmarks demonstrate the better capacity of SDT in learning safe and high-reward policies compared with existing approaches.

how it is stated · asserted 1.00 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID2402.17217 (version 2)

Pith IDHYB374Y5, Pith's permanent record of this paper and its review

CitationWeichao Zhou, Wenchao Li, Zijian Guo. "Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning." arXiv:2402.17217 (2024). https://pith.science/paper/HYB374Y5

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 4,884 input tokens.

What the abstract itself states, as probabilities: claims first 0.10 · states limitation 0.08 · human participants 0.03 · controlled comparison 0.77 · randomized assignment 0.01 · states quantitative result 0.04 · compares with prior results 0.96 · mentions code or data availability 0.02.

The abstract, sentence by sentence

arXiv listing abstract v2. Results and the main claim are underlined; show every role.

Offline safe reinforcement learning (RL) aims to train a constraint satisfaction policy from a fixed dataset. Current state-of-the-art approaches are based on supervised learning with a conditioned policy. However, these approaches fall short in real-world applications that involve complex tasks with rich temporal and logical structures. In this paper, we propose temporal logic Specification-conditioned Decision Transformer (SDT), a novel framework that harnesses the expressive power of signal temporal logic (STL) to specify complex temporal rules that an agent should follow and the sequential modeling capability of Decision Transformer (DT). Empirical evaluations on the DSRL benchmarks demonstrate the better capacity of SDT in learning safe and high-reward policies compared with existing approaches. In addition, SDT shows good alignment with respect to different desired degrees of satisfaction of the STL specification that it is conditioned on.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	Offline safe reinforcement learning (RL) aims to train a constraint satisfaction policy from a fixed dataset.	background 1.00	0.05	0.03	asserted 1.00
s1	Current state-of-the-art approaches are based on supervised learning with a conditioned policy.	background 1.00	0.04	0.02	asserted 1.00
s2	However, these approaches fall short in real-world applications that involve complex tasks with rich temporal and logical structures.	gap 0.95	0.07	0.02	asserted 1.00
s3	In this paper, we propose temporal logic Specification-conditioned Decision Transformer (SDT), a novel framework that harnesses the expressive power of signal temporal logic (STL) to specify complex temporal rules that an agent should follow and the sequential modeling capability of Decision Transformer (DT).	contribution 0.79	0.80	0.10	asserted 0.99
s4	Empirical evaluations on the DSRL benchmarks demonstrate the better capacity of SDT in learning safe and high-reward policies compared with existing approaches.	result 1.00	0.94	0.63	asserted 1.00
s5	In addition, SDT shows good alignment with respect to different desired degrees of satisfaction of the STL specification that it is conditioned on.	result 0.99	0.93	0.13	asserted 0.99

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.64.

What was examined, and what was not

sources read

arXiv listing abstract v2

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

Offline safe reinforcement learning (RL) aims to train a constraint satisfaction policy from a fixed dataset. Current state-of-the-art approaches are based on supervised learning with a conditioned policy. However, these approaches fall short in real-world applications that involve complex tasks with rich temporal and logical structures. In this paper, we propose temporal logic Specification-conditioned Decision Transformer (SDT), a novel framework that harnesses the expressive power of signal temporal logic (STL) to specify complex temporal rules that an agent should follow and the sequential modeling capability of Decision Transformer (DT). Empirical evaluations on the DSRL benchmarks demonstrate the better capacity of SDT in learning safe and high-reward policies compared with existing approaches. In addition, SDT shows good alignment with respect to different desired degrees of satisfaction of the STL specification that it is conditioned on.

Human review

How Pith checks papers → · Full audit of this paper →

References

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

OrbitTAMP: Grounding Language Models for Task and Motion Planning in Spacecraft Rendezvous
cs.RO 2026-10 conditional novelty 6.0 of 10

A hierarchical spacecraft planning framework grounds language-model intent parsing in a spacecraft-specific behavior graph and trajectory optimization, recovering 98% of partial mission specifications with a frontier LLM.
Logic-VLA: A Temporal Logic Conditioned Vision-Language-Action Model
cs.RO 2026-08 conditional novelty 5.0 of 10

A single vision-language-action policy can condition on Signal Temporal Logic formulas at inference time and improve formal requirement satisfaction without training a separate policy for each specification.

How the literature uses it

2 arXiv papers cite this in 3 sentences

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

Papers citing it · 2 sentences without a confident label

Cites it as background 1 paper

Specification-conditioned reinforcement-learning methods [ref] learn policies for temporal logic tasks.

Logic-VLA: A Temporal Logic Conditioned Vision-Language-Action Model · Planning and Learning with Signal Temporal Logic · 0.90

What it cites · 25 sentences without a confident label

Builds on 1 paper

Motivated by RCSL [ref] , we extend the supervised learning formulation to the offline safe RL setting.

RvS: What is Essential for Offline RL via Supervised Learning? · Offline RL via Supervised Learning · 0.99

Uses the method or tool of 2 papers

For the constrained optimization RL baselines, we consider two Lagrangian-based methods that use adaptive PID-based Lagrangian multipliers [ref] to penalize constraint violations: BCQ-Lagrangian (BCQ-Lag) and BEAR-Lagrangian (BEAR-Lag), which is built upon BCQ [ref] and BEAR [ref] , respectively.

Off-Policy Deep Reinforcement Learning without Exploration · Experiment · 0.90
SDT generally follows the training and evaluation schemes of RCSL [ref] .

RvS: What is Essential for Offline RL via Supervised Learning? · Specification-Conditioned Decision Transformers · 0.95

Uses data or values from 1 paper

We use the dataset from [ref] , a comprehensive benchmark specialized for offline safe RL.

Datasets and Benchmarks for Offline Safe Reinforcement Learning · Experiment · 1.00

Cites as background 10 papers

Offline safe RL is the intersection of safe RL [ref] and offline RL [ref] , where agents aim to balance safety and efficiency with reward, cost, and cost threshold information by learning from fixed trajectories.

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems · Related Work · 0.99
The primal-dual method with function approximation optimizes iteratively between the policy and the lagrangian multiplier used to penalize constraint violations [ref] .

A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement Learning · Related Work · 0.91
The Distribution Correction Estimation (DICE) family of RL algorithms [ref] optimizes the policy based on explicitly estimating the distributional shift between the target policy and the offline data distribution.

COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation · Related Work · 0.88

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.