REVIEW 10 cited by
A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The Credit Assignment Problem (CAP) refers to the longstanding challenge of Reinforcement Learning (RL) agents to associate actions with their long-term consequences. Solving the CAP is a crucial step towards the successful deployment of RL in the real world since most decision problems provide feedback that is noisy, delayed, and with little or no information about the causes. These conditions make it hard to distinguish serendipitous outcomes from those caused by informed decision-making. However, the mathematical nature of credit and the CAP remains poorly understood and defined. In this survey, we review the state of the art of Temporal Credit Assignment (CA) in deep RL. We propose a unifying formalism for credit that enables equitable comparisons of state-of-the-art algorithms and improves our understanding of the trade-offs between the various methods. We cast the CAP as the problem of learning the influence of an action over an outcome from a finite amount of experience. We discuss the challenges posed by delayed effects, transpositions, and a lack of action influence, and analyse how existing methods aim to address them. Finally, we survey the protocols to evaluate a credit assignment method and suggest ways to diagnose the sources of struggle for different methods. Overall, this survey provides an overview of the field for new-entry practitioners and researchers, it offers a coherent perspective for scholars looking to expedite the starting stages of a new study on the CAP, and it suggests potential directions for future research.
Forward citations
Cited by 10 Pith papers
-
Learning to Prepare Molecular Ground States with Transformer Models
Transformers trained on ADAPT-VQE data generate imipramine ground-state circuits in seconds at roughly reference accuracy — and beat the training data after reinforcement learning — though real-hardware energies still...
-
Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learning
ISA uses mutual information between actions and state changes to define each agent's influence scope, then uses it for credit assignment and count-based exploration in sparse-reward MARL.
-
Learning from Environmental Feedback: Credit Assignment across Multiple Timescales for Agentic Reinforcement Learning
EFCA combines immediate environment feedback and recent state-history signals into a return reweighting that improves step-level credit assignment for LLM agents in ALFWorld and WebShop.
-
Attribution Markets: A Fisher-Market Formulation for Fractional Credit Assignment Between Planned Tasks and Performed Actions
Treating planned tasks as budget-constrained buyers and logged actions as divisible goods yields a market-based attribution rule with provable conservation and credit caps, though it underperforms entropic optimal tra...
-
Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity
A shifted-value transformation turns static CVaR MDPs into a bounded, contracting Bellman operator with dense rewards, enabling discretized value iteration and Q-learning with explicit error bounds.
-
Deep Reinforcement Learning for Dynamic Origin-Destination Matrix Estimation in Microscopic Traffic Simulations Considering Credit Assignment
A PPO-based agent with per-OD binary departure actions estimates dynamic OD matrices and cuts link-flow MSE by ~43% on the Nguyen–Dupuis toy network versus the best BO baseline.
-
TextAtari: 100K Frames Game Playing with Language Agents
TextAtari is a text-based Atari benchmark for language agents; 7-8B LLMs stay below 10% of human scores in over 90% of tested conditions, and knowledge injection helps more than chain-of-thought.
-
Humans Coexist, So Must Embodied Artificial Agents
Coexistence, defined as sustained meaningful and reciprocal interaction among an agent, humans, and environment, is presented as a necessary design goal for embodied AI.
-
First Return, Entropy-Eliciting Explore
FR3E improves LLM math reasoning by finding high-entropy tokens in a trajectory and doing extra rollouts from those points to shape rewards.
-
Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities
A tutorial reviewing LLM alignment through the lens of inverse reinforcement learning, arguing that neural reward models learned from human data are central to post-training.
Discussion (0). Continue with ORCID to comment.