{"work":{"id":"b28c3265-685a-4be5-b6e3-cfcd46733d99","openalex_id":null,"doi":null,"arxiv_id":"2604.09459","raw_key":null,"title":"From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models","authors":null,"authors_text":null,"year":2026,"venue":"cs.CL","abstract":"Reinforcement learning (RL) for large language models (LLMs) increasingly relies on sparse, outcome-level rewards -- yet determining which actions within a long trajectory caused the outcome remains difficult. This credit assignment (CA) problem manifests in two regimes: reasoning RL, where credit must be distributed across tokens and steps within a single chain-of-thought generation (500--30K+ tokens); and agentic RL, where multi-turn environment interaction introduces stochastic transitions, partial observability, and horizons of 100+ turns (100K--1M tokens), making episode-level credit increasingly uninformative.\n  We survey 47 CA methods (41 core, 6 adjacent enablers) published between 2024 and early 2026, organizing them in a two-dimensional taxonomy by assignment granularity (token, segment, step, turn, multi-agent) and methodology (Monte Carlo, temporal difference, model-based, game-theoretic, information-theoretic). Beyond the survey itself, we contribute three reusable resources: (1) a structured, machine-readable paper inventory with taxonomy labels, baseline families, and evidence levels; (2) a reporting checklist for future CA papers, validated against the reviewed literature to identify systematic methodological gaps; and (3) a benchmark protocol specification with task families, metadata requirements, and controlled bifurcation tasks, accompanied by a method selection decision tree.\n  Our synthesis suggests that the shift from reasoning to agentic RL complicates and reshapes the credit assignment landscape: reasoning CA is maturing around process reward models and critic-free group comparison, while agentic CA is driving genuinely new approaches -- hindsight counterfactual analysis, privileged asymmetric critics, and turn-level MDP reformulations -- that have no direct precedent in reasoning RL.","external_url":"https://arxiv.org/abs/2604.09459","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-07-09T11:16:11.379473+00:00","pith_arxiv_id":"2604.09459","created_at":"2026-05-11T03:00:56.255344+00:00","updated_at":"2026-07-09T11:16:11.379473+00:00","title_quality_ok":true,"display_title":"From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models","render_title":"From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models"},"hub":{"state":{"tier_text":"hub","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":10,"external_cited_by_count":null},"tier":"hub","role_counts":[{"context_role":"background","n":1}],"polarity_counts":[{"context_polarity":"background","n":1}],"runs":{},"summary":{},"graph":{},"authors":[]}}