REVIEW 6 cited by
LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
The success of Large Language Models (LLMs) has sparked interest in various agentic applications. A key hypothesis is that LLMs, leveraging common sense and Chain-of-Thought (CoT) reasoning, can effectively explore and efficiently solve complex domains. However, LLM agents have been found to suffer from sub-optimal exploration and the knowing-doing gap, the inability to effectively act on knowledge present in the model. In this work, we systematically study why LLMs perform sub-optimally in decision-making scenarios. In particular, we closely examine three prevalent failure modes: greediness, frequency bias, and the knowing-doing gap. We propose mitigation of these shortcomings by fine-tuning via Reinforcement Learning (RL) on self-generated CoT rationales. Our experiments across multi-armed bandits, contextual bandits, and Tic-tac-toe, demonstrate that RL fine-tuning enhances the decision-making abilities of LLMs by increasing exploration and narrowing the knowing-doing gap. Finally, we study both classic exploration mechanisms, such as $\epsilon$-greedy, and LLM-specific approaches, such as self-correction and self-consistency, to enable more effective fine-tuning of LLMs for decision-making.
Forward citations
Cited by 6 Pith papers
-
STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle
Frontier LLM agents detect hidden supply-chain stress almost equally well (84–88% of episodes) but vary from skill 0.62 to −0.23, with two of four models acting worse than ignoring symptoms.
-
Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging
A 57-task, expert-verified benchmark shows frontier LLM coding agents systematically fail on physical conventions, inverse-solver choice, and end-to-end imaging pipelines.
-
Formalizing Learning from Language Feedback with Provable Guarantees
Introduces a formal framework for learning from language feedback, a transfer eluder dimension complexity measure, and HELiX, a no-regret algorithm whose regret scales with this dimension.
-
BioBlue: Systematic runaway-optimiser-like LLM failure modes on biologically and economically aligned AI safety benchmarks for LLMs with simplified observation format
LLMs in long-horizon multi-objective simulations show a recurrent drift from balanced, target-following behavior to single-objective, unbounded maximization, despite initial competence.
-
AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization
AIGB-R1 couples an LLM strategy planner with a prompt-conditioned Decision Transformer executor and trains both end-to-end with a decoupled-GRPO self-evolving loop, reporting the highest Score on the AuctionNet benchmark.
-
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.
Discussion (0). Continue with ORCID to comment.