REVIEW 39 cited by
Self-Reflection in LLM Agents: Effects on Problem-Solving Performance
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Self-Reflection in LLM Agents: Effects on Problem-Solving Performance
read the original abstract
In this study, we investigated the effects of self-reflection in large language models (LLMs) on problem-solving performance. We instructed nine popular LLMs to answer a series of multiple-choice questions to provide a performance baseline. For each incorrectly answered question, we instructed eight types of self-reflecting LLM agents to reflect on their mistakes and provide themselves with guidance to improve problem-solving. Then, using this guidance, each self-reflecting agent attempted to re-answer the same questions. Our results indicate that LLM agents are able to significantly improve their problem-solving performance through self-reflection ($p < 0.001$). In addition, we compared the various types of self-reflection to determine their individual contribution to performance. All code and data are available on GitHub at https://github.com/matthewrenze/self-reflection
Forward citations
Cited by 39 Pith papers
-
SkillSmith: Learning to Compose Parametric Skills and Textual Knowledge
SkillSmith, an LLM augmented to ingest prefix-weights and text, generates target-task prefix-weights that beat text-only and weight-only baselines, especially as fine-tuning initialization.
-
Agentic generation of verifiable rules for deterministic, self-expanding reaction classification
Multi-agent LLMs classify USPTO reactions and write verified SMIRKS rules, expanding a reaction taxonomy from 68 to 14,073 classes and matching proprietary classifiers on held-out and out-of-distribution data.
-
Agentic generation of verifiable rules for deterministic, self-expanding reaction classification
Multi-agent LLMs generate and verify 14,073 deterministic reaction rules from 665,901 patents, enabling 97.7% classification of unseen reactions with finer resolution than fixed proprietary systems.
-
Agentic generation of verifiable rules for deterministic, self-expanding reaction classification
Multi-agent LLMs classify USPTO reactions and write verified SMIRKS rules, expanding a 68-class taxonomy to 14,073 and classifying 97.7% of held-out reactions with a hybrid fingerprint-plus-template system.
-
Multimodal Graph RAG for Long-range Visually Rich Document Understanding
Multimodal graph RAG with DLVQA benchmark outperforms MMRAG and KG methods on multi-hop document VQA tasks.
-
LinTree: Improving LLM Reasoning with Explicitly Structured Search Histories
Adding explicit parent pointers to represent search tree structure in LLM reasoning traces (LinTree) improves task performance and search efficiency on Blocks World, grid Navigation, and Sokoban relative to implicit t...
-
VINS-120K: Ultra High-Resolution Image Editing with A Large-Scale Dataset
VINS-120K supplies the first large-scale set of instruction-image-edited-image triplets at ultra-high resolution together with an adaptation strategy that improves detail synthesis.
-
GSAR: Typed Grounding for Hallucination Detection and Recovery in Multi-Agent LLMs
GSAR is a grounding-evaluation framework for multi-agent LLMs that uses a four-way claim typology, evidence-weighted asymmetric scoring, and tiered recovery decisions to detect and mitigate hallucinations.
-
Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
Evo-Memory is a new benchmark for self-evolving memory in LLM agents across task streams, with baseline ExpRAG and proposed ReMem method that integrates reasoning, actions, and memory updates for continual improvement.
-
Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeeds
Matched two-pass experiments show human revisers improve on objective and subjective tasks, while LLM self-revision yields near-zero information gain on objective tasks and negative information gain on subjective task...
-
It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluation
In a 98-participant fact-checking study, the rhetorical style of AI advice changed accuracy, confidence, and preference, with step-by-step explanations helping most and user preference diverging from performance.
-
Wait, am I Being Fair? Characterizing Deductive Stereotyping and Mitigating It with Fair-GCG
The paper characterizes deductive stereotyping in LLMs and introduces Fair-GCG to discover injection phrases that improve fairness across benchmarks, reasoning, and real-world tasks.
-
MoG: Mixture of Experts for Graph-based Retrieval-Augmented Generation
MoG uses hub graphs for shared context and sparsely activates expert graphs with a topology-aware router, reporting over 20% relative gains on MuSiQue.
-
Unified Data Selection for LLM Reasoning
High-Entropy Sum (HES) selects high-quality reasoning data for LLMs by summing entropy of the top highest-entropy tokens, matching full-dataset performance with top 20% in SFT and outperforming baselines in RFT and RL.
-
STAR: Failure-Aware Markovian Routing for Multi-Agent Spatiotemporal Reasoning
STAR combines expert nominal routes with trace-learned recovery transitions in a failure-typed routing matrix, improving multi-agent spatiotemporal reasoning over baselines especially on error-deviating queries.
-
STAR: Failure-Aware Markovian Routing for Multi-Agent Spatiotemporal Reasoning
STAR presents a failure-aware routing framework using a state-conditioned transition policy and an agent routing matrix combining expert routes with learned recoveries from execution traces to improve multi-agent spat...
-
HAGE: Harnessing Agentic Memory via RL-Driven Weighted Graph Evolution
HAGE proposes a trainable weighted graph memory framework with LLM intent classification, dynamic edge modulation, and RL optimization that improves long-horizon reasoning accuracy in agentic LLMs over static baselines.
-
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
Across 1,140 agent traces, LLM tool-calling agents exhibit high structural consistency (same tools, same order) but high argument variance, and only structural consistency predicts task correctness.
-
Specialty-Specific Medical Language Model for Immune-Mediated Diseases
Across 1,140 traces, multi-step tool-calling agents show high tool-sequence consistency (TSS≈0.87) but lower argument consistency (AC≈0.69), and only TSS predicts task success.
-
Visual Enhanced Depth Scaling for Multimodal Latent Reasoning
Visual replay module and adaptive depth scaling improve multimodal latent reasoning, reaching SOTA benchmarks with faster inference than explicit chain-of-thought methods.
-
VineLM: Trie-Based Fine-Grained Control for Agentic Workflows
VineLM uses an annotated execution trie plus cascade profiling and online re-rooting to select models per stage invocation in agentic workflows, improving the cost-latency-accuracy frontier by up to 18% accuracy at fi...
-
A Minimal Agent for Automated Theorem Proving
A minimal agentic system achieves competitive performance in automated theorem proving with a simpler design and lower cost than state-of-the-art methods.
-
Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
Evo-Memory is a new streaming benchmark and evaluation framework for self-evolving memory in LLM agents, unifying over ten memory modules and introducing the ReMem pipeline for continual improvement on multi-turn and ...
-
MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning
MeTHanol fine-tunes an intermediate LLM layer to generate thoughts in a first pass, then uses those thoughts for a second-pass answer, showing gains on Theory of Mind and vignette tasks plus adaptation to character prompts.
-
NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability
Combining a knowledge-graph belief state, KG-augmented reflection, and TSMC-style particle planning lifts LLM agent success rates by roughly 30-95% over ReAct/Reflexion baselines on three benchmarks.
-
A Diagnostic Framework for AI Agent Behavior
A two-layer diagnostic framework for AI agent behavior: distinguishing foundational computational substrate from behavioral modulation layer.
-
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP
In CybORG CAGE-2, programmatic state abstraction improves mean return up to 76% over raw observations while adding deliberation tools to hierarchies degrades performance up to 3.4x and increases token use.
-
STAR: Failure-Aware Markovian Routing for Multi-Agent Spatiotemporal Reasoning
STAR is a failure-aware Markovian router that learns recovery transitions from both successful and unsuccessful execution traces to improve multi-agent performance on spatiotemporal benchmarks.
-
UniMesh: Unifying 3D Mesh Understanding and Generation
UniMesh unifies 3D mesh generation and understanding in one model via a Mesh Head interface, Chain of Mesh iterative editing, and an Actor-Evaluator self-reflection loop.
-
Visual Enhanced Depth Scaling for Multimodal Latent Reasoning
A visual replay module combined with adaptive depth scaling improves multimodal latent reasoning, delivering state-of-the-art benchmark results and faster inference than explicit chain-of-thought methods.
-
Visual Enhanced Depth Scaling for Multimodal Latent Reasoning
Visual replay and depth scaling in latent reasoning produce state-of-the-art multimodal results with faster inference than explicit CoT.
-
Enhancing LLM Problem Solving via Tutor-Student Multi-Agent Interaction
A same-LLM tutor-student agent pair solves coding tasks at similar or higher accuracy than self-consistency or debate baselines while using significantly fewer tokens.
-
Failure Makes the Agent Stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions
Structured reflection makes error diagnosis and repair an explicit trainable step that improves reliability and reduces redundant calls in tool-using LLM agents.
-
A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models
The authors organize LLM-based time series reasoning into three exclusive topologies (direct, chain, branch) crossed with four objectives, and use them to label 125 papers, benchmarks, and resources.
-
Self-Aligned Reward: Towards Effective and Efficient Reasoners
Self-aligned reward uses relative perplexity differences to encourage concise, query-specific reasoning in LLMs, yielding 4% accuracy gains and 30% lower inference cost when added to PPO or GRPO.
-
Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots
PivotTrace selects unlabeled data for RLVR by quantifying uncertainty via pivot density from attention dynamics, outperforming full supervision using only 29.3% annotations and converging 2.75 times faster.
-
Deconstructing Spatial Complexity: Hierarchical Decomposition for LLM Spatial Reasoning
Proposes hierarchical task decomposition plus M-GRPO to improve LLM performance on spatial navigation, planning, and games.
-
Trustworthy Agent Network: Trust in Agent Networks Must Be Baked In, Not Bolted On
Argues that trustworthiness in Agent-to-Agent networks requires a new conceptual framework with four design pillars baked in from the beginning, as retrofitting existing single-agent methods is insufficient.
-
Large Language Model-Brained GUI Agents: A Survey
A survey consolidating frameworks, data practices, large action models, benchmarks, applications, and research gaps in LLM-brained GUI agents.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.