REVIEW 18 cited by
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization
read the original abstract
Highly effective, task-specific prompts are often heavily engineered by experts to integrate detailed instructions and domain insights based on a deep understanding of both instincts of large language models (LLMs) and the intricacies of the target task. However, automating the generation of such expert-level prompts remains elusive. Existing prompt optimization methods tend to overlook the depth of domain knowledge and struggle to efficiently explore the vast space of expert-level prompts. Addressing this, we present PromptAgent, an optimization method that autonomously crafts prompts equivalent in quality to those handcrafted by experts. At its core, PromptAgent views prompt optimization as a strategic planning problem and employs a principled planning algorithm, rooted in Monte Carlo tree search, to strategically navigate the expert-level prompt space. Inspired by human-like trial-and-error exploration, PromptAgent induces precise expert-level insights and in-depth instructions by reflecting on model errors and generating constructive error feedback. Such a novel framework allows the agent to iteratively examine intermediate prompts (states), refine them based on error feedbacks (actions), simulate future rewards, and search for high-reward paths leading to expert prompts. We apply PromptAgent to 12 tasks spanning three practical domains: BIG-Bench Hard (BBH), as well as domain-specific and general NLP tasks, showing it significantly outperforms strong Chain-of-Thought and recent prompt optimization baselines. Extensive analyses emphasize its capability to craft expert-level, detailed, and domain-insightful prompts with great efficiency and generalizability.
Forward citations
Cited by 18 Pith papers
-
TSCG: Deterministic Tool-Schema Compilation for Agentic LLM Deployments
TSCG compiles JSON tool schemas into token-efficient structured text, raising tool-use accuracy for small LLMs from 0% to 84.4% on benchmarks while cutting tokens by 52-57%.
-
RosettaSearch: Multi-Objective Inference-Time Search for Protein Sequence Design
RosettaSearch applies LLM-driven multi-objective search at inference time to improve backbone-conditioned protein sequences, recovering designs with 18-68% better structural fidelity and 2.5x higher success rates than...
-
Universal Guideline-Driven Image Clustering via a Hybrid LLM Agent
A hybrid LLM agent framework performs universal image clustering by generating guideline-aware embeddings via concept proxies and using MST-based LLM traversal for automatic discovery.
-
Towards Spec Learning: Inference-Time Alignment from Preference Pairs
Proposes compiling preference pairs into readable natural-language specifications for inference-time LLM alignment, claiming outperformance over DPO on dense-preference domains.
-
PACE: Two-Timescale Self-Evolution for Small Language Model Agents
PACE coordinates low-risk prompt evolution with validated higher-risk control-logic updates to improve frozen SLM agents on benchmarks without model retraining.
-
PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement
PIVOT refines LLM agent trajectories through plan-inspect-evolve-verify stages using environment feedback, yielding up to 94% relative gains in constraint satisfaction and 3-5x token efficiency over prior refinement methods.
-
FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration
FlashEvolve accelerates LLM agent self-evolution via asynchronous stage orchestration and inspectable language-space staleness handling, reporting 3.5-4.9x proposal throughput gains over synchronous baselines on GEPA ...
-
Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization
TPGO represents multi-agent systems as graphs of textual parameters and applies group relative optimization to enable self-improvement from execution history.
-
Select Smarter, Not More: Prompt-Aware Evaluation Scheduling with Submodular Guarantees
POES frames prompt evaluation as online adaptive testing and uses a provably submodular objective to pick informative examples, delivering 6.2% higher average accuracy and 35-60% token savings versus naive full-set scoring.
-
Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning
RRPO formulates document reranking as a sequential MDP and optimizes a pointwise reranker with PPO using LLM generation rewards and a reference-anchored deterministic baseline.
-
Bridging Individual and Collective Realism in LLM-Based Human Mobility Simulation via Mobility Scaling-Law Guidance
M2LSimu uses population-level mobility statistics as a reward signal to iteratively adjust LLM prompts, improving simulated trajectories' match to real mobility patterns.
-
One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents
Repository-level issue localization can be done by a single jump-to-definition tool trained with reinforcement learning, achieving strong results on SWE-bench despite using only open-weights models.
-
Knowing How to Edit: Reliable Evaluation Signals for Diagnosing and Optimizing Prompts at Query Level
An execution-free evaluator that predicts prompt-quality metrics guides per-query prompt rewriting, but the reported consistent gains are not supported by the paper's own tables.
-
Towards Spec Learning: Inference-Time Alignment from Preference Pairs
Spec learning compiles brief instructions and preference pairs into readable natural-language specifications that condition LLMs at inference time and can outperform DPO on domains with dense preference signals.
-
EvolveNav: Proactive Preflection and Self-Evolving Memory for Zero-Shot Object Goal Navigation
EvolveNav adds an agentic rule memory with UCB retrieval and a memory-guided preflection module to enable continuous improvement in zero-shot object goal navigation, reporting a 10.1% success rate gain over baselines.
-
Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
An LLM agent can improve by storing and Monte-Carlo-refining concept-level natural-language recipes mined from its own past trajectories.
-
Towards Efficient and Robust Linguistic Emotion Diagnosis for Mental Health via Multi-Agent Instruction Refinement
A multi-agent prompt-rewriting loop is claimed to improve LLM emotion diagnosis accuracy, but its evaluation appears to optimize on the test set and lacks replication details.
-
A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
The paper delivers the first systematic review of self-evolving agents, structured around what components evolve, when adaptation occurs, and how it is implemented.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.