REVIEW 18 cited by
Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We introduce meta-prompting, an effective scaffolding technique designed to enhance the functionality of language models (LMs). This approach transforms a single LM into a multi-faceted conductor, adept at managing and integrating multiple independent LM queries. By employing high-level instructions, meta-prompting guides the LM to break down complex tasks into smaller, more manageable subtasks. These subtasks are then handled by distinct "expert" instances of the same LM, each operating under specific, tailored instructions. Central to this process is the LM itself, in its role as the conductor, which ensures seamless communication and effective integration of the outputs from these expert models. It additionally employs its inherent critical thinking and robust verification processes to refine and authenticate the end result. This collaborative prompting approach empowers a single LM to simultaneously act as a comprehensive orchestrator and a panel of diverse experts, significantly enhancing its performance across a wide array of tasks. The zero-shot, task-agnostic nature of meta-prompting greatly simplifies user interaction by obviating the need for detailed, task-specific instructions. Furthermore, our research demonstrates the seamless integration of external tools, such as a Python interpreter, into the meta-prompting framework, thereby broadening its applicability and utility. Through rigorous experimentation with GPT-4, we establish the superiority of meta-prompting over conventional scaffolding methods: When averaged across all tasks, including the Game of 24, Checkmate-in-One, and Python Programming Puzzles, meta-prompting, augmented with a Python interpreter functionality, surpasses standard prompting by 17.1%, expert (dynamic) prompting by 17.3%, and multipersona prompting by 15.2%.
Forward citations
Cited by 18 Pith papers
-
MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents
MedAgentBench is a FHIR-based virtual EHR benchmark with 300 physician-written tasks and 100 patient profiles, where the best LLM agent (Claude 3.5 Sonnet v2) succeeds on 69.67% of tasks.
-
Hierarchical Domain Generalization
Over infinite domains, hierarchy-uniform domain generalization is impossible for every nontrivial hypothesis class; a length-generalization bound is a property of the length hierarchy, not a hierarchy-free guarantee.
-
VulWeaver: Weaving Broken Semantics for Grounded Vulnerability Detection
VulWeaver improves Java vulnerability detection to 0.75 F1 by enhancing dependency graphs with LLM semantic fixes, extracting full context from slices plus implicit usage info, and applying type-specific meta-promptin...
-
Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planning
Decoupling LLM-agent planning from summarization and rewarding tool-call completeness rather than final-answer correctness improves planning by 8-12% and end-to-end answers by 5-6% over end-to-end RL baselines.
-
Controlling Context: Generative AI at Work in Integrated Circuit Design and Other High-Precision Domains
Qualitative interviews with 17 IC-design engineers show that the main difficulty with generative AI tools is not output accuracy but context control, supporting a shift toward interactive context-steering features.
-
What Makes a Good Natural Language Prompt?
A meta-analysis and experiments propose 21 prompt properties across six dimensions, finding that boosting a single property often beats combining several, and that instruction-tuning with polite prompts can help.
-
CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance
Training a small 8-billion-parameter model to guide GPT-4o between code and text raises its score on 37 math and logic tasks from 53.3 to 86.4, outpacing o1, o1-preview, and DeepSeek R1 on the aggregate.
-
MASTER: A Multi-Agent System with LLM Specialized MCTS
A multi-agent framework whose tree search is guided by LLM self-evaluation instead of simulations, reporting 76% on HotpotQA, 80% on WebShop, and 91% on MBPP.
-
PoCEvolve: Generating Proof-of-Concept Exploits from Security Patches with Vulnerability-Aware Prompt Evolution
A patch-only exploit generator using LLM-analyzed commits plus vulnerability-context-guided prompt evolution reaches 58.4-85.3% PoC success on 190 JavaScript vulnerabilities.
-
ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning
LLM reasoning benchmark scores vary substantially across repeated runs under the same model, strategy, and task, so single-run evaluation can misrank systems.
-
CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization
An open-source LLM-evolution framework matches or narrowly beats reported AlphaEvolve scores on 5 of 9 benchmarks, with an open-weight backbone at roughly an order of magnitude lower API cost, though the SOTA margins ...
-
A Hybrid Multi-Agent Prompting Approach for Simplifying Complex Sentences
A multi-agent GPT-4O pipeline with an internal semantic-lexical gate claims 70% success on simplifying 100 video game sentences, versus 48% for a single-agent version.
-
Automated Collection of Evaluation Dataset for Semantic Search in Low-Resource Domain Language
A fusion of encoder ensemble scores and LLM re-ranking improves automated relevance scoring for semantic search test collections in low-resource German, but the fusion thresholds are tuned on the test data.
-
Monadic Context Engineering
Agent workflows are re-expressed as a monad transformer stack StateT S (EitherT E IO), so state threading, error short-circuiting, and async execution become built-in composition operators.
-
Inteligencia Artificial jur\'idica y el desaf\'io de la veracidad: an\'alisis de alucinaciones, optimizaci\'on de RAG y principios para una integraci\'on responsable
Legal AI hallucination persists in commercial RAG tools (17-34%+ of queries), so the report argues the fix is consultative, source-citing system design plus mandatory human oversight, not better generative models.
-
Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Study
Testing three automatic prompt optimizers on triple extraction shows consistent gains over simple handcrafted prompts, with the largest gains at high schema complexity and long text.
-
Literature Review Of Multi-Agent Debate For Problem-Solving
A literature review concludes that multi-agent LLM debate helps up to a task-dependent point, after which extra agents and rounds add cost without reliable gains.
-
AI-Driven Scholarly Peer Review via Persistent Workflow Prompting, Meta-Prompting, and Meta-Reasoning
A persistent, structured prompt loaded into an LLM chat session can guide reasoning models through critical analysis of experimental chemistry papers, but the evidence is a single qualitative case study.
Discussion (0). Continue with ORCID to comment.