Pith. sign in

REVIEW 5 cited by

PromptWizard: Task-Aware Prompt Optimization Framework

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.18369 v2 pith:TT4XLHMI submitted 2024-05-28 cs.CL cs.AIcs.LG

Eshaan Agarwal , Joykirat Singh , Vivek Dani , Raghav Magazine , Tanuja Ganu , Akshay Nambi This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords promptpromptwizardoptimizationacrossautomatedcostframeworkllms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs. However, manual prompt engineering is both labor-intensive and domain-specific, necessitating the need for automated solutions. We introduce PromptWizard, a novel, fully automated framework for discrete prompt optimization, utilizing a self-evolving, self-adapting mechanism. Through a feedback-driven critique and synthesis process, PromptWizard achieves an effective balance between exploration and exploitation, iteratively refining both prompt instructions and in-context examples to generate human-readable, task-specific prompts. This guided approach systematically improves prompt quality, resulting in superior performance across 45 tasks. PromptWizard excels even with limited training data, smaller LLMs, and various LLM architectures. Additionally, our cost analysis reveals a substantial reduction in API calls, token usage, and overall cost, demonstrating PromptWizard's efficiency, scalability, and advantages over existing prompt optimization strategies.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A grammar-guided evolutionary search that composes prompt edits outperformed PromptWizard, OPRO, and RL-Prompt on small LLMs across four domain-specific tasks.

  2. An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Prompts

    cs.SE 2025-01 conditional novelty 6.0 of 10

    An automated linting and repair tool finds 3.46% of developer prompts biased, 10.75% injection-vulnerable, and improves a fraction of suboptimal prompts.

  3. Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search

    cs.LG 2025-06 conditional novelty 5.0 of 10

    Small language models achieve higher accuracy on math, coding, and logic benchmarks when their prompts contain LLM-generated reasoning blueprints and a per-model, per-task searched template.

  4. ACING: Actor-Critic for Instruction Learning in Black-Box LLMs

    cs.CL 2024-11 conditional novelty 5.0 of 10

    ACING uses off-policy actor-critic RL over continuous latent vectors, decoded by a frozen white-box model, to optimize discrete instructions for black-box LLMs from reward feedback alone.

  5. Chain of Grounded Objectives: Bridging Process and Goal-oriented Prompting for Code Generation

    cs.CL 2025-01 conditional novelty 4.0 of 10

    CGO, a concise goal-oriented prompting method, achieves accuracy comparable to or better than existing prompting baselines on code generation benchmarks while using fewer intermediate tokens.

Pith tools