Pith. sign in

REVIEW 10 cited by

InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.03082 v2 pith:UYHYEA6Z submitted 2023-06-05 cs.AI

Lichang Chen , Jiuhai Chen , Tom Goldstein , Heng Huang , Tianyi Zhou This is my paper · ORCID
classification cs.AI
keywords instructioninstructzeroblack-boxllmsopen-sourcesoftdifferentlanguage
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large language models~(LLMs) are instruction followers, but it can be challenging to find the best instruction for different situations, especially for black-box LLMs on which backpropagation is forbidden. Instead of directly optimizing the discrete instruction, we optimize a low-dimensional soft prompt applied to an open-source LLM to generate the instruction for the black-box LLM. On each iteration of the proposed method, which we call InstructZero, a soft prompt is converted into an instruction using the open-source LLM, which is then submitted to the black-box LLM for zero-shot evaluation, and the performance is sent to Bayesian optimization to produce new soft prompts improving the zero-shot performance. We evaluate InstructZero on different combinations of open-source LLMs and APIs including Vicuna and ChatGPT. Our results show that InstructZero outperforms SOTA auto-instruction methods across a variety of downstream tasks. Our code and data are publicly available at https://github.com/Lichang-Chen/InstructZero.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MASPOB: Bandit-Based Prompt Optimization for Multi-Agent Systems with Graph Neural Networks

    cs.LG 2026-03 conditional novelty 6.0 of 10

    MASPOB combines a GNN surrogate, LinUCB-style uncertainty, and coordinate ascent to optimize prompts in fixed-topology multi-agent LLM systems, beating AFlow and MIPRO on average across six benchmarks.

  2. Stabilizing Black-Box Prompt Optimization with Textual Regularization and Signal Aggregation

    cs.LG 2025-07 conditional novelty 6.0 of 10

    TRAS adds success-based textual regularization and Monte Carlo signal aggregation to black-box prompt optimization, improving accuracy and reducing instruction loss when moving prompts across models.

  3. Instruction Learning Paradigms: A Dual Perspective on White-box and Black-box LLMs

    cs.CL 2025-06 reject novelty 5.0 of 10

    A hybrid black-box and white-box instruction optimizer, built on InstructZero and INSTINCT, reports the highest mean score on 30 tasks but with small margins, missing error bars, and unreleased code.

  4. Meta-Prompt Optimization for LLM-Based Sequential Decision Making

    cs.LG 2025-02 conditional novelty 5.0 of 10

    EXPO uses adversarial bandit weighting over LLM-generated prompt variations to optimize the meta-prompt of LLM-based sequential decision-making agents, improving performance on optimization and bandit tasks.

  5. ACING: Actor-Critic for Instruction Learning in Black-Box LLMs

    cs.CL 2024-11 conditional novelty 5.0 of 10

    ACING uses off-policy actor-critic RL over continuous latent vectors, decoded by a frozen white-box model, to optimize discrete instructions for black-box LLMs from reward feedback alone.

  6. Interpretable Adaptive Sampling for LLM Test-Time Scaling

    cs.AI 2026-08 reject novelty 4.0 of 10

    A fuzzy controller that allocates a per-prompt sampling budget keeps LLM accuracy near a fixed full-budget baseline while reducing the average number of candidate answers on some datasets.

  7. Deploying AI for Signal Processing education: Selected challenges and intriguing opportunities

    eess.SP 2025-09 conditional novelty 4.0 of 10

    AI can be used to generate interactive signal processing courseware, but the paper offers no evidence that students learn better from it.

  8. Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings

    cs.AI 2025-05 conditional novelty 4.0 of 10

    DEEVO evolves better LLM prompts by debating outputs and selecting survivors with Elo ratings, without requiring labeled data or a hand-written fitness function.

  9. The Evolution of Natural Language Processing: How Prompt Optimization and Language Models are Shaping the Future

    cs.CL 2025-06 reject novelty 3.0 of 10

    A review that categorizes 45 prompt optimization strategies into 11 classes and surveys their use across NLP tasks, models, and datasets, but with inconsistent counts and overlapping categories.

  10. Boosting Private Domain Understanding of Efficient MLLMs: A Tuning-free, Adaptive, Universal Prompt Optimization Framework

    cs.AI 2024-12 conditional novelty 3.0 of 10

    IDEALPrompt combines strategy search with self-reflection to craft prompts that let a 2B multimodal model match or beat fine-tuning on private e-commerce data, without changing model weights.

Pith tools