A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models

Diego Klabjan; Shuyang Wang; Somayeh Moazeni

REVIEW 3 major objections 6 minor 1 cited by

A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A sequential Bayesian policy, the Knowledge-Gradient, finds strong LLM prompts within 30 evaluations and beats evolutionary and bandit baselines on instruction induction tasks.

desk verdict A sensible transfer of Knowledge-Gradient optimal learning to budget-limited prompt search, but the headline claim of 'significant' gains lacks statistical support. read the letter →

arxiv 2501.03508 v1 pith:X3GFW22K submitted 2025-01-07 cs.CL

Shuyang Wang , Somayeh Moazeni , Diego Klabjan This is my paper · ORCID

classification cs.CL

keywords automatedpromptengineeringKnowledge-GradientpolicyBayesianregressionfeature-basedrepresentationinstructioninductionlargelanguagemodelssequentialoptimallearninglimitedevaluationbudget

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that a forward-looking Bayesian decision policy, the Knowledge-Gradient, can find strong LLM prompts with very few evaluations by treating prompts as combinations of interpretable features and learning which combinations matter. On 13 instruction-induction tasks, with only 30 LLM evaluations, the KG-based workflow reports the highest average held-out score (0.6281) and the best average ranking among EvoPrompt, TRIPLE, greedy, and Thompson-sampling baselines. The authors argue this matters because prompt evaluation is often costly, and existing automated prompt engineering methods typically assume many iterations or require white-box access to the model. The feature-based representation broadens the search space and lets the policy exploit correlations among similar prompts instead of treating candidates independently.

What carries the argument

The central object is the Knowledge-Gradient quantity, $\nu_n^x = \mathbb{E}[\max_{y\in\mathcal{X}}(p_y^n + q_y^n(x)T_{2a_n}) \mid S_n] - \max_{y\in\mathcal{X}} p_y^n$, which measures how much one more evaluation of prompt $x$ is expected to raise the best predicted prompt score. The inner expectations are approximated by quantizing the Student-$t$ distribution $T_{2a_n}$, and the resulting optimization is solved as a mixed-integer second-order cone program. A normal-gamma Bayesian linear model with update equations (4)-(7) carries the correlations between similar prompts, so learning about one feature combination sharpens beliefs about others.

What would settle it

On the same 13 tasks and 30-evaluation budget, run SOPL-KG with pairwise feature interactions added to the linear model; if the KG advantage over Thompson sampling and greedy shrinks or reverses, the assumed feature-separability in equation (1) was load-bearing. A simpler companion check is to plot residuals of logit validation score against fitted values from equation (1) after the 30 evaluations: curvature or mean-dependent variance would show the linear-Gaussian model is misspecified.

Watch

Extended reading notes

Core claim

The paper's central claim is that automated prompt engineering can be cast as a finite-horizon sequential optimal learning problem, and that the Knowledge-Gradient policy solves it well under a tight evaluation budget. Given a feature-based description of prompts, the paper models the logit of validation accuracy as a Bayesian linear regression, $\eta_x = \Theta^\top x + \varepsilon$, and at each step chooses the prompt that maximizes the expected improvement in the best predicted score after one more evaluation. On 13 instruction-induction tasks with 30 LLM evaluations, this policy attains the highest average test score (0.6281) and best average ranking (1.85) among EvoPrompt, TRIPLE, Thompson sampling, and greedy, with the largest margins on tasks where prompt performance is most variable. The authors take this as evidence that forward-looking, correlation-aware search can find strong prompts where evaluation is costly and exhaustive search is infeasible.

Load-bearing premise

The load-bearing premise is that a prompt's value to the LLM, after a logit transform, is a straight-line sum of its feature indicators plus random noise with constant spread; if real prompt performance has strong interactions or uneven noise, the Bayesian beliefs and the Knowledge-Gradient selections built on them rest on a misspecified model.

Editorial extensions

If this is right

With a budget of 30 evaluations, the Knowledge-Gradient policy achieves the highest average test score (0.6281) and best average ranking (1.85) across the 13 instruction-induction tasks, compared with EvoPrompt, TRIPLE, Thompson sampling, and greedy.
The KG advantage holds when the budget is cut to 20 or 10 evaluations, and early stopping with a patience of 10 realizes about 17 evaluations with only a modest drop in test score.
On tasks where LLM responses are highly variable across prompt features, KG's relative improvement over Thompson sampling and greedy exceeds 10%, while on flatter, easier tasks a greedy policy is nearly as good.
Searching over multiple prompt features simultaneously—template, demonstration examples, roles, paraphrasing, and description—substantially outperforms searching over demonstration examples alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same constrained-feature formulation could carry over to other black-box optimization problems with expensive evaluations, such as choosing demonstration sets for retrieval-augmented generation or wording for safety guardrails; the paper does not test these settings.
Editorial inference: the reported correlation between score variability and KG's advantage suggests a practical adaptive policy—estimate the score landscape's variance in a short warm-up and then choose between KG and a cheaper exploitation policy—though the paper stops short of proposing such a rule.
Editorial inference: because the KG decision is computed via mixed-integer conic optimization, it can naturally enforce logical constraints on prompts (for example, mutually exclusive features or content-policy restrictions), which text-editing and embedding-space methods cannot easily express; a direct test would be to impose such constraints in the instruction-induction setup.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper wraps a known optimal-learning method around prompt engineering and gets consistent, modest gains on a public benchmark. The framework (SOPL) represents prompts as one-hot feature vectors, models the logit of validation accuracy as a linear Gaussian process, and uses the Knowledge-Gradient policy to decide which prompt to evaluate next under a budget of 30 LLM calls. The genuinely new part is the packaging: a feature-based, constraint-defined prompt space that goes beyond enumerating candidates, plus the demonstration that KG beats heuristics like Greedy and Thompson sampling in this setting. The mathematical machinery, including the mixed-integer SOCP reformulation, is taken directly from Moazeni et al. (2020) and the authors cite it honestly. That is reuse, not circularity.

The experiments are decent for a first pass: 20 replications per task, comparison against EvoPrompt and TRIPLE, sensitivity checks for N=30/20/10, early stopping, and a feature ablation. The observation that KG's advantage grows with response variability across tasks (correlation with coefficient of variation around 0.89) is genuinely interesting and provides some internal validity.

The soft spots are statistical and reproducibility-related. The abstract and intro repeatedly say "significantly outperforms," but there are no significance tests, no confidence intervals, and no per-task differences reported. The average gaps over EvoPrompt and TRIPLE are 0.033–0.067, while the average within-task standard deviation is 0.067–0.097, so seed noise could plausibly explain part or all of the headline difference. The post-hoc selection of 13 "challenging" tasks from the 24 available further weakens the claim, since the comparison is not over a pre-specified task population. Prior hyperparameters (θ0, Σ0, a, b, ΣΘ, quantization size) are not reported, which makes replication harder. The linear-Gaussian model is assumed without diagnostics, and the intro's claim about "capturing various interactions among prompt attributes" is overstated—the model is additive in main effects only.

These are fixable issues, not fatal flaws. The framework is transparent, the direction of the effect is consistent across tasks, and the paper gives a useful baseline for anyone working on black-box prompt optimization with expensive evaluations. I would send it to peer review and ask for paired significance tests or bootstrap intervals, a table of priors, and a modest rewrite of the "interactions" and "significant" claims. With those changes, this becomes a solid reference.

Referee Report

3 major / 6 minor

Summary. The paper proposes SOPL, a sequential optimal learning framework for automated prompt engineering. Prompts are represented by interpretable categorical features, and a Bayesian linear model links the logit of validation accuracy to one-hot-encoded features. A Knowledge-Gradient policy, computed through mixed-integer conic optimization, selects prompt representations to evaluate under a limited budget. The method is tested on instruction induction tasks using GPT-3.5, with comparisons against EvoPrompt, TRIPLE, Thompson sampling, and a greedy policy. The central claim is that SOPL-KG significantly outperforms these benchmarks within 30 or fewer evaluations, particularly on a set of 13 'challenging' tasks.

Significance. If substantiated, the contribution is useful for automated prompt engineering in cost-sensitive applications: it moves beyond static candidate sets, uses an interpretable feature representation, and applies a forward-looking Bayesian learning policy that exploits correlations among prompts. The paper supplies explicit Bayesian update equations, a benchmark comparison on a public dataset, and several sensitivity analyses. The main value, however, rests on the empirical claim of significant improvement, and the current evidence for that claim is incomplete in ways that a revision can address.

major comments (3)

[§6, Table 2] The abstract and Section 6 claim that SOPL-KG 'significantly outperforms' benchmark strategies, but no statistical significance test is reported anywhere in the paper. The absolute gaps between methods (0.033 to 0.067) are of the same order as the average within-task standard deviations (0.067 to 0.097), so the observed differences could plausibly arise from seed noise. In addition, the 13 tasks are selected post hoc as 'challenging' based on validation scores below 80% and large variance, so the comparison is not over a pre-specified task population and may be affected by selection bias. Please provide paired significance tests (ideally paired by seed across replications) or confidence intervals, per-task results with standard errors, and a clear pre-registered or prespecified definition of the task-selection rule.
[§3.3 and §5.2] The paper never reports the prior hyperparameters required by the model: the prior mean θ_0, prior covariance Σ_0, the Gamma shape and rate parameters a and b, the covariance Σ_Θ, or the quantization size J used in equation (16). These quantities directly determine the posterior updates (4)-(7) and the KG selection decisions (17)-(19). Without them the experiments are not reproducible, and the sensitivity of the reported improvements to prior specification is unknown. Please report the exact values used and add a brief sensitivity analysis over reasonable variations.
[§3.2, Eq. (1)] Equation (1) assumes that the logit of validation accuracy is an exactly linear, homoskedastic function of one-hot-encoded prompt features: η_x = Θ^T x + ε. All Bayesian updates and the KG policy inherit this assumption, yet the paper gives no validation of it. Nonlinear effects and feature interactions are not modeled, and no diagnostics are provided. Please add a robustness check, such as posterior predictive checks on held-out validation scores or a comparison against a model with pairwise feature interactions, to show that the qualitative conclusions do not depend on the linearity and homoskedasticity assumptions.

minor comments (6)

[§3.3, Eq. (4)] In equation (4), the numerator uses x_t while the rest of the update uses x_n; the index should be made consistent (presumably x_n throughout).
[Algorithm 1, line 11] The Update call is written as Update(S, logit(x)), but the argument should be the observed score logit(u_x), not the representation x.
[§4, Eqs. (13) and (16)] There are typos: 'quanitity' in equation (13), and the expression in equation (16) appears as 'JX_{j=1}' where a summation symbol is intended.
[§6] The definition of the 13 'challenging' tasks mentions validation scores below 80% and 'relatively large variance', but the exact variance threshold is not stated and the task names are not listed. Please include the task list and threshold in an appendix.
[Table 2] The row labeled 'Standard deviation' is the average of within-task standard deviations across replications; this does not equal the standard error of the reported mean and is not appropriate for comparing methods. Please report standard errors of the mean or confidence intervals.
[Table 2] The 'Improvement of SOPL-KG per task' row is not defined. State whether it is the mean of per-task relative improvements or the relative improvement of the aggregate mean, and include the formula.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the KG-based prompt optimization is evaluated against external baselines on held-out test data, and the cited optimal-learning machinery is independent methodological support.

full rationale

We found no load-bearing circularity. The paper's central empirical claim is that SOPL-KG outperforms benchmark prompt-engineering methods on instruction induction tasks; this is evaluated by comparing held-out test scores (Section 6, Table 2) across methods, with no parameter of the compared methods fitted to those test outcomes. Equation (1) posits a linear-Gaussian model eta_x = Theta^T x + epsilon; although this assumption is not empirically validated, it is a modeling choice, not a result derived from the claim. The Bayesian update equations (4)-(7) and the KG quantities (13)-(16) are standard machinery from the optimal-learning literature (e.g., [9, 26]), and the citation to [24], co-authored by Moazeni, supplies a computational reformulation (Propositions 6 and 7) rather than the paper's empirical conclusion. No 'prediction' in the paper is equivalent by construction to fitted inputs: the final prompt is selected on validation scores and scored on held-out test data, and the baseline comparisons use the same evaluation budget. The absence of significance testing is a statistical-evidence weakness, not a circularity, and it does not make the derivation self-referential. We therefore assign score 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the linear model in equation (1), the manual feature design chosen by the authors, and unstated prior and algorithmic hyperparameters. No new physical or conceptual entities are introduced.

free parameters (5)

Prior mean θ_0
Required for the Bayesian regression prior (2), not reported in the paper, so the results depend on an unstated choice.
Prior covariance Σ_0
Required for (2), not reported, and it directly affects the posterior updates and the KG decisions.
Prior shape a, b for ρ
Gamma prior (3) parameters are not reported; they influence the variance of posterior samples and the t-distribution used in the KG quantity.
Σ_Θ
Covariance of the coefficient process in (1), not reported; it controls the assumed noise from model misspecification.
Quantization size J
Number of quantization points in (16) for the KG approximation, not specified in the paper.

assumptions (4)

domain assumption Linear model η_x = Θ^T x + ε with additive independent Gaussian noise (equation 1)
The entire Bayesian learning and KG computation assume the logit of accuracy is linear in the one-hot feature vector, with no interactions or nonlinearities.
domain assumption Feature set in Table 1 adequately spans the prompt design space
The search space is restricted to the manually chosen features; any prompt outside this space cannot be found by the method.
domain assumption Prior distributions (Normal-Gamma) correctly reflect the uncertainty
The choice of prior family is standard, but the hyperparameters are unstated, so the effective prior is arbitrary and not validated.
domain assumption Validation accuracy is a reliable proxy for test performance
Prompt selection is based on validation scores, and the final prompt is chosen by best validation score; this assumes validation accuracy correlates with held-out test accuracy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models." pith.science (2026). https://pith.science/paper/X3GFW22K

@misc{pith2026250103508,
  author       = {Pith},
  title        = {Pith review of: A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X3GFW22K}},
  note         = {Machine review of arXiv:2501.03508}
}

read the original abstract

Designing effective prompts is essential to guiding large language models (LLMs) toward desired responses. Automated prompt engineering aims to reduce reliance on manual effort by streamlining the design, refinement, and optimization of natural language prompts. This paper proposes an optimal learning framework for automated prompt engineering, designed to sequentially identify effective prompt features while efficiently allocating a limited evaluation budget. We introduce a feature-based method to express prompts, which significantly broadens the search space. Bayesian regression is employed to utilize correlations among similar prompts, accelerating the learning process. To efficiently explore the large space of prompt features for a high quality prompt, we adopt the forward-looking Knowledge-Gradient (KG) policy for sequential optimal learning. The KG policy is computed efficiently by solving mixed-integer second-order cone optimization problems, making it scalable and capable of accommodating prompts characterized only through constraints. We demonstrate that our method significantly outperforms a set of benchmark strategies assessed on instruction induction tasks. The results highlight the advantages of using the KG policy for prompt learning given a limited evaluation budget. Our framework provides a solution to deploying automated prompt engineering in a wider range applications where prompt evaluation is costly.

Figures

Figures reproduced from arXiv: 2501.03508 by the authors.

**Figure 2.** Meta Prompt Templates [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Paraphrasing Template [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Evaluation Template to compute a score by comparing the LLM response with the correct output qi . The average score across all validation examples is used as the score ux. 5.1 Benchmark Methods We compare our method with two benchmarks EvoPrompt [12] and TRIPLE [33]. …

**Figure 5.** Figure 5: Test performance on 13 tasks for different methods. The height of each bar represents the average test score and the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 7.** Figure 7: Comparison of the average test score between our [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Evolutionary Computation and Large Language Models: A Survey of Methods, Synergies, and Applications
cs.NE 2025-05 conditional novelty 4.0 of 10

A survey that maps bidirectional synergies between evolutionary computation and large language models and proposes a taxonomy plus research gaps.

Reference graph

Works this paper leans on

39 extracted references · 30 canonical work pages · cited by 1 Pith paper

[1]

Normalized (pointwise) mutual infor- mation in collocation extraction

Gerlof Bouma. Normalized (pointwise) mutual infor- mation in collocation extraction. Proceedings of GSCL, 30:31–40, 2009

work page 2009
[2]

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott G...

work page 1901
[3]

Instructzero: Efficient instruc- tion optimization for black-box large language mod- els

Lichang Chen, Jiuhai Chen, Tom Goldstein, Heng Huang, and Tianyi Zhou. Instructzero: Efficient instruc- tion optimization for black-box large language mod- els. In International Conference on Machine Learning, 2024

work page 2024
[4]

Bayesian ideas and discrete event sim- ulation: why, what and how

Stephen E Chick. Bayesian ideas and discrete event sim- ulation: why, what and how. InProceedings of the 2006 Winter Simulation Conference, pages 96–106, 2006

work page 2006
[5]

Sequential sampling to myopically maximize the expected value of information

Stephen E Chick, J ¨urgen Branke, and Christian Schmidt. Sequential sampling to myopically maximize the expected value of information. INFORMS Journal on Computing, 22(1):71–80, 2010

work page 2010
[6]

Optimal information blending with measurements in the l 2 sphere

Boris Defourny, Ilya O Ryzhov, and Warren B Pow- ell. Optimal information blending with measurements in the l 2 sphere. Mathematics of Operations Research, 40(4):1060–1088, 2015

work page 2015
[7]

Rephrase and respond: Let large language models ask better questions for themselves

Yihe Deng, Weitong Zhang, Zixiang Chen, and Quan- quan Gu. Rephrase and respond: Let large language models ask better questions for themselves. arXiv preprint arXiv:2311.04205, 2023

arXiv 2023
[8]

Promptbreeder: Self-referential self-improvement via prompt evolution

Chrisantha Fernando, Dylan Sunil Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt ¨aschel. Promptbreeder: Self-referential self-improvement via prompt evolution. In International Conference on Ma- chine Learning, 2024

work page 2024

Show all 39 references

[9]

The knowledge-gradient policy for correlated normal be- liefs

Peter Frazier, Warren Powell, and Savas Dayanik. The knowledge-gradient policy for correlated normal be- liefs. INFORMS journal on Computing, 21(4):599–613, 2009

2009
[10]

Consistency of sequential bayesian sampling policies

Peter I Frazier and Warren B Powell. Consistency of sequential bayesian sampling policies. SIAM Journal on Control and Optimization, 49(2):712–731, 2011

2011
[11]

A knowledge-gradient policy for sequential information collection

Peter I Frazier, Warren B Powell, and Savas Dayanik. A knowledge-gradient policy for sequential information collection. SIAM Journal on Control and Optimization, 47(5):2410–2439, 2008

2008
[12]

Connecting large language models with evolu- tionary algorithms yields powerful prompt optimizers

Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolu- tionary algorithms yields powerful prompt optimizers. In International Conference on Learning Representa- tions, 2024

2024
[13]

Bayesian look ahead one-stage sampling allocations for selection of the best population

Shanti S Gupta and Klaus J Miescke. Bayesian look ahead one-stage sampling allocations for selection of the best population. Journal of statistical planning and inference, 54(2):229–244, 1996

1996
[14]

Op- timal learning in linear regression with combinatorial feature selection

Bin Han, Ilya O Ryzhov, and Boris Defourny. Op- timal learning in linear regression with combinatorial feature selection. INFORMS Journal on Computing , 28(4):721–735, 2016

2016
[15]

Instruction induction: From few examples to natural language task descriptions

Or Honovich, Uri Shaham, Samuel R Bowman, and Omer Levy. Instruction induction: From few examples to natural language task descriptions. arXiv preprint arXiv:2205.10782, 2022

2022 arXiv
[16]

Apeer: Automatic prompt engineering en- hances large language model reranking

Can Jin, Hongwu Peng, Shiyu Zhao, Zhenting Wang, Wujiang Xu, Ligong Han, Jiahui Zhao, Kai Zhong, Sanguthevar Rajasekaran, and Dimitris N Metaxas. Apeer: Automatic prompt engineering en- hances large language model reranking. arXiv preprint arXiv:2406.14449, 2024

2024 arXiv
[17]

Better zero-shot reasoning with role- play prompting

Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, and Xi- aohang Dong. Better zero-shot reasoning with role- play prompting. In Proceedings of the 2024 Confer- ence of the North American Chapter of the Association for Computational Lingui...

2024
[18]

Large language models understand and can be enhanced by emotional stimuli

Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, and Xing Xie. Large language models understand and can be enhanced by emotional stimuli. arXiv preprint arXiv:2307.11760, 2023

2023 arXiv
[19]

Use your INSTINCT: IN- STruction optimization for LLMs using neural bandits 11 coupled with transformers

Xiaoqiang Lin, Zhaoxuan Wu, Zhongxiang Dai, Wenyang Hu, Yao Shu, See-Kiong Ng, Patrick Jaillet, and Bryan Kian Hsiang Low. Use your INSTINCT: IN- STruction optimization for LLMs using neural bandits 11 coupled with transformers. In International Conference on Machine Learning, 2024

2024
[20]

Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Comput- ing Surveys, 55(9):1–35, 2023

2023
[21]

Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity

Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguis- tics (Volume...

2022
[22]

Prompt engineering through the lens of optimal control

Yifan Luo, Yiming Tang, Chengfeng Shen, Zhen- nan Zhou, and Bin Dong. Prompt engineering through the lens of optimal control. arXiv preprint arXiv:2310.14201, 2023

2023 arXiv
[23]

Rlprompt: Optimiz- ing discrete text prompts with reinforcement learning

Deng Mingkai and Wang Jianyu. Rlprompt: Optimiz- ing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022

2022
[24]

Sequential learning in designing marketing campaigns for market entry

Somayeh Moazeni, Boris Defourny, and Monika J Wilczak. Sequential learning in designing marketing campaigns for market entry. Management Science , 66(9):4226–4245, 2020

2020
[25]

Reinforcement Learning and Stochas- tic Optimization: A Unified Framework for Sequential Decisions

Warren B Powell. Reinforcement Learning and Stochas- tic Optimization: A Unified Framework for Sequential Decisions. John Wiley & Sons, 2022

2022
[26]

Optimal learning, volume 841

Warren B Powell and Ilya O Ryzhov. Optimal learning, volume 841. John Wiley & Sons, 2012

2012
[27]

Grips: Gradient-free, edit-based instruction search for prompting large language models

Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: Gradient-free, edit-based instruction search for prompting large language models. arXiv preprint arXiv:2203.07281, 2022

2022 arXiv
[28]

Automatic prompt op- timization with ”gradient descent” and beam search

Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chen- guang Zhu, and Michael Zeng. Automatic prompt op- timization with ”gradient descent” and beam search. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

2023
[29]

Markov decision processes: dis- crete stochastic dynamic programming

Martin L Puterman. Markov decision processes: dis- crete stochastic dynamic programming. John Wiley & Sons, 2014

2014
[30]

Language mod- els are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language mod- els are unsupervised multitask learners. OpenAI Blog, 2019

2019
[31]

A sys- tematic survey of prompt engineering in large language models: Techniques and applications

Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. A sys- tematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927, 2024

2024 arXiv
[32]

Quantifying language models’ sensitivity to spu- rious features in prompt design or: How i learned to start worrying about prompt formatting

Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models’ sensitivity to spu- rious features in prompt design or: How i learned to start worrying about prompt formatting. In Interna- tional Conference on Learning Representations, 2024

2024
[33]

Best arm identification for prompt learning under a lim- ited budget

Chengshuai Shi, Kun Yang, Jing Yang, and Cong Shen. Best arm identification for prompt learning under a lim- ited budget. arXiv preprint arXiv:2402.09723, 2024

2024 arXiv
[34]

Toward human readable prompt tuning: Kubrick’s the shining is a good movie, and a good prompt too? In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtz- man, Yulia Tsvetkov, and Luke Zettlemoyer. Toward human readable prompt tuning: Kubrick’s the shining is a good movie, and a good prompt too? In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

2023
[35]

Autoprompt: Eliciting knowledge from language models with automatically generated prompts

Taylor Shin, Yasaman Razeghi, Robert L Logan IV , Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980 , 2020

2010 arXiv
[36]

Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery

Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Gold- blum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural In- formation Processing Systems, 36, 2024

2024
[37]

Large language models are diverse role- players for summarization evaluation

Ning Wu, Ming Gong, Linjun Shou, Shining Liang, and Daxin Jiang. Large language models are diverse role- players for summarization evaluation. In International Conference on Natural Language Processing and Chi- nese Computing, pages 695–707, 2023

2023
[38]

Calibrate before use: Improving few- shot performance of language models

Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few- shot performance of language models. In International Conference on Machine Learning, pages 12697–12706, 2021

2021
[39]

Large language models are human-level prompt engi- neers

Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engi- neers. In International Conference on Learning Repre- sentations, 2023. 12

2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Normalized (pointwise) mutual infor- mation in collocation extraction

Gerlof Bouma. Normalized (pointwise) mutual infor- mation in collocation extraction. Proceedings of GSCL, 30:31–40, 2009

work page 2009

[2] [2]

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott G...

work page 1901

[3] [3]

Instructzero: Efficient instruc- tion optimization for black-box large language mod- els

Lichang Chen, Jiuhai Chen, Tom Goldstein, Heng Huang, and Tianyi Zhou. Instructzero: Efficient instruc- tion optimization for black-box large language mod- els. In International Conference on Machine Learning, 2024

work page 2024

[4] [4]

Bayesian ideas and discrete event sim- ulation: why, what and how

Stephen E Chick. Bayesian ideas and discrete event sim- ulation: why, what and how. InProceedings of the 2006 Winter Simulation Conference, pages 96–106, 2006

work page 2006

[5] [5]

Sequential sampling to myopically maximize the expected value of information

Stephen E Chick, J ¨urgen Branke, and Christian Schmidt. Sequential sampling to myopically maximize the expected value of information. INFORMS Journal on Computing, 22(1):71–80, 2010

work page 2010

[6] [6]

Optimal information blending with measurements in the l 2 sphere

Boris Defourny, Ilya O Ryzhov, and Warren B Pow- ell. Optimal information blending with measurements in the l 2 sphere. Mathematics of Operations Research, 40(4):1060–1088, 2015

work page 2015

[7] [7]

Rephrase and respond: Let large language models ask better questions for themselves

Yihe Deng, Weitong Zhang, Zixiang Chen, and Quan- quan Gu. Rephrase and respond: Let large language models ask better questions for themselves. arXiv preprint arXiv:2311.04205, 2023

arXiv 2023

[8] [8]

Promptbreeder: Self-referential self-improvement via prompt evolution

Chrisantha Fernando, Dylan Sunil Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt ¨aschel. Promptbreeder: Self-referential self-improvement via prompt evolution. In International Conference on Ma- chine Learning, 2024

work page 2024