Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that natural-language critiques can serve as the direct training signal for a query-specific prompt generator, yielding state-of-the-art prompt accuracy on the GSMHard and MATH benchmarks.

desk verdict First to train a prompt model on textual rewards directly, but the current evidence doesn't yet isolate that claim from numerical-reward conditioning. read the letter →

arxiv 2507.18618 v1 pith:XUPCXYLY submitted 2025-07-24 cs.CL cs.LG

Andreea Nica , Ivan Zakazov , Nicolas Mario Baldwin , Saibo Geng , Robert West This is my paper
classification cs.CLcs.LG
keywords promptoptimizationtextualrewardsquery-dependentpromptingmathematicalreasoningsupervisedfine-tuningself-improvementnaturallanguagefeedbacklargemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TRPrompt claims that prompt optimization can be driven by natural-language critiques rather than by numeric correctness scores, and that this makes query-specific prompts better on hard math problems. The method generates synthetic training pairs: a prompt model produces a tailored instruction for a question, a target model answers with that prompt, and a reward model writes a free-text critique explaining how well the prompt guided the reasoning. The prompt model is then fine-tuned by supervised learning to reproduce good prompts when conditioned on such critiques, and the particular critique used as the conditioning signal is re-optimized after every round. On GSMHard and MATH, the paper reports accuracy gains over numerical-reward optimizers, with a model that needs no hand-crafted seed prompts. A sympathetic reader would care because sparse scalar rewards are a bottleneck in many alignment tasks, and this is a concrete recipe for replacing them with richer text feedback.

What carries the argument

The load-bearing mechanism is the iterative three-step loop. A textual reward is a free-form natural-language critique $t=R_{\text{textual}}(q, p, \hat{y}, y^*)$ produced by a reward model from the question, the prompt, the target model's answer, and the ground truth; the prompt model is trained by supervised fine-tuning to maximize the log-likelihood of a generated prompt given the question and the reward text. The third ingredient is the optimal-textual-reward search: after each fine-tuning round, a train-free text-based optimizer rewrites the conditioning critique so that it tracks the current prompt model's failure modes, preventing overfitting to stale feedback. The work these pieces do together is to convert a flat 0/1 outcome into a content-bearing training distribution that separates one failed prompt from another.

What would settle it

Look inside the training pairs: measure how much of the critique text is predictable from the binary correctness label. If a probe trained only on the 0/1 outcome can reproduce the critiques at near-ceiling accuracy—or if two prompts that both produce wrong answers routinely receive identical critiques—then the textual reward is not higher-resolution than a numerical one, and the central claim fails. A complementary check is to strip the correctness information from the critiques and retrain; if accuracy on GSMHard and MATH drops to the numerical-reward baseline, the signal was mostly the binary echo.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that textual rewards are not just an inference-time prompt-refinement tool but can be a direct supervision signal for training a prompt generator. The paper formalizes the query-dependent prompting problem as a dual objective: maximize the target model's accuracy by choosing a prompt model, and simultaneously choose the 'optimal textual reward'—the critique that, when fed to the prompt model, yields the most accurate prompts. The experiments support the claim that with this dual loop, a single 8-billion-parameter instruction-tuned model can serve as target, reward model, and prompt model, and the resulting query-level prompts outperform numerical-reward prompt optimizers on GSMHard (31.76%) and MATH (41.37%). The key observation in the motivating example is that two different prompts can both yield a wrong answer—same numerical reward of zero—yet receive different textual critiques, and that difference is what makes the learned signal higher-resolution.

Load-bearing premise

The load-bearing premise is that the free-text critiques actually describe how the prompt led the target model astray, rather than merely echoing whether the answer was right or wrong; if they only paraphrase the correctness label, the textual reward adds no information over a binary score and the claimed advantage collapses.

Editorial extensions

If this is right

  • Prompt optimization for math reasoning can be bootstrapped from scratch: no expert-written seed prompts or manually designed prompt pools are needed to reach competitive query-level accuracy.
  • The same model family can serve as target, reward generator, and prompt generator, so the pipeline supports a form of self-improvement that does not require a separate stronger model for supervision.
  • Because the training data is synthesized on the fly, adapting TRPrompt to a new dataset reduces to supplying questions with ground-truth answers and running the three-step loop.
  • The gains concentrate where the target model fails often; on easier benchmarks the text signal is mostly redundant with the binary label, so the method's advantage is diagnostic of task difficulty.
  • The iterative reward search is necessary: without re-optimizing the conditioning critique, the prompt model overfits to stale feedback and accuracy declines across rounds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same recipe should transfer to any domain where scalar rewards are flat—code repair, creative writing, or safety judgments—where many bad outputs share the same score but differ in why they fail.
  • Editorial inference: if the advantage really comes from the critique's information content, then a controlled probe should show that the textual rewards carry information about the prompt that is not contained in the binary correctness label; a simpler test is to compare TRPrompt against SFT on the same prompt–question pairs conditioned on a one-word proxy for the critique.
  • Editorial inference: the paper's same-family design leaves open whether a stronger or weaker reward model changes the gain; a testable extension is to vary the reward model independently of the target model and measure whether the textual signal's usefulness scales with critique quality.
  • Editorial inference: replacing the expensive non-parallel reward search with a cheaper, parallelizable critique sampler would test whether the boost comes from searching the reward space or merely from conditioning on language feedback at all.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces TRPrompt, an iterative framework for query-dependent prompt optimization in which a prompt model (Llama-3-8B-Instruct) is fine-tuned by supervised learning on synthetic examples that pair a query-dependent prompt with a textual reward (a natural-language critique) generated by a reward model from the same model family. The optimal textual reward is periodically updated with Textgrad (using GPT-4o-mini), and the loop repeats for four iterations. The authors report that TRPrompt outperforms previous query-dependent prompt optimizers (QPO, Prompt-OIRL) and the CoT baseline on GSMHard and MATH, and analyze the contribution of the SFT and reward-search steps through ablations.

Significance. If the central claim is supported, TRPrompt would be a useful step toward using natural-language feedback as a training signal for prompt models, reducing reliance on handcrafted initial prompts and on numerical rewards. The framework is fully automated, does not require prior manual prompt sets, and the paper provides a clear algorithmic description plus detailed appendices on the synthetic data construction and training setup. The main weakness is that the experiments do not isolate the claimed advantage of textual rewards over numerical rewards, because the closest train-free textual-reward baselines are absent from the main comparison and no ablation replaces the textual reward with a binary label under otherwise identical conditions. The reported margins are small and are not accompanied by variance estimates or a statement that baseline numbers were produced under the same target model and evaluation protocol, so the state-of-the-art claim is conditional on additional evidence.

major comments (4)
  1. [Table 3 and Section 5] The main comparison omits the two most relevant textual-reward baselines, Textgrad and APO. Since the paper's claimed novelty is using textual rewards inside training rather than in a train-free loop, the absence of same-protocol Textgrad and APO results makes it impossible to tell whether the reported gains come from the training mechanism or simply from using a different optimizer on the same reward signal. The authors should report Textgrad and APO run under the same target model (Llama-3-8B-Instruct), same temperature, and same evaluation protocol, or clearly justify why these baselines are not applicable.
  2. [Algorithm 1 and Eq. (3)] The SFT objective in Eq. (3) trains the prompt model to reproduce its own previously generated prompts, conditioned on the critique of those same prompts. This is essentially self-imitation with relabeling, and the only new information entering the loop across iterations is the Textgrad-updated optimal reward and the critique text. The ablation in Figure 7 shows that removing the Textgrad step degrades performance, but it does not isolate the role of the textual reward versus a numerical reward. A load-bearing control is missing: keep everything identical but replace the textual reward ti with the binary label (correct/incorrect), and compare accuracies. Without this control, the claim that textual rewards provide a higher-resolution training signal than numerical rewards is not demonstrated.
  3. [Table 3 and Appendix A.2] The reported comparisons against QPO and Prompt-OIRL appear to use published numbers that may come from different target models, decoding settings, and evaluation splits. The paper states the temperature (0.001) and evaluation split for TRPrompt, but it does not state whether the baseline accuracies were produced under the same conditions, nor does it provide error bars or multiple seeds. The margins over the strongest baselines are about +1% on GSMHard and +2% on MATH, which are within the range of typical protocol differences; without matched-protocol baselines and variance estimates, the state-of-the-art claim is not yet supported.
  4. [Section 5 and Appendix A.1] The effectiveness of the textual reward signal depends on the critiques being contentful and reliable, but the paper provides no quantitative analysis of critique quality. Because the reward model is the same Llama-3-8B-Instruct family as the target model, it is possible that many critiques are hallucinated or mostly echo the binary correctness label; in that case Eq. (3) is effectively a re-labeling of a binary outcome. The authors should include an analysis or a sanity check showing that the textual rewards contain information beyond the correctness label and that this information changes as the prompt model improves.
minor comments (5)
  1. [Author affiliations] The affiliation for Saibo Geng is spelled 'EFPL' rather than 'EPFL'; this should be corrected.
  2. [Section 2] The notation 'P=∈ V∞' appears to contain a typo; it should likely be 'P ⊂ V∞'.
  3. [Abstract and Section 1] The phrase 'is being iteratively improved' in the abstract is grammatically awkward; consider rewriting to 'is iteratively improved'.
  4. [Throughout] The paper uses both 'Textgrad' and 'TextGrad' in different places; the spelling should be made consistent.
  5. [Section 5.1 and Appendix A.2] The paper reports only final accuracies and does not report the number of seeds or standard deviations for any of the TRPrompt results; figures such as Figures 4 and 6 would be more informative with error bars or at least a statement about the number of runs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main evaluation is external and the reward search is validation-based; the self-imitative SFT loop is a methodological concern, not a circular reduction.

full rationale

TRPrompt's central result is evaluated on held-out test sets (appendix A.2: 'All the final results we report in the paper are on the dedicated test set for each dataset'), with the optimal textual reward and the best checkpoint selected on a validation split. The dual objective in Eqs. 1–2 is an alternating optimization over the prompt model and the reward, not a definitional identity. Algorithm 1's SFT step (Eq. 3) does train the prompt model to reproduce its own previously generated prompts conditioned on critiques of those prompts, which is a self-bootstrapping loop and a legitimate experimental limitation: the paper does not ablate a binary-correctness conditioning string, so the specific advantage of textual rewards over numerical rewards is not isolated. However, this is not a circular reduction by construction: test-time prompts are generated for unseen questions, and the reported accuracies are not equal to any fitted training quantity. The paper contains no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled via citation; Textgrad is external prior work used as a component. The acknowledged limitations (reduced gains on easier datasets, high computational cost) also do not indicate circularity. Therefore the derivation is self-contained against external benchmarks, and the main weaknesses are experimental controls rather than circular reasoning.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the assumption that natural-language critiques carry more usable signal than the binary correctness label, that the same-family reward model writes accurate critiques, that the Textgrad search on the validation split yields a transferable optimal reward, and that the prompt model's own prior outputs are a useful SFT training distribution. Hyperparameters are reported but not swept or justified with sensitivity analysis. No new physical entities are introduced.

free parameters (7)
  • Iteration count K = 4
    Number of outer loop rounds, chosen by hand in Appendix A.2.
  • Synthetic dataset size per iteration = 800 questions
    Randomly sampled from the train split each iteration, chosen by hand in Appendix A.2.
  • SFT epochs = 2
    Fixed for all iterations, listed in Appendix A.2.
  • Learning rate = 2e-5
    Adam optimizer with linear decay, listed in Appendix A.2.
  • LoRA rank and alpha = r=256, alpha=256
    Memory and cost choice, listed in Appendix A.2.
  • Generation temperatures = 0.9 prompt generation, 0.001 evaluation and reward generation
    Chosen to balance diversity and determinism, listed in Appendix A.2.
  • Textgrad iterations = 10
    Number of reward search steps, listed in Appendix A.2.
assumptions (5)
  • domain assumption Textual rewards are more informative than binary numerical rewards for query-level prompt optimization.
    Section 1 and Figure 1 motivate the entire method with this premise; no direct measurement of information content is provided.
  • domain assumption The hand-written reward-model meta-instruction yields accurate critiques from the same-family Llama-3-8B-Instruct model.
    Appendix A.1 describes the template but reports no reward-quality validation or agreement statistics.
  • domain assumption Textgrad with GPT-4o-mini on the validation set finds an optimal textual reward that transfers to held-out test questions.
    Algorithm 1 Step 3 relies on this; no convergence or transfer analysis is reported.
  • domain assumption Using the prompt model's own previous prompts as SFT labels in Eq. 3 is a useful training distribution for learning better prompts.
    This is the bootstrapping premise; Figure 7 shows SFT alone degrades, so the assumption depends on the reward-search loop.
  • standard math Answer equality r(y*, yhat)=1{yhat=y*} is a sufficient measure of reasoning correctness.
    Defined in Section 2 and used throughout; standard for these benchmarks, though it ignores equivalent correct answers with different formatting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards." pith.science (2026). https://pith.science/paper/XUPCXYLY

@misc{pith2026250718618,
  author       = {Pith},
  title        = {Pith review of: TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XUPCXYLY}},
  note         = {Machine review of arXiv:2507.18618}
}
read the original abstract

Prompt optimization improves the reasoning abilities of large language models (LLMs) without requiring parameter updates to the target model. Following heuristic-based "Think step by step" approaches, the field has evolved in two main directions: while one group of methods uses textual feedback to elicit improved prompts from general-purpose LLMs in a training-free way, a concurrent line of research relies on numerical rewards to train a special prompt model, tailored for providing optimal prompts to the target model. In this paper, we introduce the Textual Reward Prompt framework (TRPrompt), which unifies these approaches by directly incorporating textual feedback into training of the prompt model. Our framework does not require prior dataset collection and is being iteratively improved with the feedback on the generated prompts. When coupled with the capacity of an LLM to internalize the notion of what a "good" prompt is, the high-resolution signal provided by the textual rewards allows us to train a prompt model yielding state-of-the-art query-specific prompts for the problems from the challenging math datasets GSMHard and MATH.

Figures

Figures reproduced from arXiv: 2507.18618 by the authors.

Figure 1
Figure 1. Motivating Example: Textual reward is more expressive than numerical reward. When us￾ing both prompts, the target model reaches a wrong answer. While the numerical reward is the same (0), the textual reward distinctively captures each prompt fails to guide the target language model in answering correctly. A textual reward is an evaluation expressed in the natural language of a prompt, serving as a qualitative measur… view at source ↗
Figure 2
Figure 2. Overview of our query-dependent prompt optimization iterative pipeline, containing 3 steps: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Step 1: Prompt generation and textual re￾ward calculation The quality of these generations can be evaluated using the following metric r(y ∗ i , yˆi) = 1{yˆi = y ∗ i }, which checks for equality between the generated answer and the ground-truth. Textual reward. In order to assess the quality of a query-dependent prompt, we will use a textual reward t ∈ T = V∞. The textual reward represents feedback expressed in natu… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Accuracy gain (in per￾centage points difference) com￾pared to the base model across iter￾ations. Each iteration improves the ability of the prompt model to gen￾erate prompts that guide the target model to the correct answer [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Intermediate performance obtained by the prompt model after each step in our pipeline. Both SFT and the Optimal Reward Search step contribute to enhancing the prompt model ability to generate efficient prompts [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Meta-Instruction template and example used to generate query-dependent prompts condi [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Instruction template and example used to guide the Reward Model in generating a textual [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Example of a training pair used in Supervised Fine-Tuning. The Prompt Model is trained [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning

    cs.CL 2025-11 unverdicted novelty 6.0 of 10

    Prompt-R1 is an end-to-end RL framework where a small-scale LLM collaborates with large-scale LLMs by generating prompts, using a dual-constrained reward to optimize correctness and quality, and outperforms baselines ...

Reference graph

Works this paper leans on

22 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024

    2024

  2. [2]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  3. [3]

    Rlprompt: Optimizing discrete text prompts with reinforce- ment learning

    Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. Rlprompt: Optimizing discrete text prompts with reinforce- ment learning. arXiv preprint arXiv:2205.12548, 2022

    arXiv 2022

  4. [4]

    Pal: Program-aided language models

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. arXiv preprint arXiv:2211.10435, 2022

    arXiv 2022

  5. [5]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021

    2021

  6. [6]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems , 35:22199–22213, 2022

    2022

  7. [7]

    Qpo: Query-dependent prompt optimization via multi-loop offline reinforcement learning

    Yilun Kong, Hangyu Mao, Qi Zhao, Bin Zhang, Jingqing Ruan, Li Shen, Yongzhe Chang, Xueqian Wang, Rui Zhao, and Dacheng Tao. Qpo: Query-dependent prompt optimization via multi-loop offline reinforcement learning. arXiv preprint arXiv:2408.10504, 2024

    arXiv 2024

  8. [8]

    Gsm-plus: A comprehen- sive benchmark for evaluating the robustness of llms as mathematical problem solvers

    Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. Gsm-plus: A comprehen- sive benchmark for evaluating the robustness of llms as mathematical problem solvers. arXiv preprint arXiv:2402.19255, 2024

    arXiv 2024

Show all 22 references
  1. [9]

    Test-time preference optimization: On-the-fly alignment via iterative textual feedback

    Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, and Yu Cheng. Test-time preference optimization: On-the-fly alignment via iterative textual feedback. arXiv preprint arXiv:2501.12895, 2025

    2025 arXiv

  2. [10]

    Guiding large language models via directional stimulus prompting

    Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems, 36, 2024

    2024

  3. [11]

    Large language models as evolutionary optimizers

    Shengcai Liu, Caishun Chen, Xinghua Qu, Ke Tang, and Yew-Soon Ong. Large language models as evolutionary optimizers. In 2024 IEEE Congress on Evolutionary Computation (CEC), pages 1–8. IEEE, 2024

    2024

  4. [12]

    Gpt-4 technical report

    Josh OpenAI, Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 10

    2023 arXiv

  5. [13]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems , 35:277...

    2022

  6. [14]

    gradient descent

    Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with" gradient descent" and beam search.arXiv preprint arXiv:2305.03495, 2023

    2023 arXiv

  7. [15]

    Learning performance-improving code edits

    Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob Gardner, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, and Amir Yazdanbakhsh. Learning performance-improving code edits. arXiv preprint arXiv:2302.07867, 2023

    2023 arXiv

  8. [16]

    Query-dependent prompt evaluation and optimization with offline inverse rl

    Hao Sun, Alihan Hüyük, and Mihaela van der Schaar. Query-dependent prompt evaluation and optimization with offline inverse rl. In The Twelfth International Conference on Learning Representations, 2023

    2023

  9. [17]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824–24837, 2022

    2022

  10. [18]

    Improving reward models with synthetic critiques

    Zihuiwen Ye, Fraser Greenlee-Scott, Max Bartolo, Phil Blunsom, Jon Ander Campos, and Matthias Gallé. Improving reward models with synthetic critiques. arXiv preprint arXiv:2405.20850, 2024

    2024 arXiv

  11. [19]

    differentiation

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. Textgrad: Automatic" differentiation" via text. arXiv preprint arXiv:2406.07496, 2024

    2024 arXiv

  12. [20]

    Gonzalez

    Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, and Joseph E. Gonzalez. The wisdom of hindsight makes language models better instruction followers. In Proceedings of the 40th International Conference on Machine Learning , ICML’23. JMLR.org, 2023

    2023

  13. [21]

    Tempera: Test-time prompting via reinforcement learning

    Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E Gonzalez. Tempera: Test-time prompting via reinforcement learning. arXiv preprint arXiv:2211.11890, 2022

    2022 arXiv

  14. [22]

    Large language models are human-level prompt engineers

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910, 2022. A Technical Appendices and Supplementary Material A.1 Synthetic Dataset Creati...

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.