Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GReaTer claims that small open-source language models can optimize their own prompts by computing loss gradients through generated reasoning chains, beating text-feedback prompt optimizers that rely on GPT-4 on GSM8K, BBH, and FOLIO.

desk verdict Gradient-over-reasoning prompt optimization is a genuine advance, and the reported gains are plausible, but the paper needs an explicit fallback for empty candidate sets and some statistical rigor. read the letter →

arxiv 2412.09722 v2 pith:5TYNADQM submitted 2024-12-12 cs.CL

Sarkar Snigdha Sarathi Das , Ryo Kamoi , Bo Pang , Yusen Zhang , Caiming Xiong , Rui Zhang This is my paper · ORCID
classification cs.CL
keywords promptoptimizationgradientoverreasoningsmalllanguagemodelsdiscretetokensearchchain-of-thoughttaskstransferability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that prompt optimization does not have to depend on large, closed-source language models for textual feedback. GReaTer lets a small open-source model improve its own prompts by turning promising token suggestions into discrete candidates and ranking them with true numerical gradients, computed by backpropagating the task loss through the model's generated reasoning chain. Across GSM8K, BBH, and FOLIO reasoning benchmarks, the authors report that prompts found this way consistently outperform those found by state-of-the-art text-feedback methods, including baselines that use GPT-4 as the optimizer. If correct, this would let smaller models self-optimize for reasoning tasks more cheaply and without any reliance on expensive proprietary judges.

What carries the argument

The load-bearing mechanism is the 'gradient over reasoning' computation: the loss is taken from final answer logits after the model has generated a reasoning chain, so the backward pass flows through that reasoning and guides token selection. This is coupled with a candidate proposal stage that projects the vocabulary down to the intersection of top-k tokens over q sampled inputs, encoded as a one-hot token indicator so the discrete prompt becomes locally differentiable.

What would settle it

Run GReaTer with a configuration where the top-k candidate intersection across the q sampled inputs is empty at some position (for example, k=1 or a large q); if the procedure has no fallback, it is ill-defined at that step. Alternatively, replace the gradient-ranked token choice of Equation 7 with a random choice from the same candidate set: if task accuracy does not drop, the loss gradient is not what drives the reported gains.

Watch

Extended reading notes

Core claim

GReaTer's central claim is that a smaller language model can serve as its own prompt optimizer if the optimization signal is a real loss gradient rather than natural language critique. The paper proposes token candidates at each prompt position using the model's own top-k probabilities over a handful of task inputs, then generates a reasoning chain for each sample, extracts final answer logits through a formatted extractor prompt, and computes a cross-entropy loss with a perplexity regularizer. The gradient of that loss with respect to a one-hot token indicator at the position is used to rank the candidates, with the top few candidates verified by a forward pass. The authors report that this 'gradient over reasoning' approach outperforms APE, APO, PE2, and TextGrad on GSM8K, BBH, and FOLIO when optimizing Llama-3-8B-Instruct and Gemma-2-9B-it, and that the resulting prompts transfer across small models and to larger models.

Load-bearing premise

The method assumes that at every prompt position being optimized, the common top-k token candidates proposed across the sampled inputs form a non-empty set, leaving at least one token available to score.

Editorial extensions

If this is right

  • Small open-weight models can improve their own reasoning prompts without calling closed-source LLMs for feedback, removing a major cost and dependency.
  • Because each iteration costs one forward and one backward pass per sample, prompt optimization becomes cheaper than text-feedback methods that generate long critiques on chained error samples.
  • Optimized prompts transfer across small models and to larger models, so one optimization run on a small model can serve multiple deployment targets.
  • Gradient-guided discrete token search, previously practical only for fixed-input-output classification settings, is extended to free-form reasoning tasks with generated reasoning chains.
  • The performance gains reported are consistent and often exceed those from prompts optimized with GPT-4, PaLM-2-L, and GPT-3.5 on the same tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same candidate-proposal-plus-gradient machinery could be reused for other discrete text optimization problems, such as instruction selection in agents or few-shot example mining, whenever an end-to-end loss over model output is available.
  • A clean causal test of the 'reasoning' component would be to freeze the candidate set and replace gradient ranking with random selection: if the accuracy gain vanishes, the reasoning-aware gradient is what carries the result, not the candidate proposal stage alone.
  • Because the paper notes that some optimized prompts read awkwardly, layering a fluency constraint or dynamic top-k on top of the gradient selection is a natural extension that could make the method more deployable.
  • The reported success on a 1B-parameter model suggests the approach could eventually enable on-device prompt adaptation where no external optimizer is reachable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces GReaTer, a prompt optimization method for small language models that uses gradient information over generated reasoning to select replacement tokens at each prompt position. Token candidates are proposed by taking the intersection of the top-k token sets over a small sample of inputs; the model generates a reasoning chain and answer logits, and the loss gradient with respect to a one-hot token indicator guides the choice among candidates. Experiments on GSM8K, BBH, and FOLIO with Llama-3-8B-Instruct and Gemma-2-9B-it report consistent improvements over text-feedback baselines, better transferability across model sizes, and in some cases parity with prompts optimized by larger proprietary models. The paper also includes ablations, initial prompt sensitivity analysis, and a comparison with very small models.

Significance. If the results are reliable, the method demonstrates that small, open-source models can self-optimize prompts without relying on large proprietary LLMs, which is a practically important contribution. The idea of combining discrete prompt search with a reasoning-aware loss is novel, and the public code release is a strength. The main tables cover a wide range of tasks and two backbone models, and the transferability experiments give additional evidence for the method's usefulness. However, the core algorithm has an undefined case (empty candidate intersection) that needs to be addressed, and the absence of error bars or significance tests makes it difficult to assess the strength of the claimed consistent gains.

major comments (3)
  1. [Section 4.2, Eq. (4), Algorithm 1 line 7] The candidate set at each prompt position is defined as the intersection of per-sample top-k token sets: candidates_i = ∩_{x_j∈D_q} cand_{i,j}. This intersection can be empty when the q sampled inputs disagree on the top-k continuations, and Algorithm 1 provides no fallback: line 8 builds a one-hot indicator over an empty set, and lines 14-15 perform gradient selection over no candidates, leaving the update undefined. The paper neither proves non-emptiness nor reports the frequency of empty intersections, yet the algorithm is claimed to run for T=105 steps at every position. This is a load-bearing gap in the method description; please add a fallback (e.g., current token or union) and report how often it is triggered.
  2. [Tables 1 and 6, Figure 6] The central claim of consistent gains rests on single-run accuracy values without error bars, standard deviations, or significance tests. In Table 1, the gains on FOLIO/Gemma-2-9B (69.1 vs. 67.5) and GSM8K/Gemma-2-9B (89.4 vs. 88.6) are small; without variance estimates, these differences may not be reliable. Please report scores over multiple seeds (at least three) or a significance test for the main tables.
  3. [Section 5.3, Tables 2 and 3] The five BBH tasks are described as "randomly selected," but no random seed or deterministic selection protocol is provided. Although the task names are listed, the selection process is not reproducible, and the comparison to prompts optimized by large proprietary models is based on this subset. Provide the seed or a justification for the chosen subset.
minor comments (6)
  1. [Section 4.2] The description of the one-hot indicator is unclear: it states "with a value of one only for pi and zeros for all other candidates," which conflicts with the need for gradients over all candidate tokens. Clarify that ϵ_i is a vector over the candidate set, with a one at the index of the current token.
  2. [Algorithm 1] The set D used in lines 10, 12, and 15 is not defined in the algorithm input; it should be either D_task or D_q consistently.
  3. [Algorithm 1, line 19] When the prompt length changes dynamically (as described in Appendix B), the modulo operation should use the current prompt length rather than length(p_init).
  4. [Section 4.4, Eq. (7)] The notation "arg max_{ϵ_i}" is not well-defined for a vector; use a set of candidate indices or a similar clear formulation.
  5. [Figure 3] The ablation study of "Gradient Over Reasoning" reports only two BBH tasks; adding more tasks would strengthen the claim that the reasoning component is essential.
  6. [Section 5.1] The paper does not specify whether the reported scores in Table 1 are from a single run or averaged; please clarify the number of runs in the experimental setup.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GReaTer's optimization signal is task-label loss, and reported scores are held-out external evaluations.

full rationale

The paper's derivation chain is self-contained against the target claim. Candidate tokens are proposed by the LM's own forward probabilities (Eq. 3) and intersected over q samples (Eq. 4); reasoning is generated and answer logits are extracted (Eq. 5); the loss is cross-entropy against ground-truth labels plus a perplexity regularizer (Eq. 6); and token replacement is driven by gradients of that label-based loss (Eq. 7), with a forward-pass validation on the training set among the top-3 gradient choices. The external signal throughout is the task label y, which is not produced by the method or by the optimized prompt, so the prediction (held-out task accuracy) is not equivalent to the optimization objective by construction. Reported gains are measured on test/validation splits (e.g., GSM8K test set, FOLIO validation set, BBH held-out splits) rather than on the fitted training set, so the 'consistently outperforms' claim is an external empirical comparison. No load-bearing step is justified by self-citation; the cited related work is contextual, and the method does not invoke a uniqueness theorem or import an ansatz from the authors' prior work. The empty-intersection concern for Eq. 4 is a potential algorithmic robustness gap, not a circularity: it does not make the derivation equivalent to its inputs. Overall, the central claim has independent content and the optimization signal is genuinely external.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical entities. The free parameters are optimization hyperparameters, and the key domain assumption is that reasoning-chain gradients are a good search signal. The most fragile unstated assumption is the non-empty intersection of candidate tokens.

free parameters (6)
  • top-k (k=10) = 10
    Number of candidate tokens proposed per position in Eq. 3. Hyperparameter, not swept.
  • q = 5
    Number of sampled inputs used to propose candidates in Eq. 4. Hyperparameter, not swept.
  • mu = 3
    Number of top-gradient candidates forwarded for selection in Algorithm 1. Hyperparameter, not swept.
  • lambda = 0.2
    Weight for perplexity regularization in Eq. 6. Chosen by hand, not swept.
  • T = 105
    Total optimization steps in Algorithm 1. Stopping criterion, not justified by convergence analysis.
  • Initial prompt = 'Use proper logical reasoning and think step by step. Finally, give the actual correct answer.'
    Default initialization; ablation shows robustness, but still a design choice.
assumptions (3)
  • domain assumption Gradients over generated reasoning provide a useful optimization signal for prompt tokens.
    This is the core hypothesis of the paper, validated only empirically via ablation; no theoretical justification.
  • domain assumption A single sampled reasoning chain r per input yields a stable enough loss estimate for gradient-based selection.
    The loss in Eq. 2 uses one draw of r; the paper does not analyze variance or use multiple samples.
  • ad hoc to paper The intersection of top-k candidate sets across q inputs is non-empty.
    Eq. 4 defines candidates_i as an intersection; an empty intersection is not handled in Algorithm 1 or the text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers." pith.science (2026). https://pith.science/paper/5TYNADQM

@misc{pith2026241209722,
  author       = {Pith},
  title        = {Pith review of: GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5TYNADQM}},
  note         = {Machine review of arXiv:2412.09722}
}
read the original abstract

The effectiveness of large language models (LLMs) is closely tied to the design of prompts, making prompt optimization essential for enhancing their performance across a wide range of tasks. Many existing approaches to automating prompt engineering rely exclusively on textual feedback, refining prompts based solely on inference errors identified by large, computationally expensive LLMs. Unfortunately, smaller models struggle to generate high-quality feedback, resulting in complete dependence on large LLM judgment. Moreover, these methods fail to leverage more direct and finer-grained information, such as gradients, due to operating purely in text space. To this end, we introduce GReaTer, a novel prompt optimization technique that directly incorporates gradient information over task-specific reasoning. By utilizing task loss gradients, GReaTer enables self-optimization of prompts for open-source, lightweight language models without the need for costly closed-source LLMs. This allows high-performance prompt optimization without dependence on massive LLMs, closing the gap between smaller models and the sophisticated reasoning often needed for prompt refinement. Extensive evaluations across diverse reasoning tasks including BBH, GSM8k, and FOLIO demonstrate that GReaTer consistently outperforms previous state-of-the-art prompt optimization methods, even those reliant on powerful LLMs. Additionally, GReaTer-optimized prompts frequently exhibit better transferability and, in some cases, boost task performance to levels comparable to or surpassing those achieved by larger language models, highlighting the effectiveness of prompt optimization guided by gradients over reasoning. Code of GReaTer is available at https://github.com/psunlpgroup/GreaTer.

Figures

Figures reproduced from arXiv: 2412.09722 by the authors.

Figure 1
Figure 1. Comparison of textual feedback-based prompt optimization and GR [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overall workflow of GREATER. (i) The language model fLLM generates token candidates by conditioning on input samples. (ii) fLLM uses task input and current prompt to generate reasoning and extract final answer logits. (iii) The logits are used to calculate loss and compute gradient over generated reasoning with respect to the candidate tokens. These gradients determine the selection of candidate token to update the … view at source ↗
Figure 3
Figure 3. Ablation study on “Gradient Over Reasoning” in GR [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Efficacy of GREATER in zero-shot setting compared to five-shot inference with Llama-3-8B-Instruct. In-context learning has proven to be highly effective for reasoning tasks in large language models. This raises the question of whether prompt optimization provides any a…
Figure 5
Figure 5. Figure 5: Win/Draw/Loss Comparison of GREATER and SOTA prompt optimization techniques APO, TextGrad, APE, and PE2 in optimization with Llama-3-8B-Instruct. GREATER maintains a significant winning margin over these methods, highlighting its effectiveness in optimization. Finally,…
Figure 6
Figure 6. Figure 6: Full performance breakdown across 21 BBH tasks of GR [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]
Figure 7
Figure 7. Figure 7: Win/Draw/Loss Comparison of GREATER and SOTA prompt optimization techniques APO, TextGrad, APE, and PE2 in optimization with Gemma-2-9B-it. GREATER maintains winning margin over these methods, highlighting its effectiveness in optimization. 31 [PITH_FULL_IMAGE:figures…
Figure 8
Figure 8. Figure 8: Full performance breakdown across 21 BBH tasks of GR [PITH_FULL_IMAGE:figures/full_fig_p032_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Prompt Smart, Pay Less: Cost-Aware APO for Real-World Applications

    cs.LG 2025-07 conditional novelty 4.0 of 10

    APE-OPRO, a hybrid of APE and OPRO, achieves similar weighted F1 to OPRO at roughly 18% lower API cost on a 2,500-product commercial classification benchmark.

  2. SI-Agent: An Agentic Framework for Feedback-Driven Generation and Tuning of Human-Readable System Instructions for Large Language Models

    cs.AI 2025-07 reject novelty 4.0 of 10

    The paper proposes a multi-agent loop (instructor, follower, feedback) to auto-generate human-readable system prompts, claiming good benchmark performance and readability, but the supporting experiments are not reprod...

Reference graph

Works this paper leans on

42 extracted references · 9 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [3]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14165,

    arXiv 2005

  3. [5]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168,

    arXiv

  4. [6]

    In contrast, other methods show markedly less consistency due to their sole reliance on LLM judgment

    Upon closer inspection, it becomes evident that GR EATER consistently matches or surpasses the performance of other SOTA methods, highlighting the robustness and reliability of our approach. In contrast, other methods show markedly less consistency due to their sole reliance on LLM judgment. 17 Published as a conference paper at ICLR 2025 Figure 5: Win/Dr...

    work page 2025

  5. [8]

    Gradient-based adversarial attacks against text transformers

    Chuan Guo, Alexandre Sablayrolles, Herv ´e J ´egou, and Douwe Kiela. Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733,

    arXiv

  6. [9]

    Connecting large language models with evolutionary algorithms yields powerful prompt optimizers

    Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532,

    arXiv

  7. [10]

    Cold-attack: Jailbreaking llms with stealthiness and controllability

    Xingang Guo, Fangxu Yu, Huan Zhang, Lianhui Qin, and Bin Hu. Cold-attack: Jailbreaking llms with stealthiness and controllability. arXiv preprint arXiv:2402.08679,

    arXiv

  8. [12]

    Localized zeroth-order prompt optimization

    Wenyang Hu, Yao Shu, Zongmin Yu, Zhaoxuan Wu, Xiangqiang Lin, Zhongxiang Dai, See-Kiong Ng, and Bryan Kian Hsiang Low. Localized zeroth-order prompt optimization. arXiv preprint arXiv:2403.02993,

    arXiv

Show all 42 references
  1. [13]

    Dspy: Compiling declarative language model calls into self-improving pipelines

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T Joshi, Hanna Moazam, et al. Dspy: Compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714,

    arXiv

  2. [14]

    The power of scale for parameter-efficient prompt tuning

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691,

    arXiv

  3. [15]

    Prefix-tuning: Optimizing continuous prompts for generation.arXiv preprint arXiv:2101.00190,

    Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation.arXiv preprint arXiv:2101.00190,

    arXiv

  4. [16]

    Large language models as evolutionary optimizers

    11 Published as a conference paper at ICLR 2025 Shengcai Liu, Caishun Chen, Xinghua Qu, Ke Tang, and Yew-Soon Ong. Large language models as evolutionary optimizers. In 2024 IEEE Congress on Evolutionary Computation (CEC) , pp. 1–8. IEEE,

    2025

  5. [17]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451,

    arXiv

  6. [18]

    Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786,

    arXiv

  7. [19]

    Text and patterns: For effective chain of thought, it takes two to tango

    Aman Madaan and Amir Yazdanbakhsh. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint arXiv:2209.07686,

    arXiv

  8. [20]

    Grips: Gradient-free, edit-based in- struction search for prompting large language models

    Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: Gradient-free, edit-based in- struction search for prompting large language models. arXiv preprint arXiv:2203.07281,

    arXiv

  9. [21]

    Automatic prompt optimization with” gradient descent” and beam search

    Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with” gradient descent” and beam search. arXiv preprint arXiv:2305.03495,

    arXiv

  10. [22]

    Learning how to ask: Querying lms with mixtures of soft prompts

    Guanghui Qin and Jason Eisner. Learning how to ask: Querying lms with mixtures of soft prompts. arXiv preprint arXiv:2104.06599,

    arXiv

  11. [23]

    Gem- ini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean- baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gem- ini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv ...

    arXiv

  12. [24]

    Prompt programming for large language models: Beyond the few-shot paradigm

    Laria Reynolds and Kyle McDonell. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended abstracts of the 2021 CHI conference on human factors in computing systems, pp. 1–7,

    2021

  13. [25]

    Toward human readable prompt tuning: Kubrick’s the shining is a good movie, and a good prompt too? arXiv preprint arXiv:2212.10539,

    Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtzman, Yulia Tsvetkov, and Luke Zettlemoyer. Toward human readable prompt tuning: Kubrick’s the shining is a good movie, and a good prompt too? arXiv preprint arXiv:2212.10539,

    arXiv

  14. [26]

    Autoprompt: Eliciting knowledge from language models with automatically generated prompts

    Taylor Shin, Yasaman Razeghi, Robert L Logan IV , Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980,

    2010 arXiv

  15. [27]

    Challenging big-bench tasks and whether chain-of-thought can solve them

    Mirac Suzgun, Nathan Scales, Nathanael Sch¨arli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261,

    arXiv

  16. [28]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram ´e, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118,

    arXiv

  17. [29]

    Promptagent: Strategic planning with language models enables expert- level prompt optimization

    Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P Xing, and Zhiting Hu. Promptagent: Strategic planning with language models enables expert- level prompt optimization. arXiv preprint arXiv:2310.16427,

    arXiv

  18. [30]

    Large language models as optimizers

    12 Published as a conference paper at ICLR 2025 Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. arXiv preprint arXiv:2309.03409,

    2025 arXiv

  19. [31]

    Prompt engineering a prompt engineer

    Qinyuan Ye, Maxamed Axmed, Reid Pryzant, and Fereshte Khani. Prompt engineering a prompt engineer. arXiv preprint arXiv:2311.05661,

    arXiv

  20. [32]

    Textgrad: Automatic” differentiation” via text

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. Textgrad: Automatic” differentiation” via text. arXiv preprint arXiv:2406.07496 ,

    arXiv

  21. [33]

    Revisiting opro: The limitations of small-scale llms as optimizers

    Tuo Zhang, Jinyue Yuan, and Salman Avestimehr. Revisiting opro: The limitations of small-scale llms as optimizers. arXiv preprint arXiv:2405.10276,

    arXiv

  22. [34]

    Large language models are human-level prompt engineers

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910,

    arXiv

  23. [35]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043,

    arXiv

  24. [36]

    A.1 D ATASETS As discussed in Section 5.1, we use GSM8K (Cobbe et al., 2021), BBH (Suzgun et al., 2022), and FOLIO (Han et al.,

    A B ENCHMARK DATASETS , MODELS , AND BASELINES In this section, we discuss the details of Datasets, Models, and Baselines. A.1 D ATASETS As discussed in Section 5.1, we use GSM8K (Cobbe et al., 2021), BBH (Suzgun et al., 2022), and FOLIO (Han et al.,

    2021

  25. [37]

    This covers almost all types of tasks in BBH dataset. We skip word sorting and dyck languages tasks from our evaluation since we found that the smaller LLM outputs are very difficult to reliably evaluate due to highly inconsistent output pattern. Finally, for logical deduction...

    2024

  26. [38]

    We use the natural language reasoning task with premises text, conclusion text to infer the labels

    for our evaluation. We use the natural language reasoning task with premises text, conclusion text to infer the labels. The original validation split (203 rows) are used for the evaluation of FOLIO, whereas 50/100 samples are taken for train and dev set respectively out of the...

    2023

  27. [39]

    And we used TextGrad’s own library for evaluating their results

    for benchmarking. And we used TextGrad’s own library for evaluating their results. Other than that, we also compared against original Zero-Shot CoT (Wei et al., 2022), and the larger model optimized reported prompts from APE, APO, PE2, OPRO (Yang et al., 2023), EvoPrompt (Guo ...

    2020

  28. [40]

    T EXT-BASED FEEDBACK APPROACHES Assume each sample contains L input-output tokens on average

    42 35 45 73 GREATER 56 70 57 86 D C OMPLEXITY COMPARISON : GR EATER VS . T EXT-BASED FEEDBACK APPROACHES Assume each sample contains L input-output tokens on average. FOR GREATER • Operations Per Sample: – Forward Pass: O(L2) – Backward Pass: O(L2) • Total Complexity: – For N ...

    2025

  29. [41]

    Use proper logical reasoning and think step by step. Finally, give the actual correct answer

    required a total of ∼14 hours for prompt optimization in our own setup. E P ROMPT OPTIMIZATION VS . F EW-S HOT IN-CONTEXT LEARNING Figure 4: Efficacy of GREATER in zero-shot setting compared to five-shot inference with Llama-3-8B-Instruct. In-context learning has proven to be ...

    2025

  30. [2020]

    Black-box prompt optimization: Aligning large language models without model training

    Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. Black-box prompt optimization: Aligning large language models without model training. arXiv preprint arXiv:2311.04155,

    arXiv

  31. [2021]

    Rlprompt: Optimizing discrete text prompts with reinforcement learning

    Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. Rlprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205.12548,

    arXiv

  32. [2022]

    Making pre-trained language models better few-shot learners

    Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723,

    2012 arXiv

  33. [2023]

    Detecting language model attacks with perplexity

    Gabriel Alon and Michael Kamfonas. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308.14132,

    arXiv

  34. [2024]

    Folio: Natural language reasoning with first-order logic

    Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, et al. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv:2209.00840,

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.