Pith. sign in

REVIEW 3 major objections 4 minor 43 references

Prompt optimization methods are complementary, and an error-driven method called ERGO wins specifically on classification tasks where errors concentrate in a few confused label pairs, by turning mistakes into explicit decision rules.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:03 UTC pith:EENI3XX3

load-bearing objection Solid empirical map of prompt-optimization paradigms; the 'boundary-learnable' label is post hoc, but the core finding that no method dominates holds. the 3 major comments →

arxiv 2607.20497 v1 pith:EENI3XX3 submitted 2026-06-15 cs.AI

From Errors to Rules: Iterative Prompt Optimization for Text Classification

Yueying Cui , Renhao Xue , Yi Zhang , Mukul Prasad This is my paper
classification cs.AI
keywords prompt optimizationtext classificationerror-driven learningin-context learningdecision ruleslarge language modelsfew-shot classificationparadigm comparison
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the three main approaches to prompt optimization for text classification—selecting demonstrations, searching over prompt variants, and diagnosing errors—are complementary, not competing: no single method outperforms the others across eight benchmarks. It introduces ERGO, which iterates over the training set in non-overlapping batches, asks the language model to diagnose why examples are misclassified, and rewrites the instruction, demonstrations, and decision rules in one pass. On tasks where errors cluster in specific confused label pairs ('boundary-learnable' tasks), ERGO reaches 90.0% on TREC and 94.4% on CLINC150, converging in 3-5 iterations and producing human-readable rules. The practical payoff is a selection framework: start with free demonstration-based in-context learning, use ERGO when boundaries are learnable from error patterns, and use exploration-based search for many-class tasks.

Core claim

On the paper's own terms, the central discovery is that error-driven refinement—diagnosing a small batch of mistakes, prescribing rules, and rewriting the full prompt—captures task conventions that neither demonstration selection nor accuracy-based search can, and that this makes ERGO the best method on boundary-learnable tasks while remaining statistically tied overall. The paper does not claim universal dominance; it claims a complementary niche. Evidence includes case studies where a single misclassified query triggers a rule that fixes dozens of test examples, and transfer experiments where prompts refined by a stronger model serve a weaker model with no measurable loss.

What carries the argument

ERGO's diagnose→prescribe→rewrite meta-prompt: a loop that classifies a non-overlapping 20-example batch, partitions the outputs into mistakes and correct predictions, asks the LLM to identify confused label pairs and explain why, then jointly rewrites the instruction, demonstrations, and decision guidelines. The work it does: single errors become generalizable rules (e.g., one misclassified company query yields 'organizations → human beings,' fixing many test queries), and the best-on-validation prompt is selected across iterations.

Load-bearing premise

ERGO assumes that the mistakes in a random 20-example batch reflect the task's true confusions, and that the rules written from those few errors generalize to the whole test distribution; on many-class tasks a 20-item batch covers only 10-15% of classes per iteration, so this is exactly where ERGO is weakest.

What would settle it

Run ERGO on a many-class task where errors are spread thinly across many label pairs rather than concentrated: if it fails to beat a zero-cost demonstration baseline, then the boundary-learnable condition is doing the work, not the error-driven mechanism. More directly, take a task with known confusable pairs, hold those pairs out of the training batches, and check whether the learned rules still fix test errors; if they do not, the single-error-to-rule generalization fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Practitioners can choose the cheapest adequate method: a zero-cost demonstration baseline first, ERGO for tasks with concentrated confusions, and exploration-based search for many-class tasks with overlapping categories.
  • Rules learned by a strong model transfer to a weaker model with no measurable accuracy loss, so one optimization run can be reused for cheaper inference.
  • ERGO's interpretable rules double as annotation guidelines or audit trails, unlike black-box searched prompts.
  • On boundary-learnable tasks, prompt optimization is not a coin flip: error-driven feedback converges in 3-5 iterations and is more stable across seeds than search methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 'boundary-learnable' property could be measured before optimization, e.g., by the concentration of a confusion matrix on a seed prompt, turning the retrospective label into a testable predictor of when ERGO will win.
  • Combining ERGO's explicit rules with demonstration-based in-context learning might capture both precise boundaries and broad coverage; the paper's ablation hints that demonstrations carry much of the signal, so a hybrid is a natural next step.
  • The same diagnose-prescribe-rewrite loop may apply beyond classification to structured outputs such as slot filling or relation extraction, where label pairs become semantic confusion classes.
  • Because ERGO's rules are plain text, they could be distilled into a small model or used to synthesize training data for fine-tuning, extending the transfer result beyond prompt reuse.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript reports an empirical comparison of three prompt-optimization paradigms for few-shot text classification—demonstration selection (ICL-Uniform/ICL-Diversity), exploration-based search (APE, DSPy, GEPA), and the authors' newly proposed error-driven diagnosis method ERGO. ERGO iterates over non-overlapping 20-example training batches, diagnoses confused label pairs from classification errors, and rewrites the full prompt (instruction, demonstrations, guidelines) via a diagnose-prescribe-rewrite meta-prompt, selecting the best prompt on a validation set. Across 8 benchmarks (2–150 classes), no method dominates: ICL wins on coverage-dependent tasks, DSPy/GEPA on many-class tasks, and ERGO on tasks the authors call 'boundary-learnable' (TREC, CLINC150, RTE, Ethos), with best accuracy on TREC (90.0%) and CLINC150 (94.4%), convergence in 3–5 iterations, interpretable rules, and transfer across model families. The paper also proposes a paradigm-selection framework based on task characteristics.

Significance. If the results hold, the paper makes a useful empirical contribution: a nuanced map of when iterative prompt optimization pays off, concrete evidence that error-driven rule refinement can discover non-obvious labeling conventions, and a practical recommendation to start from zero-cost ICL. Strengths include the honest statement that ERGO is not universally better, the inclusion of ablations, cross-model and cross-model-transfer experiments, seed variation, and detailed case studies with sample-level support, plus the appendix-level algorithm and meta-prompt for reproducibility. The 'no single method dominates' claim is well evidenced by Table 1's small average differences. However, the significance of the central practical contribution—the paradigm-selection framework—is currently limited because the key condition for choosing ERGO ('boundary-learnable') is defined post hoc and not measured prospectively; the paper also contains an internal inconsistency in the cross-model sample count and reports individual-win p-values without correction. These are fixable but affect the strength of the headline guidance.

major comments (3)
  1. [§7.3, Table 4] The proposed decision rule — 'if classification errors concentrate in specific confused label pairs, apply ERGO' — cannot be applied without an independent, pre-hoc measure of error concentration. The term 'boundary-learnable' is introduced after observing that ERGO wins on TREC, CLINC150, RTE, and Ethos, and the only operational test offered is to run a classifier and inspect its confusion matrix. This makes the paradigm-selection framework circular in practice: a practitioner must already have a working classifier and know its error pattern before deciding to use ERGO, and no evidence is given that a cheap baseline's error-concentration statistic predicts ERGO's relative gain on held-out tasks. Please (a) define a concrete, computable measure (e.g., top-1 confused-pair share or error entropy from ICL-Diversity predictions on validation data); (b) report its values for the eight benchma
  2. [§4.1–4.2, Table 1] The significance infrastructure is not calibrated for the headline comparisons. The paper says 'paired t-test at 5% level across all 40 configurations' but then reports individual p-values for dataset-level wins (e.g., CLINC150 p=0.043, Yahoo p=0.046, 20Newsgroups p=0.013) without any multiple-comparison correction. With 8 datasets and multiple method pairs, these marginal p-values would not survive a Bonferroni or FDR control; the CLINC150 win in particular is very weak evidence as reported. Similarly, 'all pairwise p>0.6' for non-dominance is presented as if it established equivalence, but this ignores the multiple-comparison problem and the fact that absence of significant difference is not evidence of equality. Please report corrected p-values, confidence intervals, or explicitly demote the dataset-level win claims to descriptive observations. The qualitative complementarity story ca
  3. [§5.2, Tables 2 and 10] There is a quantitative inconsistency in the homogeneous-transfer experiment. The text says 'five models from four families' and 'Pooled across all 200 (model, dataset, seed) triples,' but Table 2 and Table 10 list only four models (Claude Haiku 4.5, Claude Sonnet 4.5, Gemma 3 27B, Mistral Large 3). Four models × 8 datasets × 5 seeds = 160 triples, not 200. This discrepancy directly affects the reported pooled p-value (p=1.5×10−9) and the cross-model generalization claim. Please reconcile: either add the missing model and its table rows, or correct the text and recompute the pooled statistics. As written, the cross-model claim rests on an internally inconsistent N.
minor comments (4)
  1. [Algorithm 1 vs §5.3] Algorithm 1 sets T = ceil(|D_train|/k), while §5.3 states the default is T=10. For a 150-example training set with k=20 this would give T=8, not 10. Align the pseudocode with the actual implementation or explain why T is fixed at 10 in the experiments.
  2. [Table 5] CLINC150 is listed with 150 training examples and 150 classes, implying exactly one training example per class. This makes the batch-coverage concern in the Limitations section even more severe and should be stated in the table caption.
  3. [Table 2] The p-value annotation uses stars only. Since the table reports one non-significant model (Gemma 3 27B) and several significance levels, include the exact p-values in the table or a footnote.
  4. [§3.2 / Figure 1] Figure 1 is referenced in the overview but does not appear in the submitted text. Ensure the figure is included in the camera-ready version.

Circularity Check

0 steps flagged

No significant circularity: ERGO's gains are established on held-out data; the 'boundary-learnable' label is a post hoc descriptive term, not a construction that makes the empirical claims true.

full rationale

The paper does not contain a derivation in which a predicted quantity is defined in terms of the fitted quantity. ERGO's pipeline (Algorithm 1) uses only training-batch errors to rewrite prompts, selects the best prompt by full validation accuracy, and reports test accuracy on held-out splits; the main comparisons (Tables 1–3) are external baseline numbers, not quantities reconstructed from the method's own inputs. The 'boundary-learnable' taxonomy is offered after observing the benchmark results (e.g., 'These are tasks where classification errors concentrate in specific confused pairs amenable to natural-language rules') and functions as an empirical generalization from case studies, not as a definition that forces ERGO to win. The Section 7.3 selection rule ('if classification errors concentrate in specific confused label pairs, apply ERGO') is in principle checkable from a baseline confusion analysis; the lack of a prospective concentration measure is a predictive-validity limitation, not a circular reduction. There is no load-bearing self-citation: the method is compared against independent published baselines, and the paper's own limitations admit tasks where ERGO loses (MASSIVE, 20Newsgroups), which is inconsistent with a circularly constrained framework. Hence no enumerated circularity step is warranted.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

The paper makes no mathematical derivation, so the ledger is dominated by empirical assumptions and design choices. The main free parameters are ERGO's batch size and iteration budget, plus the 20-demonstration setup for ICL baselines. The load-bearing assumptions are that small-batch errors are representative and that LLM-generated rules encode true conventions. One invented conceptual entity, the 'boundary-learnable' task type, carries most of the framework's explanatory weight but has no independent evidence.

free parameters (4)
  • ERGO batch size k = 20 (default; k=50 gives +1.29 pp)
    Central method's contrast batch size; the ablation shows k=50 significantly outperforms k=5, so k=20 is a cost-driven choice rather than an optimal value.
  • ERGO iteration budget T = 10 (default; T=5 reaches 99% of T=10 accuracy)
    Arbitrary budget cap; ablation shows robustness, but all convergence claims depend on this maximum.
  • ICL demonstration count = 20
    Used for ICL-Uniform and ICL-Diversity baselines; not tuned and no sensitivity analysis is reported.
  • Seed set = {42, 123, 7, 1, 2}
    Five seeds underpin all statistical significance claims; the small sample is a free choice that affects p-values.
axioms (5)
  • domain assumption Misclassification patterns in 20-item non-overlapping batches are representative of test-time confusions.
    ERGO's diagnose-prescribe-rewrite loop updates prompts from each batch; the paper's limitations admit that on CLINC150 and MASSIVE such batches cover only 10-15% of classes per iteration.
  • domain assumption LLM-generated diagnoses and rules reflect true task conventions rather than annotation noise or spurious correlations.
    Section 5.5 checks unanimous errors against annotation noise, but does not audit whether each generated rule is correct on unseen cases.
  • domain assumption Validation-set accuracy is a reliable selection signal for best-of-T prompt choice.
    Algorithm 1 selects the best prompt by validation accuracy; the paper criticizes DSPy's minibatch overfitting but does not quantify overfitting of ERGO's own val-based selection.
  • standard math Paired t-tests without multiple-comparison correction are an acceptable significance standard.
    Section 4.1 says paired t-test at 5% across 40 configurations; several headline p-values are 0.043-0.046, so multiplicity is a real risk.
  • ad hoc to paper The three-paradigm taxonomy is exhaustive and the task categories are measurable a priori.
    The paradigm-selection guide (Table 4) is derived from post-hoc categorization; no operational metric for 'boundary-learnable' is given, so the framework is not yet a predictive instrument.
invented entities (1)
  • boundary-learnable task category no independent evidence
    purpose: Labels tasks where ERGO should be used (TREC, Ethos, CLINC150, RTE) in the paradigm-selection framework.
    The term is defined by error concentration but operationalized post hoc on tasks where ERGO wins; no held-out task or independent error-concentration threshold validates the category.

pith-pipeline@v1.3.0-alltime-deepseek · 14302 in / 15869 out tokens · 148588 ms · 2026-08-02T11:03:09.832404+00:00 · methodology

0 comments
read the original abstract

Prompt optimization for text classification spans diverse approaches, from demonstration selection to exploration-based search to error-driven diagnosis, each with known but incompletely characterized strengths and limitations. We conduct a comprehensive empirical study across diverse classification benchmarks (2 to 150 classes) comparing these paradigms through both quantitative evaluation and qualitative analysis of optimization traces, revealing that each paradigm excels on structurally different task types and that no single method dominates. Guided by these insights, we propose Error-Guided Optimization (ERGO), an error-driven method that iterates over the full training set in non-overlapping batches, diagnoses classification failures, and generates targeted decision rules through a diagnose-prescribe-rewrite feedback loop. ERGO achieves the best accuracy on tasks where errors concentrate in specific confused label pairs (which we term boundary-learnable tasks): TREC: 90.0%, CLINC150: 94.4%, converges in 3-5 iterations, and produces interpretable decision rules. While ERGO does not achieve the highest overall average, it fills a complementary role: demonstration-based ICL wins on coverage-dependent tasks, exploration-based search wins on many-class intent, and ERGO wins where decision boundaries are learnable from error patterns. We provide a complementarity framework linking task characteristics to optimal paradigm selection, offering practical guidance for practitioners.

Figures

Figures reproduced from arXiv: 2607.20497 by Mukul Prasad, Renhao Xue, Yi Zhang, Yueying Cui.

Figure 1
Figure 1. Figure 1: ERGO overview. Inputs: a seed prompt π0, training set Dtrain, validation set Dval, and iteration budget T. Training data is shuffled once into non-overlapping batches of size k. Loop (steps 1–4): each iteration selects the next batch, classifies it with the current prompt, partitions predictions into mistakes Et and correct Ct, then applies the diagnose→prescribe→rewrite meta-prompt to produce an updated p… view at source ↗
Figure 2
Figure 2. Figure 2: ERGO convergence on boundary-learnable tasks. Dashed lines: ICL-Div, GEPA, and DSPy baselines. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 2 canonical work pages

  1. [1]

    2022 , publisher =

    Mollas, Ioannis and Chrysopoulou, Zoe and Karlos, Stamatis and Tsoumakas, Grigorios , journal =. 2022 , publisher =

    2022

  2. [2]

    Proceedings of the 2018 EMNLP workshop BlackboxNLP: Analyzing and interpreting neural networks for NLP , pages=

    GLUE: A multi-task benchmark and analysis platform for natural language understanding , author=. Proceedings of the 2018 EMNLP workshop BlackboxNLP: Analyzing and interpreting neural networks for NLP , pages=

    2018

  3. [3]

    International Conference on Learning Representations , year =

    Generate Rather Than Retrieve: Large Language Models Are Strong Context Generators , author =. International Conference on Learning Representations , year =

  4. [4]

    Sentence-bert: Sentence embeddings using siamese bert-networks , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=

    2019

  5. [5]

    What Makes Good In-Context Examples for

    Liu, Jiachang and Shen, Dinghan and Zhang, Yizhe and Dolan, Bill and Carin, Lawrence and Chen, Weizhu , booktitle =. What Makes Good In-Context Examples for. 2022 , address =. doi:10.18653/v1/2022.deelio-1.10 , url =

    doi:10.18653/v1/2022.deelio-1.10 2022

  6. [6]

    2019 , address =

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , booktitle =. 2019 , address =. doi:10.18653/v1/N19-1423 , url =

    doi:10.18653/v1/n19-1423 2019

  7. [7]

    Advances in Neural Information Processing Systems , volume =

    Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  8. [8]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages =

    Active Learning Principles for In-Context Learning with Large Language Models , author =. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages =. 2023 , address =. doi:10.18653/v1/2023.findings-emnlp.334 , url =

    doi:10.18653/v1/2023.findings-emnlp.334 2023

  9. [9]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    A Survey on In-Context Learning , author =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. 2024 , address =. doi:10.18653/v1/2024.emnlp-main.64 , url =

    doi:10.18653/v1/2024.emnlp-main.64 2024

  10. [10]

    2023 , eprint =

    Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection , author =. 2023 , eprint =

    2023

  11. [11]

    2024 , eprint =

    A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications , author =. 2024 , eprint =

    2024

  12. [12]

    Advances in Neural Information Processing Systems , volume =

    Language Models are Few-Shot Learners , author =. Advances in Neural Information Processing Systems , volume =. 2020 , publisher =

    2020

  13. [13]

    Intent Detection in the Age of

    Arora, Gaurav and Jain, Shreya and Merugu, Srujana , booktitle =. Intent Detection in the Age of. 2024 , address =. doi:10.18653/v1/2024.emnlp-industry.114 , url =

    doi:10.18653/v1/2024.emnlp-industry.114 2024

  14. [14]

    Applied Sciences , volume =

    Comparing Fine-Tuning and Prompt Engineering for Multi-Class Classification in Hospitality Review Analysis , author =. Applied Sciences , volume =. 2024 , publisher =. doi:10.3390/app14146254 , url =

    work page doi:10.3390/app14146254 2024

  15. [15]

    International Conference on Learning Representations , year =

    Large Language Models Are Human-Level Prompt Engineers , author =. International Conference on Learning Representations , year =

  16. [16]

    2025 , address =

    Agarwal, Eshaan and Magazine, Raghav and Singh, Joykirat and Dani, Vivek and Ganu, Tanuja and Nambi, Akshay , booktitle =. 2025 , address =

    2025

  17. [17]

    International Conference on Learning Representations , year =

    Large Language Models as Optimizers , author =. International Conference on Learning Representations , year =

  18. [18]

    Advances in Neural Information Processing Systems , volume =

    Self-Refine: Iterative Refinement with Self-Feedback , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  19. [19]

    2025 , eprint =

    A Survey of Automatic Prompt Engineering: An Optimization Perspective , author =. 2025 , eprint =. doi:10.48550/arXiv.2502.11560 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.11560 2025

  20. [20]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =

    A Systematic Survey of Automatic Prompt Optimization Techniques , author =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =. 2025 , address =. doi:10.18653/v1/2025.emnlp-main.1681 , url =

    doi:10.18653/v1/2025.emnlp-main.1681 2025

  21. [21]

    Advances in Neural Information Processing Systems , volume =

    Many-Shot In-Context Learning , author =. Advances in Neural Information Processing Systems , volume =. 2024 , url =

    2024

  22. [22]

    More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for

    Yao, Bingsheng and Chen, Guiming and Zou, Ruishi and Lu, Yuxuan and Li, Jiachen and Zhang, Shao and Sang, Yisi and Liu, Sijia and Hendler, James and Wang, Dakuo , booktitle =. More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for. 2024 , address =. doi:10.18653/v1/2024.findings-naacl.115 , url =

    doi:10.18653/v1/2024.findings-naacl.115 2024

  23. [23]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages =

    Self-Augmented In-Context Learning for Unsupervised Word Translation , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages =. 2024 , address =. doi:10.18653/v1/2024.acl-short.67 , url =

    work page doi:10.18653/v1/2024.acl-short.67 2024

  24. [24]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages =

    What Makes a Good Order of Examples in In-Context Learning , author =. Findings of the Association for Computational Linguistics: ACL 2024 , pages =. 2024 , address =. doi:10.18653/v1/2024.findings-acl.884 , url =

    doi:10.18653/v1/2024.findings-acl.884 2024

  25. [25]

    Misconfidence-Based Demonstration Selection for

    Xu, Shangqing and Zhang, Chao , year =. Misconfidence-Based Demonstration Selection for. 2401.06301 , archivePrefix =

    Pith/arXiv arXiv

  26. [26]

    2024 , eprint =

    In-Context Learning Demonstration Selection via Influence Analysis , author =. 2024 , eprint =

    2024

  27. [27]

    2023 , eprint =

    Dynamic Demonstrations Controller for In-Context Learning , author =. 2023 , eprint =

    2023

  28. [28]

    2022 , address =

    Deng, Mingkai and Wang, Jianyu and Hsieh, Cheng-Ping and Wang, Yihan and Guo, Han and Shu, Tianmin and Song, Meng and Xing, Eric and Hu, Zhiting , booktitle =. 2022 , address =. doi:10.18653/v1/2022.emnlp-main.222 , url =

    doi:10.18653/v1/2022.emnlp-main.222 2022

  29. [29]

    2024 , eprint =

    Intent-Based Prompt Calibration: Enhancing Prompt Optimization with Synthetic Boundary Cases , author =. 2024 , eprint =

    2024

  30. [30]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Instruction Induction: From Few Examples to Natural Language Task Descriptions , author =. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2023 , address =. doi:10.18653/v1/2023.acl-long.108 , url =

    doi:10.18653/v1/2023.acl-long.108 2023

  31. [31]

    2024 , eprint =

    The Prompt Report: A Systematic Survey of Prompt Engineering Techniques , author =. 2024 , eprint =. doi:10.48550/arXiv.2406.06608 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.06608 2024

  32. [32]

    and Moazam, Hanna and Miller, Heather and Zaharia, Matei and Potts, Christopher , booktitle =

    Khattab, Omar and Singhvi, Arnav and Maheshwari, Paridhi and Zhang, Zhiyuan and Santhanam, Keshav and Vardhamanan, Sri and Haq, Saiful and Sharma, Ashutosh and Joshi, Thomas T. and Moazam, Hanna and Miller, Heather and Zaharia, Matei and Potts, Christopher , booktitle =. 2024 , url =

    2024

  33. [33]

    A Comparative Study of

    Sarmah, Bhaskarjit and Dutta, Kriti and Grigoryan, Anna and Tiwari, Sachin and Pasquali, Stefano and Mehta, Dhagash , year =. A Comparative Study of. doi:10.48550/arXiv.2412.15298 , url =. 2412.15298 , archivePrefix =

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.15298

  34. [34]

    and Tan, Shangyin and Soylu, Dilara and Ziems, Noah and Khare, Rishi and Opsahl-Ong, Krista and Singhvi, Arnav and Shandilya, Herumb and Ryan, Michael J

    Agrawal, Lakshya A. and Tan, Shangyin and Soylu, Dilara and Ziems, Noah and Khare, Rishi and Opsahl-Ong, Krista and Singhvi, Arnav and Shandilya, Herumb and Ryan, Michael J. and Jiang, Meng and Potts, Christopher and Sen, Koushik and Dimakis, Alex and Stoica, Ion and Klein, Dan and Zaharia, Matei and Khattab, Omar , booktitle =. 2026 , url =

    2026

  35. [35]

    gradient descent

    Automatic prompt optimization with “gradient descent” and beam search , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    2023

  36. [36]

    2002 , url =

    Learning Question Classifiers , author =. 2002 , url =

    2002

  37. [37]

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , pages =

    An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction , author =. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , pages =. 2019 , address =. doi:10.18653/v1/D19-1131 , url =

    doi:10.18653/v1/d19-1131 2019

  38. [38]

    2023 , address =

    FitzGerald, Jack and Hench, Christopher and Peris, Charith and Mackie, Scott and Rottmann, Kay and Sanchez, Ana and Nash, Aaron and Urbach, Liam and Kakarala, Vishesh and Singh, Richa and Ranganath, Swetha and Crist, Laurie and Britan, Misha and Leeuwis, Wouter and Tur, Gokhan and Natarajan, Prem , booktitle =. 2023 , address =. doi:10.18653/v1/2023.acl-l...

    doi:10.18653/v1/2023.acl-long.235 2023

  39. [39]

    1995 , publisher =

    Lang, Ken , booktitle =. 1995 , publisher =. doi:10.1016/B978-1-55860-377-6.50048-7 , url =

    doi:10.1016/b978-1-55860-377-6.50048-7 1995

  40. [40]

    Prompt Optimization Is a Coin Flip: Diagnosing When It Helps in Compound

    Zhang, Xing and Wang, Guanghui and Cui, Yanwei and Qiu, Wei and Li, Ziyuan and Zhu, Bing and He, Peiyang , year =. Prompt Optimization Is a Coin Flip: Diagnosing When It Helps in Compound. doi:10.48550/arXiv.2604.14585 , url =. 2604.14585 , archivePrefix =

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.14585

  41. [41]

    International Conference on Learning Representations , year =

    Selective Annotation Makes Language Models Better Few-Shot Learners , author =. International Conference on Learning Representations , year =

  42. [42]

    2020 , url =

    Stefano Leone , title =. 2020 , url =

    2020

  43. [43]

    Character-level Convolutional Networks for Text Classification , url =

    Zhang, Xiang and Zhao, Junbo and LeCun, Yann , booktitle =. Character-level Convolutional Networks for Text Classification , url =