Pith. sign in

REVIEW 3 major objections 5 minor 52 references

iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces iPrOp, an interactive prompt optimization loop with a human in the loop, and claims it improves prompt performance by generating and selecting among paraphrased prompts over repeated iterations.

desk verdict A tidy framework paper whose headline result is a simulation of the interactive loop, not a test of it. read the letter →

arxiv 2412.12644 v2 pith:76ZQVCYK submitted 2024-12-17 cs.CL

Jiahui Li , Roman Klinger This is my paper · ORCID
classification cs.CL
keywords interactivepromptoptimizationhuman-in-the-loopengineeringlargelanguagemodelsemotionclassificationparaphrasingexplainabilityF1evaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces iPrOp, an interactive prompt optimization workflow that places a human in the loop between automatic prompt rephrasing and performance evaluation. The goal is to make prompt engineering usable by non-experts while letting the user weigh competing criteria: classification performance, prompt readability, explanation quality, and alignment with user expectations. The central claim is that this loop yields better prompts: in a simulation over 15 iterations on three emotion classification datasets, F1 scores on both training and validation data rise. If correct, iPrOp provides a practical middle ground between fully manual prompt crafting and fully automated prompt search, and a platform for studying how example selection, explanation strategy, and model choice influence prompt optimization.

What carries the argument

The central mechanism is the iPrOp loop, formalized as $p^* = \arg\max_{p \in P \cup M(P)} H(I(p_i))$, where $M(P)$ is a prompt paraphrasing model that generates variants, and $I(p_i)$ packages the properties presented to the user: the prompt itself, a sample of informative training instances with the model's labeled predictions and explanations, and an F1 score computed on a held-out subset. The loop is iterative: the user (or, in the simulation, the F1-based selector) chooses the preferred prompt, which becomes the seed for the next round of paraphrasing. The workflow is designed so that each step can be performed by the user or automatically, and it is this interchangeability that lets iPrOp serve both interactive and simulated modes.

What would settle it

Run the same 15-iteration optimization on the three datasets using a train/validation/test split and report F1 on the held-out test fold; if the final prompt does not outperform the initial seed prompt on that test fold, the central claim of enhanced task performance fails.

Watch

Extended reading notes

Core claim

The paper proposes iPrOp and claims that its iterative workflow—starting from an initial seed prompt, repeatedly generating paraphrased prompt variants, selecting the preferred prompt on the basis of informative examples, model-produced explanations, and performance metrics—can generate improved prompts that lead to enhanced task performance. In the reported experiment, the selection step is simulated: a program picks the prompt with the highest weighted F1 on a fixed training subset at each iteration, using the Llama 3.1 8B instruct model on the TEC, GROUNDED-EMOTIONS, and TALES-EMOTION datasets. Over 15 iterations, F1 increases on both the training subset and a validation set for all three datasets, which the paper takes as evidence for the approach's effectiveness. The paper also formalizes the selection process, where the user preference function H chooses the prompt maximizing over the presented prompt properties.

Load-bearing premise

The experiments replace the human assessment that the method is designed around with automatic selection by weighted F1 on a training subset, so the claim that the interactive loop improves prompts depends on this score-based proxy standing in for real user choices and on the small validation sets supporting generalization.

Editorial extensions

If this is right

  • Users without prompt-engineering expertise can reach competitive prompts for text classification by following a structured, information-rich selection loop rather than writing prompts from scratch.
  • The loop makes trade-offs explicit: a user can reject a high-F1 prompt in favor of one that is more readable or whose explanations better match their expectations, which is not possible in fully automatic optimization.
  • Because each step can be automated or human-performed, the approach supports partial automation, letting the human intervene only on decisions that most need judgment.
  • The framework provides a testbed for studying how example selection, explanation generation, and LLM choice affect the trajectory of prompt optimization across iterations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported simulation suggests the performance gain may come primarily from the rephrasing-and-score loop itself, not from the human; a controlled comparison between full automatic selection and human-guided selection would reveal how much the human judgment actually adds.
  • If the F1-based selection is a faithful proxy, then the same loop should transfer to other classification tasks and even to non-classification tasks by replacing F1 with a suitable metric, a testable extension the paper does not run.
  • The limitation that some datasets are insensitive to rephrasing implies that an upfront diagnostic—e.g., measuring how much paraphrases change the model's predictions—could predict which tasks will benefit from interactive optimization.
  • A user study comparing iPrOp against both manual prompt engineering and fully automatic optimization would settle whether the interactive interface actually reduces effort while improving or matching end performance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces iPrOp, an interactive prompt optimization framework for LLM-based text classification, formalized as an iterative loop in which a user assesses candidate prompts on the basis of performance metrics, predictions with explanations, informative instances, and readability, and selects the preferred prompt for further paraphrasing. The authors describe a prototype web interface and report a simulation experiment on three emotion classification datasets (TEC, GROUNDED-EMOTIONS, TALES-E-MOTION) using llama3.1:8b-instruct-fp16, in which the user assessment is replaced by automatic selection according to the weighted F1 score on a fixed training subset. The paper claims that the approach can generate improved prompts and lead to enhanced task performance, based on an observed increasing trend in training and validation F1 over 15 iterations.

Significance. If the central claim were fully supported, iPrOp would offer a useful bridge between manual prompt engineering and automatic prompt optimization, with a structured interface for non-technical users and a modular workflow that could support systematic studies of prompt properties. Strengths include the clear formalization of the user assessment function, the publicly available prototype web interface and explanation video, and a candid Limitations section that acknowledges several unresolved challenges. However, the significance is currently limited by the fact that the reported experiment does not actually test the human-in-the-loop component: it substitutes automatic F1-based selection for human judgment, and it provides no baselines, error bars, or statistical tests. The result is a plausible pilot for an automatic optimization loop, but it does not yet substantiate the interactive contribution claimed in the title and abstract.

major comments (3)
  1. [Section 4 (Evaluation) and Section 3 (Methods)] The central empirical claim—that iPrOp benefits from a human in the loop—is not evaluated. The formal definition of user assessment H in Eq. (1) includes prompt performance, readability, explanations, and alignment with user expectations, but the experiment replaces H with automatic selection by weighted F1 on a fixed training subset, stating that "the prompt is selected corresponding to the weighted F1 score over a fixed subset of the training data" and that "we only consider automated classification performance scores." A user study is deferred to future work. Consequently, the reported results validate a fully automatic F1-driven prompt search, not the interactive human-in-the-loop process that the title and abstract claim. To support the central claim, the authors need either a user study or a carefully validated simulation of human preferences (e.g., a model trained on actual human judgments), and the abstract and title should be tempered to match the actual evaluation.
  2. [Section 4, Figure 4] The evidence for the effectiveness of the optimization loop is statistically weak. The F1 curves over 15 iterations are presented without error bars, multiple random seeds, or significance tests, and no baselines are provided (e.g., random prompt paraphrasing, best-of-N selection without iteration, or a fixed initial prompt). Without such controls, the observed upward trend cannot be attributed to the iterative refinements or to the information presented to the simulated user; the trend could in principle arise from the paraphrase model's drift or from variance. The paper should include at least one baseline and repeat the experiment with different seeds and train/validation splits, reporting variance, in order to substantiate the claim that the approach "has the capability to generate improved prompts."
  3. [Section 5 (Limitations)] The paper's own limitations weaken the generality of the empirical claim. It states that "certain datasets exhibit limited sensitivity to divergent prompts, allowing a simple or even naive initial prompt to achieve superior performance." This is directly relevant to the three emotion datasets evaluated: if one or more of them is insensitive to prompt variation, the rising F1 curve may reflect noise or incidental improvements rather than the method's capability. The manuscript should report per-dataset initial vs. final performance and explicitly discuss which datasets support the claimed improvement, rather than aggregating the trend across datasets.
minor comments (5)
  1. [Section 3, Eq. (1)] There is a notation inconsistency: the argmax ranges over p ∈ P ∪ M(P), but I(pi) uses an index i that is not introduced; it should be I(p) for consistency. Also, the definition of M(P) as "a prompt paraphrasing model that varies the prompts P selected from the previous iteration" could be made more precise by distinguishing the set of candidate prompts P from the set of paraphrased variants M(P).
  2. [Figure 4] The dual y-axes and color-only legend make the figure difficult to read, especially in grayscale; consider plotting each dataset in a separate panel with a shared scale, or using distinct line styles in addition to color.
  3. [Section 2.3] The sentence "Wu et al.'s (2024) analysis of strategies to enhance the transparency of LLMs." is a sentence fragment; it should be revised into a complete sentence, for example by adding a main verb and the finding or contribution.
  4. [Abstract] The phrase "but also enable to study the intrinsic parameters" is ungrammatical; it should be "but also enables the study of the intrinsic parameters."
  5. [Section 4] The dataset name "TALES-E-MOTION" appears inconsistently as "TALES -E MOTION" and "TALES -E MOTION"; please standardize the formatting throughout the text and figure.

Circularity Check

1 steps flagged · score 3.0 of 10

The training-F1 curve is the selection objective itself, making that part of the evidence self-referential; the validation-F1 curve remains a partially independent check.

  1. fitted input called prediction [Section 4 (Evaluation), Result and simulation setup]
    "In this simulation, the prompt is selected corresponding to the weighted F1 score over a fixed subset of the training data. We expect to demonstrate a rising trend during the optimization process to verify the effectiveness of our approach."

    The simulation's selection rule is exactly the weighted F1 over a fixed training subset, and Figure 4 then reports an increasing F1 trend on training data as evidence of effectiveness. Since the selected prompt is chosen to maximize that same training-F1 objective, an improving (or at least high) training curve is forced by the selection procedure rather than being an independent measurement of prompt quality. The validation-F1 trace is not used for selection and therefore provides a partially independent check, so the circularity is confined to the training curve and does not by itself establish the validation result.

full rationale

The paper's formal method (Section 3) defines prompt choice via a human assessment H over a presentation I(pi) that includes F1, explanations, readability, and alignment. The evaluation (Section 4) replaces H entirely with weighted F1 on a fixed training subset and states that a user study is future work. The central circular step is that the resulting training-F1 curve in Figure 4 is the optimization objective itself: reporting it as part of the 'overall increasing trend' that verifies effectiveness presents the selection criterion as its own evidence. The validation-F1 curve is not selected on and is a partly independent signal, which keeps the paper from being wholly circular. No load-bearing self-citation chain is present: the authors' self-citations concern datasets and related work, not the main result. Other weaknesses, such as the absence of error bars, multiple seeds, statistical tests, and baselines like random prompt selection, are robustness and external-validity concerns rather than circularity. The score of 3 reflects one by-construction training-curve result while the central validation claim retains independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on several unstated domain assumptions: that F1 on a training subset proxies human preference, that LLM explanations are useful, that rephrasing preserves task intent, and that the small validation sets support generalization. No new physical entities are introduced; one conceptual module, a simulated human behavior predictor, is sketched but not implemented.

free parameters (3)
  • Number of optimization iterations = 15
    The loop was run for 15 iterations; no convergence criterion or justification is given, and the final F1 depends on this stopping point (Section 4, Figure 4).
  • Training subset T_beta for F1 evaluation = unspecified
    The fixed subset used to compute the weighted F1 that selects prompts is not sized or described; the choice affects which prompt is picked each iteration (Section 3).
  • Informative instance subset T_alpha = unspecified
    The subset shown to users is sampled according to a strategy alpha that is not defined; the simulation does not use this module, but the interactive framework depends on it for user decisions (Section 3).
assumptions (4)
  • ad hoc to paper F1 on a fixed training subset is a valid proxy for human prompt preference in the validation simulation.
    Section 4 replaces the human assessment H with weighted F1 selection to measure the framework's effectiveness; if users would choose differently when shown explanations and readability, the experiment does not validate the interactive approach.
  • domain assumption LLM-generated explanations of predictions are informative and accurate enough to support user decisions.
    Section 3 generates explanations by prompting 'In your answer, provide only the label you choose and the explanation of your choice'; the quality of these explanations is neither measured nor compared.
  • domain assumption Paraphrases generated by the LLM preserve the task instruction and span a useful search space.
    The prompt update module paraphrases via an LLM meta-prompt; the Limitations section acknowledges that rephrased prompts may retain low quality across iterations, making this assumption load-bearing.
  • domain assumption Validation F1 on the three datasets generalizes beyond the small evaluation subsets.
    Figure 4 reports validation F1 trends of unknown sample size; no confidence intervals are given, so the rising trend may not be stable.
invented entities (1)
  • Simulated human behavior prediction module
    purpose: In the workflow design, it predicts human assessments from instances, explanations, performance, and structure to reduce user interaction load; it is not implemented or evaluated.
    Figure 2 lists a 'human behavior prediction by instances, explanation, performance, structure' module as a possible simulation of user interactions, but the paper provides no definition, implementation, or evaluation of it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop." pith.science (2026). https://pith.science/paper/76ZQVCYK

@misc{pith2026241212644,
  author       = {Pith},
  title        = {Pith review of: iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/76ZQVCYK}},
  note         = {Machine review of arXiv:2412.12644}
}
abstract

Prompt engineering has made significant contributions to the era of large language models, yet its effectiveness depends on the skills of a prompt author. This paper introduces $\textit{iPrOp}$, a novel interactive prompt optimization approach, to bridge manual prompt engineering and automatic prompt optimization while offering users the flexibility to assess evolving prompts. We aim to provide users with task-specific guidance to enhance human engagement in the optimization process, which is structured through prompt variations, informative instances, predictions generated by large language models along with their corresponding explanations, and relevant performance metrics. This approach empowers users to choose and further refine the prompts based on their individual preferences and needs. It can not only assist non-technical domain experts in generating optimal prompts tailored to their specific tasks or domains, but also enable to study the intrinsic parameters that influence the performance of prompt optimization. The evaluation shows that our approach has the capability to generate improved prompts, leading to enhanced task performance.

Figures

Figures reproduced from arXiv: 2412.12644 by the authors.

Figure 1
Figure 1. Screenshot of the iPrOp Web application, where key components are annotated. on some annotated data, for instance measured by F1 (we focus in this paper on text classification tasks); (b) The readability and interpretability of the prompt; (c) The quality of an explanation of the predictions of the prompt; and (d), the align￾ment of the annotations with the users expectations. We therefore propose an interactive pro… view at source ↗
Figure 2
Figure 2. The conceptual workflow of our iPrOp approach. The general workflow is shown in the middle. The left part shows potential human interaction in the various modules. To limit the amount of user interactions, each module can be supported by a simulated interaction. retrieval (Manning et al., 2008), and chatbot inter￾actions (Hancock et al., 2019). More recent papers draw their attention on collaborative annotation pro￾… view at source ↗
Figure 3
Figure 3. User interface prototype for an emotion analy [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 16 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Anum Afzal, Alexander Kowsik, Rajna Fani, and Florian Matthes. 2024. https://doi.org/10.18653/v1/2024.dash-1.2 Towards optimizing and evaluating a retrieval augmented QA chatbot using LLM s with human-in-the-loop . In Proceedings of the Fifth Workshop on Data Science with Human-in-the-Loop (DaSH 2024), pages 4--16, Mexico City, Mexico. Association for Com...

    work page doi:10.18653/v1/2024.dash-1.2 2024

  4. [4]

    Cecilia Ovesdotter Alm and Richard Sproat. 2005. https://doi.org/10.1007/11573548\_86 Emotional sequencing and development in fairy tales . In Affective Computing and Intelligent Interaction, First International Conference, ACII 2005, Beijing, China, October 22-24, 2005, Proceedings , volume 3784 of Lecture Notes in Computer Science, pages 668--674. Springer

    work page doi:10.1007/11573548 2005

  5. [5]

    Anthropic. 2024. https://www.anthropic.com/news/claude-3-family The claude 3 model family: Opus, sonnet, haiku

    work page 2024

  6. [6]

    o rn W. Schuller, and Elisabeth Andr \' e . 2020. https://doi.org/10.1007/S13218-020-00632-3 explainable cooperative machine learning with NOVA . K \

    Tobias Baur, Alexander Heimerl, Florian Lingenfelser, Johannes Wagner, Michel F. Valstar, Bj \" o rn W. Schuller, and Elisabeth Andr \' e . 2020. https://doi.org/10.1007/S13218-020-00632-3 explainable cooperative machine learning with NOVA . K \" u nstliche Intell. , 34(2):143--164

    work page doi:10.1007/s13218-020-00632-3 2020

  7. [7]

    Farah Benamara and Patrick Saint Dizier. 2003. https://aclanthology.org/E03-1060 WEBCOOP : A cooperative question answering system on the web . In 10th Conference of the E uropean Chapter of the Association for Computational Linguistics , Budapest, Hungary. Association for Computational Linguistics

    work page 2003

  8. [8]

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html Language models can explain neurons in language models . Online: https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html

    work page 2023

Show all 52 references
  1. [9]

    Laura-Ana-Maria Bostan and Roman Klinger. 2018. https://aclanthology.org/C18-1179 An analysis of annotated corpora for emotion classification in text . In Proceedings of the 27th International Conference on Computational Linguistics, pages 2104--2119, Santa Fe, New Mexico, USA...

    2018

  2. [10]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jef...

    2020 arXiv

  3. [11]

    Tanise Ceron, Neele Falk, Ana Bari \'c , Dmitry Nikolaev, and Sebastian Pad \'o . 2024. https://doi.org/10.1162/tacl_a_00710 Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLM s . Transactions of the Association for Computation...

    2024 doi

  4. [12]

    Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, and Yanghua Xiao. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.215 MAPO : Boosting large language model performance with model-adaptive prompt optimization . In Findings of the Ass...

    2023 doi

  5. [13]

    Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang. 2021. https://doi.org/10.18653/v1/2021.findings-acl.161 Template-based named entity recognition using BART . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1835--1845, Online. Associati...

    2021 doi

  6. [14]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  7. [15]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al - Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Z...

    2024 doi arXiv

  8. [16]

    Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \" a schel. 2024. https://openreview.net/forum?id=9ZxnPZGmPU Promptbreeder: Self-referential self-improvement via prompt evolution . In Forty-first International Conference on Machine Learnin...

    2024

  9. [17]

    Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston. 2019. https://doi.org/10.18653/v1/P19-1358 Learning from dialogue after deployment: Feed yourself, chatbot! In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pag...

    2019 doi

  10. [18]

    Louisa Heidrich, Emanuel Slany, Stephan Scheele, and Ute Schmid. 2023. https://doi.org/10.3390/MAKE5040076 Faircaipi: A combination of explanatory interactive and fair machine learning for human and machine bias reduction . Mach. Learn. Knowl. Extr., 5(4):1519--1538

    2023 doi

  11. [19]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L \' e lio Renard Lavaud, Marie - Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, ...

    2023 doi arXiv

  12. [20]

    Xu, Jun Araki, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. https://doi.org/10.1162/tacl_a_00324 How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423--438

    2020 doi

  13. [21]

    Shubhra Kanti Karmaker Santu and Dongji Feng. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.946 TEL e R : A general taxonomy of LLM prompts for benchmarking complex tasks . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14197--14203, Si...

    2023 doi

  14. [22]

    Ana Kotarcic, Dominik Hangartner, Fabrizio Gilardi, Selina Kurer, and Karsten Donnay. 2022. https://aclanthology.org/2022.findings-emnlp.548 Human-in-the-loop hate speech classification in a multilingual context . In Findings of the Association for Computational Linguistics: E...

    2022

  15. [23]

    Piyawat Lertvittayakumjorn, Lucia Specia, and Francesca Toni. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.24 FIND : H uman-in-the- L oop D ebugging D eep T ext C lassifiers . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP...

    2020 doi

  16. [24]

    Xiang Lisa Li and Percy Liang. 2021. https://doi.org/10.18653/v1/2021.acl-long.353 Prefix-tuning: Optimizing continuous prompts for generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen...

    2021 doi

  17. [25]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. https://doi.org/10.18653/v1/2022.deelio-1.10 What makes good in-context examples for GPT -3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extr...

    2022 doi

  18. [26]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. https://doi.org/10.1145/3560815 Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing . ACM Comput. Surv. , 55(9):195:1--195:35

    2023 doi

  19. [27]

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. https://doi.org/10.18653/v1/2022.acl-long.556 Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity . In Proceedings of the 60th Annual Meeting of th...

    2022 doi

  20. [28]

    Lundberg and Su - In Lee

    Scott M. Lundberg and Su - In Lee. 2017. https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html A unified approach to interpreting model predictions . In Advances in Neural Information Processing Systems 30: Annual Conference on Neural In...

    2017

  21. [29]

    Manning, Prabhakar Raghavan, and Hinrich Sch \" u tze

    Christopher D. Manning, Prabhakar Raghavan, and Hinrich Sch \" u tze. 2008. https://doi.org/10.1017/CBO9780511809071 Introduction to information retrieval . Cambridge University Press

    2008 doi

  22. [30]

    Philip Mavrepis, Georgios Makridis, Georgios Fatouros, Vasileios Koukos, Maria Margarita Separdani, and Dimosthenis Kyriazis. 2024. https://doi.org/10.48550/ARXIV.2401.13110 XAI for all: Can large language models simplify explainable ai? CoRR, abs/2401.13110

    2024 doi arXiv

  23. [31]

    Yarik Menchaca Resendiz and Roman Klinger. 2025. Mopo: Multi-objective prompt optimization for affective text generation. In Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, UAE. International Committee on Computational Linguistics

    2025

  24. [32]

    Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. https://doi.org/10.18653/v1/2022.acl-long.244 Cross-task generalization via natural language crowdsourcing instructions . In Proceedings of the 60th Annual Meeting of the Association for Computationa...

    2022 doi

  25. [33]

    Saif Mohammad. 2012. https://aclanthology.org/S12-1033 \# emotional tweets . In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics -- Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth Internation...

    2012

  26. [34]

    OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  27. [35]

    Flor Miriam Plaza-del Arco, Mar \' a-Teresa Mart \' n-Valdivia, and Roman Klinger. 2022. https://aclanthology.org/2022.coling-1.592 Natural language inference prompts for zero-shot emotion classification in text across corpora . In Proceedings of the 29th International Confere...

    2022

  28. [36]

    Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.494 Automatic prompt optimization with `` gradient descent '' and beam search . In Proceedings of the 2023 Conference on Empirical Methods in Natural L...

    2023 doi

  29. [37]

    Guanghui Qin and Jason Eisner. 2021. https://doi.org/10.18653/v1/2021.naacl-main.410 Learning how to ask: Querying LM s with mixtures of soft prompts . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human L...

    2021 doi

  30. [38]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. https://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . J. Mach. Learn. Res.,...

    2020

  31. [39]

    why should I trust you?

    Marco T \' u lio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. https://doi.org/10.1145/2939672.2939778 "why should I trust you?": Explaining the predictions of any classifier . In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data M...

    2016

  32. [40]

    Marco T \' u lio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. https://doi.org/10.1609/AAAI.V32I1.11491 Anchors: High-precision model-agnostic explanations . In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Appl...

    2018 doi

  33. [41]

    Duarte, and Jochen Garcke

    Ribana Roscher, Bastian Bohn, Marco F. Duarte, and Jochen Garcke. 2020. https://doi.org/10.1109/ACCESS.2020.2976199 Explainable machine learning for scientific insights and discoveries . IEEE Access , 8:42200--42216

    2020

  34. [42]

    Logan IV, Eric Wallace, and Sameer Singh

    Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.346 A uto P rompt: E liciting K nowledge from L anguage M odels with A utomatically G enerated P rompts . In Proceedings of the 2020 Conference o...

    2020 doi

  35. [43]

    Stefano Teso and Kristian Kersting. 2019. https://doi.org/10.1145/3306618.3314293 Explanatory Interactive Machine Learning . In Proceedings of the 2019 AAAI / ACM Conference on AI , Ethics , and Society , pages 239--245, Honolulu, HI, USA. ACM

    2019

  36. [44]

    Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang

    Zijie J. Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang. 2021. https://aclanthology.org/2021.hcinlp-1.8 Putting humans in the natural language processing loop: A survey . In Proceedings of the First Workshop on Bridging Human -- Computer Interaction and Natural Language Processi...

    2021

  37. [45]

    Colin Wei, Sang Michael Xie, and Tengyu Ma. 2021. https://proceedings.neurips.cc/paper/2021/hash/86b3e165b8154656a71ffe8a327ded7d-Abstract.html Why do pretrained language models help in downstream tasks? an analysis of head and prompt tuning . In Advances in Neural Information...

    2021

  38. [46]

    Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/a00548031e4647b13042c97c922fadf1-Abstract-Conference.html Hard prompts made easy: Gradient-based discrete optimization for prompt...

    2023

  39. [47]

    Xuansheng Wu, Haiyan Zhao, Yaochen Zhu, Yucheng Shi, Fan Yang, Tianming Liu, Xiaoming Zhai, Wenlin Yao, Jundong Li, Mengnan Du, and Ninghao Liu. 2024. https://doi.org/10.48550/ARXIV.2403.08946 Usable XAI: 10 strategies towards exploiting explainability in the LLM era . CoRR, a...

    2024 doi arXiv

  40. [48]

    Lei Xu, Yangyi Chen, Ganqu Cui, Hongcheng Gao, and Zhiyuan Liu. 2022. https://doi.org/10.18653/v1/2022.findings-naacl.137 Exploring the universal vulnerability of prompt-based learning paradigm . In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1...

    2022 doi

  41. [49]

    Qinyuan Ye, Mohamed Ahmed, Reid Pryzant, and Fereshte Khani. 2024. https://doi.org/10.18653/v1/2024.findings-acl.21 Prompt engineering a prompt engineer . In Findings of the Association for Computational Linguistics: ACL 2024, pages 355--385, Bangkok, Thailand. Association for...

    2024 doi

  42. [50]

    Wenpeng Yin, Jamaal Hay, and Dan Roth. 2019. https://doi.org/10.18653/v1/D19-1404 Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th In...

    2019 doi

  43. [51]

    Chunting Zhou, Junxian He, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.192 Prompt consistency for zero-shot task generalization . In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 26...

    2022 doi

  44. [52]

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2023. https://openreview.net/forum?id=92gvk82DE- Large language models are human-level prompt engineers . In The Eleventh International Conference on Learning Representation...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.