Pith. sign in

REVIEW 3 major objections 4 minor 47 references

A theory-grounded, RAG-augmented multi-stage pipeline ties the top rank in a multilingual humor-generation task, while its two agentic variants add no detectable quality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 05:54 UTC pith:567G5KRX

load-bearing objection A useful SemEval system report whose headline negative finding overreaches: the agentic comparison is only properly controlled for GPT-5. the 3 major comments →

arxiv 2607.13189 v1 pith:567G5KRX submitted 2026-07-14 cs.CL cs.AI

RAGthoven at SemEval-2026 Task 1: A Multi-Stage Pipeline Walks Into a Benchmark and Barely Clears the Bar

Marek \v{S}uppa , Vikt\'oria Ondrejov\'a , Lucia Ganajov\'a , Gregor Karetka , Daniel Skala This is my paper
classification cs.CL cs.AI
keywords humor generationmultilingual constrained generationretrieval-augmented generationmulti-stage promptingagentic tool-callingElo leaderboardcomputational humor theoryself-critique pipeline
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish two things about using large language models to write humor under constraints. First, decomposing the task into a theory-grounded pipeline—an ideation stage boosted by retrieval from a small curated joke corpus, a best-of-N writer, a self-critique reflector, and a rubric-driven judge—reaches the top official rank in all three languages of a shared multilingual humor benchmark, statistically tied with a strong single-prompt frontier-model baseline. Second, converting those stages into autonomous tool-calling agents (a sequential ReAct-style variant and a multi-branch exploration variant) produced no quality advantage the authors could detect on a 12-instance English sample, despite far higher compute. The sympathetic reader cares because the negative result pushes against the assumption that more scaffolding and agentic complexity is always better for creative generation, and the language-dependent gap pattern raises the question of where structured prompting actually matters.

Core claim

Central claim: a multi-stage structure plus retrieval can match, but not clearly beat, a strong frontier model prompted simply, on a constrained humor-generation task. The final configuration—a Planner seeded with retrieved joke mechanisms, a Writer producing a dozen candidates, a Reflector offering rewrites, and a rubric-driven Judge—shares the top official rank group with the organizers' strong single-prompt baseline in English, Spanish, and Chinese, with overlapping 95% confidence intervals; its raw Elo leads by 42 points in Spanish and trails by 36 and 8 in English and Chinese, all within the same statistical tie. The second claim is negative: re-implementing the same stages as tool-call

What carries the argument

The work is carried by a four-stage prompt pipeline: a Planner that produces a structured joke blueprint (expected versus opposed scripts, benign-violation angle, mechanism label), a Writer that instantiates the plan into a best-of-N set of candidates with inline constraint checks, a Reflector that diagnoses failures and rewrites one or two candidates, and a Judge that scores survivors on a five-criterion rubric and returns only the final joke text. The main lever is retrieval-augmented generation at the ideation stage: a curated 98-joke corpus annotated with humor mechanisms is embedded, with the top four exemplars retrieved by cosine similarity plus cross-encoder re-ranking injected into t

Load-bearing premise

The negative finding rests entirely on a 12-instance English sample judged qualitatively by the authors across four models; if a larger, blinded human evaluation finds any consistent agentic advantage, the diminishing-returns conclusion collapses.

What would settle it

Run a blinded preference study on at least 100 instances per language comparing the non-agentic pipeline against both agentic variants with identical backbones and recorded tool-call budgets; if either variant wins a significant share of pairwise comparisons in any language, the paper's central negative claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the negative agentic finding holds, plan-write-critique-audit scaffolding does not justify its added latency and cost for constrained creative generation once the base model is strong.
  • RAG at the ideation stage is the paper's clearest qualitative inflection point: it helps most when the two required words share no obvious semantic link, suggesting the mechanism library widens the candidate space.
  • Because the pipeline and a single-prompt baseline are statistically tied in every language, the simple baseline is a strong lower bound that any future scaffolding must beat.
  • All prompts are deliberately language-agnostic, so the largest raw lead over the baseline appearing in Spanish indicates the model's multilingual capacity, not prompt localization—an effect the paper flags as needing controlled ablations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the 12-instance English sample is thin evidence for a three-language negative claim; the more likely regime for agentic payoff is a low-resource language or a novel constraint type, both untested here.
  • Editorial inference: the reported ties all rest on the organizer's overlapping confidence intervals, which the paper does not print; without the interval widths, readers cannot tell how much the Elo gaps are separated.
  • Editorial inference: a matched-budget comparison—one agentic loop over the same candidate pool versus the four fixed calls—would separate the compute cost from the orchestration quality, which the current experiment does not do.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper describes RAGthoven, a multi-stage LLM pipeline for multilingual constrained humor generation in SemEval-2026 Task 1, Subtask A. The pipeline combines a theory-grounded Planner, Best-of-N Writer, Reflector, and Judge with a 98-joke RAG corpus, and the paper reports ten experimental configurations. On the official leaderboard, RAGthoven is in the tied Rank 1 group in all three languages; in Spanish it has the highest raw Elo (1182 vs. 1140), but all cross-language gaps are within the same official tied rank group with overlapping confidence intervals. The paper's main novel claim is a negative result: in a 12-instance English sample across four frontier models, ReAct-style (Exp09) and autonomous multi-branch (Exp10) agentic variants did not produce outputs judged superior to the non-agentic pipeline, despite substantially higher tool-call budgets. The paper also presents a qualitative ablation of the pipeline stages and releases prompts, configurations, and run scripts.

Significance. If the leaderboard results are taken as reported, the paper provides an honest, externally evaluated system description and a useful data point: a moderately complex RAG pipeline reaches the top tied rank group in three languages without language-specific prompting. The larger contribution is the negative result on agentic scaffolding, which could help practitioners avoid expensive tool-calling orchestration. Strengths include reliance on organizer-run human pairwise Elo rankings for the main rank claim (so the central rank claim is not circular), a clearly specified set of ten configurations, and the release of prompts/configurations/run scripts. The main empirical caveat is that the agentic comparison is small-scale, self-judged, English-only, and, for three of four models, confounded by model identity because the non-agentic baseline is GPT-5 in all cases. These caveats currently place the headline 'diminishing returns' claim beyond what the evidence supports, but the underlying contribution is recoverable with additional baselines and careful rewording.

major comments (3)
  1. [§4.2, §5, §6, Abstract] The negative agentic finding is claimed across four frontier models, but the comparison is controlled for only one. Section 4.2 states that Exp09/10 evaluate GPT-5, Gemini 3 Pro, Claude Sonnet 4.5, and Claude Opus 4.5 'against a non-agentic GPT-5 baseline using the EXP08 pipeline.' For three of the four models, a failure of agentic-Gemini or agentic-Sonnet against non-agentic-GPT-5 could reflect model-specific joke-writing ability rather than the uselessness of scaffolding. The paper already has non-agentic EXP08 outputs for Gemini 3 Pro and Claude Sonnet 4.5 in Appendix B, but does not use them as baselines in the agentic comparison; for Opus, a same-model non-agentic run is needed. The limitations section does not flag this baseline mismatch. The claim should be restricted to GPT-5 or the missing same-model baselines should be supplied.
  2. [§5, §6, Limitations] The evidence for the central negative result is too thin to support the general conclusion. The 12-instance English sample is of unspecified provenance, judged qualitatively by the authors on four frontier models, with no error bars, inter-annotator agreement, or significance testing. The paper correctly labels this as a held-out sample but never says how the instances were selected or whether the sample is representative. Given that the negative agentic finding is presented as the paper's main novel contribution, the evaluation needs either (a) a larger sample with independent, blind judges and a pre-specified measure, or (b) a carefully hedged claim limited to 'no advantage was observed in an informal 12-instance English sample.' As written, the conclusion in §6 that 'increased agentic complexity is difficult to justify for this task' overgeneralizes.
  3. [Abstract, §6] The phrase 'language-dependent diminishing returns' overstates the evidence. The cross-language differences in raw Elo all fall within the same tied rank group, so the presence of a language-dependent effect is not established; and the agentic comparison was conducted only on English, so it cannot support a language-dependent claim about scaffolding. The conclusion should be softened to something like: 'raw Elo gaps differ across languages, but all systems are statistically tied, and the agentic variants showed no clear advantage in a small English sample.' This would align the abstract with the actual experimental design.
minor comments (4)
  1. [Throughout] Notation is inconsistent: the title uses 'RAGthoven', many text passages use 'RAGTHOVEN', and experiment labels appear as both 'Exp09' and 'EXP09'. Please standardize. Also, in the conclusion, 'MW AHAHA' contains an erroneous space.
  2. [§4.2] The description of the 12-instance held-out English sample should state how it was constructed (random, adversarial, stratified?) and whether it is drawn from trial or test data. This is important because the null result can only be interpreted relative to the sample.
  3. [§5, Table 2] The paper uses 'statistically tied' to mean systems sharing the same official rank group with overlapping 95% confidence intervals. Overlapping confidence intervals are a weaker criterion than a formal equivalence test; consider using 'same official rank group' to avoid implying a stronger statistical statement than the organizer-reported ranking provides.
  4. [Limitations] The claim that Exp10 'proved less efficient while requiring substantially more tool calls' is presented as an observed pattern, but no tool-call counts or latency numbers are reported because runs were not instrumented. A concrete range of tool-call counts per instance for Exp09 and Exp10 would make this claim more informative and would not require a full cost–quality study.

Circularity Check

0 steps flagged

No significant circularity: the headline results are externally benchmarked, and the only self-citation is non-load-bearing infrastructure.

full rationale

The paper's central empirical claims do not reduce to their own inputs. The Rank 1 / tied-Elo result is produced by the organizers' external human pairwise annotation and Elo leaderboard (Table 2), not by any quantity fitted inside the paper. The negative agentic finding (Exp09/10) is an author-judged qualitative comparison on a 12-instance English sample; this is a validity and generalizability concern, especially because only GPT-5 is compared against a same-backbone non-agentic baseline while the other three models are compared against a different model, but that is a confound, not a circular derivation. The only self-citation, 'RAGthoven framework (Karetka et al., 2025),' is cited as the toolkit the pipeline is built on; no theoretical or leaderboard claim depends on that citation for its truth, and no prediction is renamed from a fitted parameter. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via self-citation. The limitations section explicitly concedes unresolved questions about which pipeline stage is the binding constraint, which further confirms that the paper is not asserting a derived equivalence between its inputs and outputs. The paper is therefore self-contained against external benchmarks, and no circular step is established.

Axiom & Free-Parameter Ledger

10 free parameters · 6 axioms · 0 invented entities

This is an ML systems paper; the free parameters are engineering choices, not theoretical constants. The most consequential assumptions are domain-level: the humor theories, the Elo benchmark's validity, the embedding model's relevance for joke retrieval, and the transfer of an English-centric pipeline to Spanish/Chinese. No entities beyond software tools (ConstraintAudit) are introduced.

free parameters (10)
  • N_cands (Writer candidates) = 12
    Best-of-12 chosen in Exp04; affects diversity and Judge pool; not formally tuned.
  • sampling temperature = 1.0
    For GPT-5 Writer; hand-set, no ablation.
  • RAG corpus size = 98 jokes
    Curated corpus, English-centric; retrieval quality depends on this hand-built resource.
  • retrieval top-K = 12 -> 4
    Top-12 by embedding cosine, top-4 after cross-encoder re-rank, passed to Planner; hand-chosen.
  • Reflector rewrite count = 1-2
    Number of revised candidates; prompt-defined.
  • Exp09 max iterations = 24
    Tool-calling round cap; arbitrary policy choice.
  • Exp10 max iterations = 36
    Round cap for multi-branch orchestration; arbitrary.
  • Exp10 branch count = 2-4
    Autonomous branch count range; not fixed.
  • Judge rubric weights = equal, 1-5 scale per criterion
    Five criteria scored equally; no calibration against human judgments.
  • ConstraintAudit thresholds = anchor token length >= 4; overlap >= 6 tokens
    Deterministic checker parameters chosen without tuning.
axioms (6)
  • domain assumption Benign Violation Theory and Script-based Semantic Theory of Humor are valid generative models for humor
    The entire prompt design is grounded in these theories; if they are incomplete, the scaffolding has no principled basis. Invoked in Section 2 and throughout planner/reflector prompts.
  • domain assumption Human pairwise Elo rating on the shared task leaderboard is a valid measure of humor quality
    The central rank claim depends on the organizer's evaluation protocol. Paper relies on this without questioning reliability (Section 5; Castro et al., 2026).
  • domain assumption Cosine similarity in all-MiniLM-L6-v2 embedding space retrieves relevant joke mechanisms
    RAG component uses this embedding to fetch top-4 exemplars; no evaluation of retrieval quality is given (Section 4.1).
  • domain assumption An LLM-as-a-judge with a five-point rubric reliably selects the funniest candidate
    Judge stage chooses output; no validation against human preference, though the organizer leaderboard later provides external signal. Invoked in Section 3.1.
  • domain assumption Language-agnostic prompts and English-centric RAG corpus transfer to Spanish and Chinese
    Prompts are identical across languages; the authors flag this as a variable for future work, but the cross-language Elo comparison assumes fair transfer (Section 4.2, Limitations).
  • ad hoc to paper A 12-instance English sample judged by the authors is representative for comparing agentic and non-agentic variants
    Section 5 ('Agentic experiments') draws the negative finding from this sample; no power analysis or external judges.

pith-pipeline@v1.3.0-alltime-deepseek · 13503 in / 15138 out tokens · 125834 ms · 2026-08-02T05:54:49.648912+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of RAGthoven at SemEval-2026 Task 1: A Multi-Stage Pipeline Walks Into a Benchmark and Barely Clears the Bar." pith.science (2026). https://pith.science/paper/567G5KRX

@misc{pith2026260713189,
  author       = {Pith},
  title        = {Pith review of: RAGthoven at SemEval-2026 Task 1: A Multi-Stage Pipeline Walks Into a Benchmark and Barely Clears the Bar},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/567G5KRX}},
  note         = {Machine review of arXiv:2607.13189}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We present RAGthoven, our system for SemEval-2026 Task 1 (MWAHAHA), Subtask A (multilingual constrained humor generation in English, Spanish, and Chinese). RAGthoven decomposes creative text generation into a multi-stage large language model (LLM) pipeline (Planner, Best-of-N Writer, Reflector for self-critique, LLM-as-a-judge Judge) grounded in computational humor theory (Benign Violation Theory, Script-based Semantic Theory of Humor) and refined across ten experiments. In our final configuration, we augment the Planner with retrieval-augmented generation (RAG) from a curated joke corpus, seeding generation with diverse joke mechanisms. We also evaluate two agentic variants -- ReAct-style sequential tool-calling (Exp09) and autonomous multi-branch orchestration (Exp10) -- that expose the same four stages with a deterministic ConstraintAudit checker. Across four frontier models on a held-out 12-instance English sample, neither agentic variant produced outputs we judged superior to the non-agentic pipeline despite substantially higher tool-call budgets. RAGthoven shares Rank 1 with the Gemini 2.5 Flash baseline in all three languages, with overlapping organizer-reported confidence intervals. In Spanish, it leads the baseline by 42 raw Elo points (1182 vs. 1140), while in English (1045 vs. 1081) and Chinese (1045 vs. 1053) the baseline holds the higher raw rating within the same statistical tie. Together, these results suggest language-dependent diminishing returns from elaborate multi-stage prompt engineering and agentic scaffolding once a strong frontier model is in the loop.

Figures

Figures reproduced from arXiv: 2607.13189 by Daniel Skala, Gregor Karetka, Lucia Ganajov\'a, Marek \v{S}uppa, Vikt\'oria Ondrejov\'a.

Figure 1
Figure 1. Figure 1: Full pipeline for EXP08. 3 RAGthoven: A Multi-Stage RAG Pipeline RAGTHOVEN is a configuration-driven pipeline built on the RAGthoven framework (Karetka et al., 2025).1 All stages are implemented as prompted LLM calls, meaning no model weights are modi￾fied [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Interactive viewer used during development, displaying outputs from multiple experimental configurations [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 5 canonical work pages

  1. [1]

    Salvatore Attardo and Victor Raskin. 1991. Script theory revis(it)ed: Joke similarity and joke representation model. Humor: International Journal of Humor Research, 4(3--4):293--347

    1991

  2. [2]

    Tian Bai, Yongwang Cao, Yan Ge, and Haitao Yu. 2025. https://doi.org/10.1609/AAAI.V39I22.34514 MP : Endowing large language models with lateral thinking . In Proceedings of the AAAI Conference on Artificial Intelligence

    work page doi:10.1609/aaai.v39i22.34514 2025

  3. [3]

    Bunescu and Oseremen O

    Razvan C. Bunescu and Oseremen O. Uduehi. 2022. https://doi.org/10.18653/v1/2022.flp-1.10 Distribution-based measures of surprise for creative language: Experiments with humor and metaphor . In Proceedings of the 3rd Workshop on Figurative Language Processing (FLP), pages 68--78

    work page doi:10.18653/v1/2022.flp-1.10 2022

  4. [4]

    Santiago Castro, Luis Chiruzzo, Santiago G \'o ngora, Salar Rahili, Naihao Deng, Ignacio Sastre, Victoria Amoroso, Guillermo Rey, Aiala Ros \'a , Guillermo Moncecchi, J. A. Meaney, Juan Jos \'e Prada, and Rada Mihalcea. 2026. SemEval-2026 Task 1: MWAHAHA, Models Write Automatic Humor And Humans Annotate . In Proceedings of the 20th International Workshop ...

    2026

  5. [5]

    Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. 2024. https://doi.org/10.1145/3613904.3642731 Art or artifice? L arge language models and the false promise of creativity . In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery

    arXiv 2024

  6. [6]

    Jordan, Joseph E

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. https://arxiv.org/abs/2403.04132 Chatbot Arena : An open platform for evaluating LLMs by human preference . In Forty-first International Conference on Machine Learning, ICML...

    Pith/arXiv arXiv 2024

  7. [11]

    Gregor Karetka, Demetris Skottis, Lucia Dutkov \'a , Peter Hra s ka, and Marek S uppa. 2025. https://aclanthology.org/2025.coling-demos.12/ RAGthoven : A configurable toolkit for RAG -enabled LLM experimentation . In Proceedings of the 31st International Conference on Computational Linguistics: System Demonstrations, pages 117--125, Abu Dhabi, UAE. Associ...

    2025

  8. [12]

    Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2023. https://openreview.net/forum?id=_nGgzQjzaRy Decomposed prompting: A modular approach for solving complex tasks . In The Eleventh International Conference on Learning Representations

    2023

  9. [13]

    u ttler, Mike Lewis, Wen tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen tau Yih, Tim Rockt \"a schel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, volume 33, pages 9459--9474

    2020

  10. [14]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. https://arxiv.org/abs/2303.17651 Self-refine: Iterative refinement with self-feedback . In Adva...

    Pith/arXiv arXiv 2023

  11. [15]

    Peter McGraw and Caleb Warren

    A. Peter McGraw and Caleb Warren. 2010. https://doi.org/10.1177/0956797610376073 Benign violations: Making immoral behavior funny . Psychological Science, 21(8):1141--1149

    work page doi:10.1177/0956797610376073 2010

  12. [16]

    Victor Raskin. 1985. Semantic Mechanisms of Humor. D. Reidel, Dordrecht

    1985

  13. [17]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.18653/v1/D19-1410 Sentence- BERT : Sentence embeddings using S iamese BERT -networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982--3992, Hong Kong, Chi...

    doi:10.18653/v1/d19-1410 2019

  14. [18]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. https://arxiv.org/abs/2302.04761 Toolformer: Language models can teach themselves to use tools . In Advances in Neural Information Processing Systems, volume 36

    Pith/arXiv arXiv 2023

  15. [19]

    Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. https://arxiv.org/abs/2303.11366 Reflexion: Language agents with verbal reinforcement learning . In Advances in Neural Information Processing Systems, volume 36

    Pith/arXiv arXiv 2023

  16. [20]

    Jerry M. Suls. 1972. A two-stage model for the appreciation of jokes and cartoons: An information-processing analysis. In Jeffrey H. Goldstein and Paul E. McGhee, editors, The Psychology of Humor, pages 81--100. Academic Press, New York

    1972

  17. [21]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Conference on Learning Representations

    2023

  18. [23]

    Yubo Xie, Junze Li, and Pearl Pu. 2021. https://doi.org/10.18653/v1/2021.acl-short.6 Uncertainty and surprisal jointly deliver the punchline: Exploiting incongruity-based features for humor recognition . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Languag...

    work page doi:10.18653/v1/2021.acl-short.6 2021

  19. [24]

    Griffiths, Yuan Cao, and Karthik Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023 a . https://arxiv.org/abs/2305.10601 Tree of thoughts: Deliberate problem solving with large language models . In Advances in Neural Information Processing Systems, volume 36

    Pith/arXiv arXiv 2023

  20. [25]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023 b . https://openreview.net/forum?id=WE_vluYUL-X ReAct : Synergizing reasoning and acting in language models . In The Eleventh International Conference on Learning Representations

    2023

  21. [26]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://arxiv.org/abs/2306.05685 Judging LLM -as-a-judge with MT -bench and C hatbot A rena . In Advances in Neural Information Processing Systems, volume 36

    Pith/arXiv arXiv 2023

  22. [27]

    Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026) , year=

    Castro, Santiago and Chiruzzo, Luis and G. Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026) , year=

    2026

  23. [28]

    Victor Raskin , title =

  24. [29]

    Peter McGraw and Caleb Warren , title =

    A. Peter McGraw and Caleb Warren , title =. Psychological Science , volume =. 2010 , doi =

    2010

  25. [30]

    Retrieval-Augmented Generation for Knowledge-Intensive

    Patrick Lewis and Ethan Perez and Aleksandra Piktus and Fabio Petroni and Vladimir Karpukhin and Naman Goyal and Heinrich K. Retrieval-Augmented Generation for Knowledge-Intensive. Advances in Neural Information Processing Systems , volume =

  26. [31]

    Nils Reimers and Iryna Gurevych , title =. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , address =. 2019 , doi =

    2019

  27. [32]

    Jordan and Joseph E

    Wei-Lin Chiang and Lianmin Zheng and Ying Sheng and Anastasios Nikolas Angelopoulos and Tianle Li and Dacheng Li and Banghua Zhu and Hao Zhang and Michael I. Jordan and Joseph E. Gonzalez and Ion Stoica , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  28. [33]

    Metacognitive Prompting Improves Understanding in Large Language Models , author =. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , month = jun, year =. doi:10.18653/v1/2024.naacl-long.106 , url =

    doi:10.18653/v1/2024.naacl-long.106 2024

  29. [34]

    The Eleventh International Conference on Learning Representations , year =

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author =. The Eleventh International Conference on Learning Representations , year =

  30. [35]

    Humor: International Journal of Humor Research , volume =

    Script theory revis(it)ed: Joke Similarity and Joke Representation Model , author =. Humor: International Journal of Humor Research , volume =

  31. [36]

    2023 , address =

    Jentzsch, Sophie and Kersting, Kristian , booktitle =. 2023 , address =. doi:10.18653/v1/2023.wassa-1.29 , pages =

    doi:10.18653/v1/2023.wassa-1.29 2023

  32. [37]

    2023 , url =

    Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle =. 2023 , url =

    2023

  33. [38]

    Art or Artifice?

    Chakrabarty, Tuhin and Laban, Philippe and Agarwal, Divyansh and Muresan, Smaranda and Wu, Chien-Sheng , booktitle =. Art or Artifice?. 2024 , publisher =

    2024

  34. [39]

    The Eleventh International Conference on Learning Representations , year =

    Decomposed Prompting: A Modular Approach for Solving Complex Tasks , author =. The Eleventh International Conference on Learning Representations , year =

  35. [40]

    Proceedings of the 31st International Conference on Computational Linguistics: System Demonstrations , pages =

    Karetka, Gregor and Skottis, Demetris and Dutkov. Proceedings of the 31st International Conference on Computational Linguistics: System Demonstrations , pages =. 2025 , address =

    2025

  36. [41]

    2025 , doi =

    Bai, Tian and Cao, Yongwang and Ge, Yan and Yu, Haitao , booktitle =. 2025 , doi =

    2025

  37. [42]

    Suls , title =

    Jerry M. Suls , title =. The Psychology of Humor , editor =. 1972 , publisher =

    1972

  38. [43]

    Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognition , author =. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) , pages =. 2021 , url =

    2021

  39. [44]

    Proceedings of the 3rd Workshop on Figurative Language Processing (FLP) , pages =

    Distribution-Based Measures of Surprise for Creative Language: Experiments with Humor and Metaphor , author =. Proceedings of the 3rd Workshop on Figurative Language Processing (FLP) , pages =. 2022 , url =

    2022

  40. [45]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , month = aug, year =

    Getting Serious about Humor: Crafting Humor Datasets with Unfunny Large Language Models , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , month = aug, year =. doi:10.18653/v1/2024.acl-short.76 , pages =

    doi:10.18653/v1/2024.acl-short.76 2024

  41. [46]

    Advances in Neural Information Processing Systems , volume =

    Self-Refine: Iterative Refinement with Self-Feedback , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  42. [47]

    and Zhang, Hao and Gonzalez, Joseph E

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , booktitle =. Judging. 2023 , url =

    2023

  43. [48]

    Advances in Neural Information Processing Systems , volume =

    Toolformer: Language Models Can Teach Themselves to Use Tools , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  44. [49]

    Advances in Neural Information Processing Systems , volume =

    Reflexion: Language Agents with Verbal Reinforcement Learning , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  45. [50]

    Advances in Neural Information Processing Systems , volume =

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  46. [51]

    Do Androids Laugh at Electric Sheep?

    Hessel, Jack and Marasovi. Do Androids Laugh at Electric Sheep?. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month = jul, year =. doi:10.18653/v1/2023.acl-long.41 , url =

    doi:10.18653/v1/2023.acl-long.41 2023

  47. [52]

    Pun Generation with Surprise , author =. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , month = jun, year =. doi:10.18653/v1/N19-1172 , url =

    work page doi:10.18653/v1/n19-1172 2019