Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper builds a ten-category taxonomy of laughable contexts in spontaneous Japanese text conversation and reports that GPT-4o recognizes them with an F1 score of 43.14%.

desk verdict A clearly-written but modest paper whose human binary annotation is real and whose LLM-generated taxonomy is honestly flagged as unvalidated; the central claim about human laughter reasons still needs human validation before it carries weight. read the letter →

arxiv 2501.16635 v2 pith:RCUSEKT2 submitted 2025-01-28 cs.CL cs.AI

Koji Inoue , Mikey Elmers , Divesh Lala , Tatsuya Kawahara This is my paper · ORCID
classification cs.CLcs.AI
keywords laughterlaughablecontextannotationtaxonomygenerationspontaneoustextdialogueconversationalAIlargelanguagemodelshumorrecognitionRealPersonaChat
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Why do people laugh in ordinary text conversations? This paper tries to make that question answerable by machines. It annotates nearly 900 dialogues from an existing Japanese chat corpus with five annotators marking each utterance as laughable or not, then feeds the majority-voted laughable examples to an LLM to produce explanations, which are iteratively organized into a ten-category taxonomy of laugh reasons ranging from 'Empathy and Affinity' to 'Exaggeration.' The same LLM, GPT-4o, is then tested in a zero-shot setting on recognizing the human laughable labels and reaches an F1 score of 43.14%, above the 14.8% majority-class baseline. The paper's contribution is a semi-automated pipeline for turning cheap binary annotations into a structured account of why laughter happens, plus a benchmark result showing how far current LLMs still are from capturing it.

What carries the argument

The central mechanism is a semi-automated annotation-and-taxonomy-generation pipeline. Stage one is a light human task: five annotators give a binary laughable/non-laughable label per utterance, and majority voting selects positive cases. Stage two uses GPT-4o to write a short explanation of why a third party might judge the utterance laughable. Stage three applies an iterative LLM-based taxonomy-generation procedure: starting from the first subset of explanations, GPT-4o proposes initial category labels; each new subset updates the taxonomy under LLM guidance, with occasional manual validation, until all 3,739 explanations are organized into ten final labels. The same model then assigns possibly multiple taxonomy labels to each sample, yielding the label distribution and a correlation matrix among labels.

What would settle it

Run the same 3,739 laughable contexts through a new annotation study in which humans freely write the reason they laughed or judged the context laughable, then compare those free-text reasons with GPT-4o's generated explanations and with the ten taxonomy labels the model assigned. If human reasons and model reasons fail to converge, or if a separate panel of humans cannot reproduce the ten labels at better-than-chance agreement, the paper's claim that the taxonomy explains laughter would be falsified.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a usable taxonomy of laughable contexts can be produced without expensive fine-grained human annotation. Five annotators made binary laughable/non-laughable decisions for each utterance in 900 dialogues of the RealPersonaChat corpus; a majority vote selected 3,739 laughable contexts. GPT-4o then generated natural-language reasons for each context, and an iterative LLM-driven clustering process, manually checked at each step, converged on ten taxonomy labels (Empathy and Affinity, Humor and Surprise, Relaxed Atmosphere, Self-Disclosure and Friendliness, Cultural Background and Shared Understanding, Nostalgia and Fondness, Self-Deprecating Humor, Defying Expectations, Positive Energy, and Exaggeration), with multiple labels allowed per sample. Tested against the human majority labels in a zero-shot chain-of-thought setting, GPT-4o attained 41.66% precision, 44.72% recall, and 43.14% F1; the taxonomy-based breakdown shows it is comparatively better at Self-Deprecating Humor and Defying Expectations and much worse at Nostalgia and Fondness and Positive Energy. The paper reads these patterns as evidence that current LLMs lack the conversational, cultural, and temporal understanding needed to know when laughter would occur.

Load-bearing premise

The load-bearing premise is that GPT-4o's post-hoc explanations of why an utterance is laughable are valid stand-ins for the human annotators' real reasons, and that a taxonomy built from those explanations is therefore a taxonomy of human laughter; this premise is assumed rather than tested.

Editorial extensions

If this is right

  • The ten-label taxonomy gives conversational AI systems a structured target: to laugh appropriately, a system could first predict laughability, then infer which reason category applies.
  • The zero-shot F1 score of 43.14% implies that current LLM-based dialogue systems should not be trusted to time laughter on their own, and the per-category analysis points to missing capabilities such as story-time comprehension and positive reframing.
  • The pipeline generalizes to other annotation tasks where humans can reliably produce only simple binary labels but richer explanations are needed for model training or analysis.
  • Because the annotation covers 900 of the corpus's roughly 14,000 dialogues, the paper frames its results as a foundation for a larger future dataset rather than a complete resource.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the taxonomy is taken as descriptive of actual human laughter reasons, the natural next step is a human agreement study on the ten labels for the same 3,739 contexts; the paper reports no such reliability check.
  • The strong skew in label frequencies (Empathy and Affinity appears in 80.6% of samples, Cultural Background in only 4.7%) may reflect the LLM's explanation style as much as genuine distribution in the corpus, which a human-labeling study could disentangle.
  • The human binary annotations themselves show high subjectivity, since 47.17% of utterances were judged laughable by exactly one of five annotators; a graded laughter-probability output might fit the data better than the hard binary used here.
  • The paper does not measure human performance on the same recognition task, so the 43.14% F1 number has no human benchmark; collecting human-human agreement would calibrate how much headroom actually remains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper annotates laughable contexts in 900 Japanese spontaneous text dialogues from the RealPersonaChat corpus. Five annotators give binary laughable judgments per utterance; majority voting yields 3,739 positive contexts (14.8%). For these positive contexts, GPT-4o generates free-text explanations of why the context is laughable, proposes a ten-category taxonomy from those explanations, and assigns (multiple) taxonomy labels to each sample. The paper then evaluates GPT-4o in a zero-shot setting on the binary laughable recognition task against the majority labels, reporting an F1 score of 43.14%. The authors position the taxonomy as a resource for conversational AI and as a step toward understanding the reasons for human laughter.

Significance. If the taxonomy were validated against human judgments, the contribution would be useful: it offers a moderately sized, manually binary-annotated subset of RealPersonaChat, a ten-category taxonomy of laughable-context reasons, and a reproducible zero-shot GPT-4o recognition result with a reasonable baseline comparison. Credit is due for the manual binary annotation effort, the use of majority voting, the inclusion of related literature for each taxonomy label, and the correlation analysis in the appendix. However, the central claim that the taxonomy captures the underlying reasons for human laughter is not yet supported: the taxonomy is constructed and applied entirely by GPT-4o, with no human agreement study for the explanations or the taxonomy labels. The paper's own conclusion defers validation of the taxonomy to future work, which makes the current version a proposal of a semi-automated taxonomy-generation pipeline rather than a demonstrated taxonomy of human laughter.

major comments (3)
  1. [§3 and Conclusion] The taxonomy-generation pipeline is entirely LLM-mediated: GPT-4o generates the reason sentences, GPT-4o proposes the taxonomy labels, and GPT-4o assigns the labels, with only an unquantified 'manually validated when necessary' step. The Conclusion explicitly defers 'validating the generated taxonomy' to future work. Since the abstract and introduction describe the taxonomy as the underlying reasons for laughable contexts (human laughter), this missing human validation is load-bearing. Concretely, report inter-annotator agreement between human and GPT-4o assignments on a held-out sample, or relabel the contribution as 'an LLM-generated taxonomy' and adjust the claims accordingly.
  2. [§2, Table 2] No inter-annotator agreement coefficient (e.g., Cohen's kappa or Krippendorff's alpha) is reported for the binary laughable labels. The taxonomy is built on all 3,739 majority-positive samples, of which 2,731 (about 73%) received only 3/5 agreement, meaning two of five annotators judged each of those contexts as non-laughable. Reporting an agreement coefficient overall and for the 3/5 subset, and analyzing the taxonomy on high-agreement samples, is necessary to establish reliability of the input labels that the taxonomy is derived from.
  3. [§4, Table 4] Table 4 is described as 'accuracy within each label', but the numbers are conditional recall (the fraction of GPT-4o positive outputs among samples carrying each label), and samples can appear in multiple rows because multiple labels can be assigned to one context. This makes cross-label comparisons, such as the claim that 'Nostalgia and Fondness' and 'Positive Energy' are harder for the LLM, unsubstantiated. Report multi-label-aware precision, recall, and F1 for each label, and avoid inferential claims (e.g., that the model 'may effectively capture' certain contexts) without significance tests or confidence intervals.
minor comments (5)
  1. [§4] The statement that F1=43.14% is 'significantly above the chance level (14.8%)' conflates the positive-class prevalence with chance performance on F1; a majority-class baseline or a random-prediction baseline with the same prevalence should be reported.
  2. [Appendix A] The correlation matrix uses abbreviated axis labels (e.g., 'Empathy', 'Humor'), and the text refers to 'Defying Expressions' while Table 3 uses 'Defying Expectations'; please fix the terminology and provide full, readable label names in the figure.
  3. [§3] The sentence that the cited references 'substantiate the explanatory power of our taxonomy' is too strong: the references show that the categories resemble constructs already discussed in the literature, which is not the same as validating the taxonomy.
  4. [§4] The recognition prompt is described but not reproduced; to make the zero-shot result reproducible, the full prompt should be included or released alongside the annotations and code.
  5. [§2] Please clarify the annotation protocol in more detail, including the exact written instructions given to annotators and their background, since the definition of 'laughable' as 'whether the next person would laugh' may be interpreted differently across annotators.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the binary laughable labels are human annotations; the taxonomy is explicitly LLM-generated and not used as a fitted predictor.

full rationale

The paper's only quantitative prediction, GPT-4o's laughable-context recognition, is evaluated against human majority labels in Section 4, so the F1 score is not fitted from the same labels. The taxonomy in Section 3 is explicitly generated by GPT-4o from GPT-4o-generated explanations, and the paper itself defers validation to future work; this is an external-validity limitation, not a circular derivation. Table 4's per-label breakdown uses GPT-4o-generated taxonomy labels to interpret GPT-4o's binary errors, which limits interpretability, but the accuracy numbers are still conditioned on human majority laughable labels and are not definitionally equal to any fitted input. The dataset citation (RealPersonaChat) is a resource citation, not a load-bearing self-citation of the central argument, and no uniqueness theorem or ansatz is imported from the authors' prior work. Therefore the claimed derivation chain is self-contained with respect to its human binary annotations, and no step reduces by construction to its own inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no numeric fitted parameters. It rests on three unvalidated assumptions: the reliability of the majority-vote human labels, the fidelity of GPT-4o's explanations to human reasoning, and the completeness of the ten-category taxonomy. The first could be checked with standard inter-annotator agreement metrics; the latter two would require a human validation study, which is absent.

assumptions (3)
  • domain assumption Five annotators' majority vote is a reliable ground truth for laughable contexts.
    The paper reports agreement distributions in Table 2 but no inter-annotator reliability coefficient, so label noise is unquantified.
  • ad hoc to paper GPT-4o-generated explanations accurately describe the reasons for human laughter judgments.
    Section 3 uses GPT-4o to write the reasoning text; there is no human validation of these explanations.
  • ad hoc to paper The ten-category taxonomy is a complete and non-redundant representation of laughable reasons.
    The taxonomy is produced by iterative GPT-4o clustering and is not compared to prior laughter taxonomies like Mazzocconi et al. (2020) or validated for coverage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation." pith.science (2026). https://pith.science/paper/RCUSEKT2

@misc{pith2026250116635,
  author       = {Pith},
  title        = {Pith review of: Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RCUSEKT2}},
  note         = {Machine review of arXiv:2501.16635}
}
read the original abstract

Laughter serves as a multifaceted communicative signal in human interaction, yet its identification within dialogue presents a significant challenge for conversational AI systems. This study addresses this challenge by annotating laughable contexts in Japanese spontaneous text conversation data and developing a taxonomy to classify the underlying reasons for such contexts. Initially, multiple annotators manually labeled laughable contexts using a binary decision (laughable or non-laughable). Subsequently, an LLM was used to generate explanations for the binary annotations of laughable contexts, which were then categorized into a taxonomy comprising ten categories, including "Empathy and Affinity" and "Humor and Surprise," highlighting the diverse range of laughter-inducing scenarios. The study also evaluated GPT-4o's performance in recognizing the majority labels of laughable contexts, achieving an F1 score of 43.14%. These findings contribute to the advancement of conversational AI by establishing a foundation for more nuanced recognition and generation of laughter, ultimately fostering more natural and engaging human-AI interactions.

Figures

Figures reproduced from arXiv: 2501.16635 by the authors.

Figure 1
Figure 1. Correlation matrix of assigned taxonomy la [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 30 canonical work pages

  1. [1]

    Camille Atkinson. 2015. Self-deprecation and the habit of laughter. Florida Philosophical Review, 15(1)

    work page 2015

  2. [2]

    Salvatore Attardo. 2009. Linguistic theories of humor. Walter de Gruyter

    work page 2009

  3. [3]

    Doris G Bazzini, Elizabeth R Stack, Penny D Martincin, and Carmen P Davis. 2007. The effect of reminiscing about laughter on relationship satisfaction. Motivation and Emotion, 31(1):25--34

    work page 2007

  4. [4]

    Dario Bertero and Pascale Fung. 2016. A long short-term memory framework for predicting humor in dialogues. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 130--135

    work page 2016

  5. [5]

    Gregory A Bryant and Constance M Bainbridge. 2022. Laughter and culture. Philosophical Transactions of the Royal Society B, 377(1863):20210179

    work page 2022

  6. [6]

    Akshat Choube and Mohammad Soleymani. 2020. Punchline detection using context-aware hierarchical multimodal fusion. In International Conference on Multimodal Interaction (ICMI), pages 675--679

    work page 2020

  7. [7]

    Marta Dynel. 2009. Beyond a joke: Types of conversational humour. Language and linguistics compass, 3(5):1284--1299

    work page 2009

  8. [8]

    Dana Garbarski, Nora Cate Schaeffer, and Jennifer Dykema. 2016. Interviewing practices, conversational practices, and rapport: Responsiveness and engagement in the standardized survey interview. Sociological methodology, 46(1):1--38

    work page 2016

Show all 33 references
  1. [9]

    Marc Gelkopf and Shulamith Kreitler. 1996. Is humor only fun, an alternative cure or magic? the cognitive therapeutic potential of humor. Journal of Cognitive Psychotherapy, 10(4)

    1996

  2. [10]

    Jonathan Ginzburg, Chiara Mazzocconi, and Ye Tian. 2020. Laughter as language. Glossa: a journal of general linguistics, 5(1)

    2020

  3. [11]

    Phillip Glenn. 2003. Laughter in interaction. Cambridge University Press

    2003

  4. [12]

    Jennifer Hay. 2001. The pragmatics of humor support. Walter de Gruyter

    2001

  5. [13]

    Hwang, Lillian Lee, Jeff Da, Rowan Zellers, Robert Mankoff, and Yejin Choi

    Jack Hessel, Ana Marasovic, Jena D. Hwang, Lillian Lee, Jeff Da, Rowan Zellers, Robert Mankoff, and Yejin Choi. 2023. Do androids laugh at electric sheep? humor `` understanding '' benchmarks from the new yorker caption contest. In Annual Meeting of the Association for Computa...

    2023

  6. [14]

    Koji Inoue, Divesh Lala, and Tatsuya Kawahara. 2022. Can a robot laugh with you?: Shared laughter generation for empathetic spoken dialogue. Frontiers in Robotics and AI, 9

    2022

  7. [15]

    Sophie Jentzsch and Kristian Kersting. 2023. C hat GPT is fun, but it is not funny! humor is still challenging large language models. In Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis (WASSA) , pages 325--340

    2023

  8. [16]

    Roza G Kamilo g lu, Akihiro Tanaka, Sophie K Scott, and Disa A Sauter. 2022. Perception of group membership from spontaneous and volitional laughter. Philosophical Transactions of the Royal Society B, 377(1841):20200404

    2022

  9. [17]

    Dayoon Ko, Sangho Lee, and Gunhee Kim. 2023. Can language models laugh at Y ou T ube short-form videos? In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2897--2916

    2023

  10. [18]

    Bogdan Ludusan and Petra Wagner. 2023. The effect of conversation type on entrainment: Evidence from laughter. In Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 168--174

    2023

  11. [19]

    Rod A Martin and Thomas Ford. 2018. The psychology of humor: An integrative approach. Academic press

    2018

  12. [20]

    Chiara Mazzocconi, Ye Tian, and Jonathan Ginzburg. 2020. What’s your laughter doing there? A taxonomy of the pragmatic functions of laughter. IEEE Transactions on Affective Computing, 13(3):1302--1321

    2020

  13. [21]

    Michael McCarthy and Ronald Carter. 2004. ``there's millions of them'': hyperbole in everyday conversation. Journal of pragmatics, 36(2):149--184

    2004

  14. [22]

    Neal R Norrick. 1993. Conversational joking: Humor in everyday talk

    1993

  15. [23]

    Nynaeve Perkins Booker, Michelle Cohn, and Georgia Zellou. 2024. Linguistic patterning of laughter in human-socialbot interactions. Frontiers in Communication, 9

    2024

  16. [24]

    Ye Tian, Chiara Mazzocconi, and Jonathan Ginzburg. 2016. When do we laugh? In Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 360--369

    2016

  17. [25]

    Truong and Jürgen Trouvain

    Khiet P. Truong and Jürgen Trouvain. 2012. On the acoustics of overlapping laughter in conversational speech. In INTERSPEECH, pages 851--854

    2012

  18. [26]

    u rker, Zana Bu c inca, Engin Erzin, Y \

    Bekir Berker T \"u rker, Zana Bu c inca, Engin Erzin, Y \"u cel Yemez, and T Metin Sezgin. 2017. Analysis of engagement and user experience with a laughter responsive social robot. In INTERSPEECH, pages 844--848

    2017

  19. [27]

    Julia Vettin and Dietmar Todt. 2004. Laughter in conversation: Features of occurrence and acoustic structure. Journal of Nonverbal Behavior, 28:93--115

    2004

  20. [28]

    Mengting Wan, Tara Safavi, Sujay Kumar Jauhar, Yujin Kim, Scott Counts, Jennifer Neville, Siddharth Suri, Chirag Shah, Ryen W White, Longqi Yang, Reid Andersen, Georg Buscher, Dhruv Joshi, and Nagu Rangan. 2024. TnT-LLM: Text mining at scale with large language models. In SIGK...

    2024

  21. [29]

    Kexin Wang, Carlos Ishi, and Ryoko Hayashi. 2024. A multimodal analysis of different types of laughter expression in conversational dialogues. In INTERSPEECH, pages 4673--4677

    2024

  22. [30]

    Ge Xu. 2022. An analysis of humor discourse in friends from the perspective of the cooperative principle. Open Journal of Modern Linguistics, 12(4):460--470

    2022

  23. [31]

    Sanae Yamashita, Koji Inoue, Ao Guo, Shota Mochizuki, Tatsuya Kawahara, and Ryuichiro Higashinaka. 2023. Realpersonachat: A realistic persona chat corpus with interlocutors’ own personalities. In Pacific Asia Conference on Language, Information and Computation (PACLIC), pages 852--861

    2023

  24. [32]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  25. [33]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.