On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation

Jing Ma; Wenbo Shang; Xin Huang; Yuxi Sun

REVIEW 3 major objections 4 minor 4 references

On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HOMER claims that funnier cartoon captions come from extracting conflicting scripts and building imaginative association trees before the caption is written, and that this outperforms reasoning-only baselines.

desk verdict HOMER is a solid, interpretable framework for humor captioning, but the headline win rates are only as trustworthy as GPT-5's transfer to generated captions, and the human backstop is inconsistent. read the letter →

arxiv 2602.06423 v2 pith:7ZALKQIU submitted 2026-02-06 cs.CL

Wenbo Shang , Yuxi Sun , Jing Ma , Xin Huang This is my paper · ORCID

classification cs.CL

keywords humorgenerationmultimodalcaptioningscriptoppositionGTVHlargelanguagemodelsimaginationtreesretrieval-augmentedcartooncaptions

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that large language models generate funnier cartoon captions when generation is explicitly organized around a humor theory rather than left to open-ended reasoning. It introduces HOMER, which first extracts the situation and the conflicting scripts that make an image funny, then builds imagination trees linking humor targets to surprising associations via LLM free association and joke retrieval, and finally generates a caption from that material. Across two public cartoon-caption benchmarks, HOMER reports average gains of roughly 8.6% at pass@1, 6.5% at pass@3, and 5.9% at pass@5 over seven baselines with a strong commercial base model, with consistent gains on other base models and a meme dataset. A sympathetic reader would care because the intermediate steps are interpretable: you can see which script opposition and which associative path produced a joke, something chain-of-thought approaches do not offer.

What carries the argument

The central machinery is the General Theory of Verbal Humor applied as script opposition—the idea that humor arises when two conflicting semantic frames are set against each other—plus a hierarchical imagination tree. The tree is built in two passes: deep chains from LLM free association form the backbone, and retrieval from a joke database adds leaves, with a humor-relevance score combining semantic similarity, conceptual opposition, humor frequency, and part-of-speech diversity to prune weak branches. This machinery carries the creative search: it converts a static image into a structured space of possible joke paths that the generator can sample.

What would settle it

Collect pairwise human funniness judgments on a random sample of, say, 100 HOMER captions versus the strongest baseline captions on the same images, using multiple raters; if the human win-rate margin is much smaller than the LLM-judge pass@k margin, or reverses, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that script opposition can serve as a computational scaffold for humor generation. HOMER divides caption writing into three roles: an extractor that names the situation and conflicting scripts; an imaginator that grows each humor target into an imagination tree through LLM associations and joke-corpus retrieval, pruned by a humor-relevance score; and a generator that samples a script, a target, and an associative path. The paper reports that this structure outperforms seven state-of-the-art baselines on two cartoon-caption benchmarks, that removing any one module degrades performance, and that the largest drop occurs when conflicting scripts

Load-bearing premise

The load-bearing premise is that a large language model humor judge, validated on 200 human-ranked caption pairs at about 73.5% accuracy, transfers that accuracy to ranking generated captions from a different distribution; if it does not, HOMER's reported margins may reflect judge preference rather than real funniness.

Editorial extensions

If this is right

If the claim holds, LLM cartoon captioning improves measurably just by adding theory-structured intermediate steps, without task-specific fine-tuning.
The two public benchmarks' pass@k win rates over human captions rise with HOMER, not only with strong proprietary base models but also with smaller open models.
The interpretable intermediate outputs—conflicting scripts and sampled imagination paths—allow a user to see and edit the joke logic before the caption is fixed.
Each component (situation description, conflicting scripts, imagination trees) is reported necessary; removing scripts causes the largest performance drop.
Generalization results on a meme dataset suggest the same conflicting-script recipe transfers beyond the benchmark's cartoon style.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The explicit conflicting-script representation may transfer to other constrained creative-generation tasks—puns, headlines, ad copy—where a surprising juxtaposition of two frames is the core move.
The humor-relevance pruning score could be reused as a cheap filter for joke retrieval in dialogue or recommendation systems, independent of HOMER.
A testable extension: compare HOMER against a version given the same script-opposition annotations but forced to generate linearly, to isolate whether the hierarchical tree or the script signal drives the gains.
If future human studies reproduce the reported margins, this would support the stronger claim that LLMs' humor weakness is a problem of structured imagination, not of reasoning depth.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the framework is real: GTVH script-opposition extraction, hierarchical imagination trees, and humor-relevance retrieval are integrated into a coherent pipeline, and the ablations actually support each piece. Second, the headline numbers should be read as conditional. All pass@k margins are computed by GPT-5 as the humor judge, validated only on 200 human-written pairs (73.5% accuracy). The human evaluation meant to backstop this is internally inconsistent—20 raters and 5,600 ratings in the main text, 2,800 in Appendix B.8, and 12 raters in the detailed human evaluation, with different means (3.51/3.38 vs 3.54/3.31). No generated captions or artifacts are released, so you cannot audit the judge on the actual distribution.

What is genuinely good: the design is interpretable and modular. The conflicting-script extractor gives a traceable reason for each caption; the imagination tree with retrieval is a reasonable way to inject creativity; the HOMER-pruning score is a bit ad hoc, but the ablation shows each term matters. Hyperparameters k and delta are explored, and the method is stable across the tested range. Diversity metrics and toxicity checks are included. The GPT-4.1 corroboration exists, though it is also an LLM judge with 68.5% accuracy and moderate human correlation.

Weak spots, in proportion: the evaluation chain is the load-bearing issue. If GPT-5 systematically prefers HOMER's explicit script-opposition phrasing, the reported 8.62% pass@1 improvement could be an artifact. That is not a manufactured concern; the paper does not demonstrate judge transfer to generated captions. The human evaluation inconsistencies are real and need fixing, not just cosmetic. Pass@k is reported without variance, and the significance tests run on GPT-5 scores inherit any judge bias. The theoretical analysis of H_rel in Appendix C is fine but not deep—boundedness and monotonicity, which is enough for this purpose.

Who this is for: anyone working on humor generation or creative captioning. The framework is worth citing and building on, even if the empirical margins need independent verification. I would send it to a serious referee before trusting the numbers; the design merits review, and the evaluation flaws are fixable. Recommendation: engage with the work, but ask for released outputs and a clean human evaluation on generated captions.

Referee Report

3 major / 4 minor

Summary. The paper proposes HOMER, a three-role LLM framework for multimodal humor caption generation. Grounded in the General Theory of Verbal Humor (GTVH), HOMER first extracts situation descriptions and conflicting scripts from an image, then builds hierarchical 'imagination trees' through LLM free association and retrieval from a joke database with a humor-relevance pruning score, and finally generates captions conditioned on the selected scripts, targets, and imagination paths. The method is evaluated on the Humor in AI and Electronic Sheep New Yorker Cartoon benchmarks, with GPT-5 as the primary humor judge for pass@k metrics, plus human ratings, ablations, a meme generalization experiment, and toxicity checks. The paper claims average improvements of 8.62%, 6.48%, and 5.91% over strong baselines on pass@1, pass@3, and pass@5, respectively, with GPT-4o as the base generator.

Significance. If the headline results are accepted, HOMER would be a meaningful advance in multi-modal humor generation: it is one of the few frameworks that operationalizes a specific humor theory (GTVH) in an interpretable pipeline, and its design choices are tested unusually thoroughly. Strengths include exhaustive module ablations (Table 4), hyperparameter sensitivity analyses (Figures 4-5), evaluation across four base LLMs (Table 3), a cross-domain meme experiment (Table 5), a leakage analysis (Appendix B.15), and significance tests (Appendix B.13). The paper also is candid about failure cases in Appendix D. The central reservation is that the headline pass@k numbers rest on GPT-5 as a stand-in for human humor judgment, validated only on human-written caption pairs, while the direct human evaluation is small and internally inconsistent; the load-bearing evaluator-transfer assumption is therefore not yet established.

major comments (3)

[Section 3.5 and Appendix B.8] The human evaluation numbers are irreconcilable as reported. Section 3.5 states 20 raters and 5,600 rating scores. Appendix B.8 first describes 2,800 data points (20 images × 20 raters × 7 methods), then the 'Detailed human evaluation' paragraph says 12 raters and reports HOMER means of 3.51 (Humor in AI) and 3.38 (Electronic Sheep), while Table 6 and Table 13 report 20 raters and means of 3.54±0.59 and 3.31±0.85. These are different experiments or errors; as written, the only direct human backstop for the GPT-5 judge cannot be used to verify the claimed margins. This must be corrected and the actual protocol, rater count, and per-method scores reported consistently.
[Section 3.1, Section 3.2, Appendix B.14] GPT-5 is validated only on 200 human-written caption pairs (73.5% and 70.0% ranking accuracy in Table 2), but the main pass@k results in Table 3 use GPT-5 to compare model-generated captions against human-written captions. No validation on the generated-caption distribution is provided. The GPT-4.1 corroboration (68.5% accuracy, Pearson 0.5639 against human ratings) is another LLM judge and does not resolve the transfer concern. If GPT-5 systematically prefers HOMER's explicit script-opposition style, the reported improvements could be an artifact. The paper should either report judge accuracy/correlation on a sample of generated captions labeled by humans, or release the generated outputs so the judge's behavior can be independently audited.
[Table 3 and Appendix B.13] The headline pass@k results are means over five runs with no standard deviations, confidence intervals, or per-run distributions. Generation uses temperature 1.0 and random selection of scripts/paths, so run-to-run variability is expected. Several margins over the strongest baseline are small (e.g., Claude-4 rows: +1.25 and +1.26 on #200-209 groups), and the reported Wilcoxon tests are pairwise tests on aggregate pass@k values without correction. Please report the five per-run scores (or confidence intervals) and make the significance testing procedure explicit; otherwise the 1-3 point margins are hard to distinguish from sampling noise.

minor comments (4)

[Table 1] The 'Avg #captions' entry for Humor in AI is shown as '6,044', which is implausible as an average per cartoon. Please clarify whether this is a total, a misprint, or a different statistic.
[Eq. (7) and Appendix B.3] The notation for n_i and c_i should be tightened. The main text leaves n_i implicit until Appendix B.3 (where it is set to 5); state this in the main text and explain the interpretation of pass@5 when n_i=5, since pass@5 then reduces to the empirical win-rate over the five generated captions.
[Section 2.2 and Appendix C] The text after Eq. (4) promises a 'detailed proof of convergence and monotonicity' in Appendix C, but Appendix C contains only informal boundedness and monotonicity remarks; no formal proof or theorem statement is given. Either supply a precise statement/proof or soften the claim.
[Section 3.6 and Figure 7] The text reports the summed toxicity as 0.023 and 0.015, while Figure 7's axis is labeled 'Sum of toxicity (%)' with values 2.35 and 1.52. Make the units consistent (proportion vs. percentage) so the numbers can be compared directly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: HOMER's generation pipeline is self-contained, and the GPT-5 evaluator is an external judge rather than a fitted predictor.

full rationale

Walking the claimed derivation chain—GTVH-driven script extraction (Extract(I)→(C,D)), hierarchical imagination with retrieval and humor-relevance pruning (H(e)=H_rel+H_freq+H_div, Eqs. 1–6), and prompt-conditioned generation (Cap(I)=Gen(Φ(C,D,T_im,Ω)))—no step is defined in terms of the quantity it predicts. The script extractor is prompted with GTVH's definition of script opposition but does not use the test captions or GPT-5 labels; the humor-relevance score uses WordNet similarity, joke-corpus frequency, and POS diversity, all external to the target outputs; the generator samples associations from retrieval rather than being trained on or fitted to the humor-judgment labels. The main pass@k numbers are produced by GPT-5, whose selection is justified by ranking accuracy on 200 human-written caption pairs in Table 2/Sec. 3.1; this is an evaluator-choice step, not a fitted parameter, and HOMER is not optimized toward GPT-5's preferences. Human ratings are used only as a backstop; their internal inconsistency (20 vs. 12 raters; 5600 vs. 2800 ratings; Sec. 3.5 vs. App. B.8) is a serious reliability limitation, but it is not a definitional reduction of the method's output to its input. Self-citations (Shang et al. 2021/2022; Shang & Huang 2025) appear in related work and as one of several supporting citations for GTVH; none is load-bearing or invoked to force a modeling choice. No specific equation or fitted value can be exhibited as equivalent to the claimed prediction, so no circular step is found.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical or conceptual entities requiring independent evidence. Imagination trees and conflicting scripts are abstract structures, not entities with external falsifiable handles.

free parameters (2)

k (top-K retrieved jokes) = 5
Selected by sweeping [3,5,7,9] on the evaluation benchmarks in Section 3.3 / Appendix B.4, not by held-out validation.
delta (rank threshold for humor-relevance pruning) = 5
Selected by sweeping [3,5,7,9] on the evaluation benchmarks in Section 3.3 / Appendix B.4, not by held-out validation.

assumptions (4)

domain assumption GTVH script opposition is a sufficient organizing principle for image-humor caption generation.
The entire framework builds on this, operationalized through LLM prompts without independent validation of the mapping from images to scripts.
domain assumption GPT-5 humor rankings are a valid proxy for human funniness in pass@k.
Section 3.1 justifies this with 73.5%/70% ranking accuracy on 200 human caption pairs; this is assumed to transfer to generated captions.
domain assumption The WordNet-based relevance-opposition score H captures humorous relevance between entities.
Equations 2-4 define H from TSS and CO; it is validated only through ablations on the same benchmarks.
domain assumption The joke retrieval database has no leakage with the test captions.
Appendix B.15 claims zero overlap via exact-match and normalized comparisons, but the method is underspecified and no data/code is shipped.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation." pith.science (2026). https://pith.science/paper/7ZALKQIU

@misc{pith2026260206423,
  author       = {Pith},
  title        = {Pith review of: On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7ZALKQIU}},
  note         = {Machine review of arXiv:2602.06423}
}

read the original abstract

Humor is a commonly used and intricate human language in daily life. Humor generation, especially in multi-modal scenarios, is a challenging task for large language models (LLMs), which is typically as funny caption generation for images, requiring visual understanding, humor reasoning, creative imagination, and so on. Existing LLM-based approaches rely on reasoning chains or self-improvement, which suffer from limited creativity and interpretability. To address these bottlenecks, we develop a novel LLM-based humor generation mechanism based on a fundamental humor theory, GTVH. To produce funny and script-opposite captions, we introduce a humor-theory-driven multi-role LLM collaboration framework augmented with humor retrieval (HOMER). The framework consists of three LLM-based roles: (1) conflicting-script extractor that grounds humor in key script oppositions, forming the basis of caption generation; (2) retrieval-augmented hierarchical imaginator that identifies key humor targets and expands the creative space of them through diverse associations structured as imagination trees; and (3) caption generator that produces funny and diverse captions conditioned on the obtained knowledge. Extensive experiments on two New Yorker Cartoon benchmarking datasets show that HOMER outperforms state-of-the-art baselines and powerful LLM reasoning strategies on multi-modal humor captioning.

Figures

Figures reproduced from arXiv: 2602.06423 by the authors.

**Figure 2.** Framework of HOMER with three LLM-based roles: (a) Conflicting script extractor, de [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Ablation study of humor-relevance score. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (10 more)

**Figure 5.** Figure 5: δ hyperparameter. effectiveness of modeling semantic relevance and conceptual opposition. The w/o frequency and w/o diversity variants also show a great drop, indicating that they are useful for exploring imagination. Robustness of hyperparameters. We conduct an ablati…

**Figure 6.** Figure 6: Case Study. We show two cases in [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 7.** Figure 7: Harmful detection. We evaluate harmful content in HOMER’s generated captions using Detoxify(Hanu & Unitary team, 2020), a widely used toxicity detector, across seven dimensions: toxicity, severe toxicity, obscene, identity attack, insult, threat, and sexual explicit,…

**Figure 8.** Figure 8: Robustness of the LLM temperature [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

**Figure 9.** Figure 9: Example of human evaluation. Detailed human evaluation. The results of our human evaluation, summarized in [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

**Figure 11.** Figure 11: Specifically, harmfulness is assessed across seven dimensions: toxicity, severe toxicity, obscene language, identity attack, insult, threat, and sexual explicitness. This evaluation is performed on two datasets, Humor in AI and Electronic Sheep, and covers captions p…

**Figure 11.** Figure 11: Harmful detection of three base models. three API calls at the imaginator stage, reflecting real-time imagination processing. This comparison highlights the efficiency gained by pre-processing imagination results in the naive model [PITH_FULL_IMAGE:figures/full_fig_p…

**Figure 12.** Figure 12: Example Prompt for Conflict Script Extractor [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]

**Figure 13.** Figure 13: Example Prompt for Hierarchical Imaginator [PITH_FULL_IMAGE:figures/full_fig_p028_13.png]

**Figure 14.** Figure 14: Example Prompt for Caption Generator SYSTEM Using the provided free-association chains, conflict scripts, and the cartoon description, generate a witty, funny and smart caption that spotlights the central incongruity and naturally combines key keywords in chains. Cons…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

4 extracted references · 1 linked inside Pith

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

@esa (Ref

\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...
[3]

\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...
[4]

[?+ F sF' Fp0 gf !y mtB ,`m g+j : * W4iuT7D5 eFk iWI 8iPZ.mj g : q X>QEX>) 2'F *9dlT4 ׋

@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

arXiv 1976

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

@esa (Ref

\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

[3] [3]

\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

[4] [4]

[?+ F sF' Fp0 gf !y mtB ,`m g+j : * W4iuT7D5 eFk iWI 8iPZ.mj g : q X>QEX>) 2'F *9dlT4 ׋

@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

arXiv 1976