Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that current multimodal models fail to decode cross-concept idiom figures, with the strongest model reaching only 50.7% and answer-space search as the main bottleneck.

desk verdict A solid benchmark for evaluating cross-concept decoding in MLLMs, but the RQ2 'search bottleneck' claim needs an image-ablated control before it can be taken at face value. read the letter →

arxiv 2608.06501 v1 pith:J4XXOOEO submitted 2026-08-06 cs.AI cs.CLcs.MM

Ming Wang , Yuqing Zhang , Tingna Xie , Xiangju Li , Xiaocui Yang , Daling Wang , Shi Feng , Yifei Zhang This is my paper · ORCID
classification cs.AIcs.CLcs.MM
keywords multimodallargelanguagemodelscross-conceptunderstandingchengyubenchmarkcreativecognitionMLLMevaluationcreativitydecodingbottleneck
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces C4, a benchmark framework that treats creative understanding as two directed operations: encoding, where a target idiom is replaced by imageable substitutes along annotated associative bridges, and decoding, where a model must reverse those bridges to recover the idiom. It instantiates C4-Eval with 221 items and evaluates ten multimodal models. The central empirical claim is that current models decode these figures poorly: the best closed model reaches 50.7% primary accuracy, open models stay below 18.1%, and offering a small candidate set lifts accuracy far more than giving a general bridge hint or asking for an explanation. The paper concludes that open answer-space search, not visual perception or relation inference, is the main bottleneck, and that replacing more idiom slots creates more difficulty than deepening the bridges. If right, C4 gives creative understanding an exact-answer, measurable form and points to open-vocabulary retrieval as the next problem.

What carries the argument

The load-bearing object is a manually annotated chengyu-oriented cross-concept network: 47 target idioms, 168 anchored slots, and 758 deduplicated bridge chains built by two independent annotators and reviewed by a third. A bridge chain is a sequence of associative steps, such as phonetic, lexical, semantic, object, role, part-whole, or cultural relations, from an imageable landing concept back to a character slot of the target idiom. Encoding follows chains forward from slot to substitute, and decoding must follow them in reverse; item difficulty is indexed by the number of replaced slots and the multiset of bridge depths (levels L1-L4). Five task views, image-only recovery, bridge-hint recovery, four-way candidate recognition, free explanation, and known-answer explanation, separate perception, relation inference, and open answer-space search. Because latent structure is known by construction, a score gap between candidate recognition and open recovery can be attributed to answer-space search rather than to missing visual cues.

What would settle it

Rerun the image-only task with the same landing concepts rendered by a different image generator or by human illustration: if primary accuracy moves by more than ten percentage points, the rendering pipeline, not cross-concept decoding, is carrying the measured difficulty. Likewise, give the same images and scoring to fluent Chinese speakers: if they fail to recover the annotated targets, the bridge paths are not recoverable as intended.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that cross-concept understanding can be operationalized as exact-answer decoding: each item fixes a four-character chengyu, replaces one or two character slots with imageable substitutes reached through reviewed associative bridge paths, and asks the model to reverse the paths. Across ten MLLMs and 884 answer-bearing cases, the strongest closed model scores 50.7% primary accuracy and the best open model 18.1%. Candidate-set recognition exceeds the mean of the three open-recovery settings by 17.3 to 56.0 percentage points, while bridge hints and explanation requests add little, which the paper reads as evidence that models often possess enough signal to recognize the answer but cannot retrieve it from the open space. Level-wise, adding a second replaced slot produces roughly three times the accuracy drop of deepening one bridge path, and 83 of 221 items are solved by no model in any open setting, with 6 unsolved even under candidates. The paper's conclusion is that current MLLMs decode creatively encoded meaning by consuming visible substitutes literally rather than traversing their cross-concept relations back to the target.

Load-bearing premise

The load-bearing premise, stated in Sections 3.1, 3.2, and Appendix A, is that the manually annotated bridge paths are the intended and recoverable relations and that the automatically generated images actually render the substituted landing concepts without leaking the target; if either fails, the accuracy figures measure annotation artifacts or perception errors rather than cross-concept decoding.

Editorial extensions

If this is right

  • If the measured gap is real, the C4 leaderboard is far from saturated, leaving headroom for decoding improvements rather than a benchmark ceiling.
  • Because candidate constraints lift accuracy by 17.3 to 56.0 points, models often hold enough evidence to recognize the right idiom in a small set, so improving open-vocabulary retrieval is a concrete direction.
  • Slot count dominates bridge depth: two-slot encodings account for most of the difficulty, so evaluation and training should concentrate on integrating multiple substituted characters.
  • Asking for an explanation changes which items are solved instead of raising overall accuracy, and correct answers or valid JSON do not imply faithful reconstruction of the annotated bridge paths.
  • Fully unsolved items are decoding failures rather than perception gaps, since models name the visible substitutes correctly but consume them literally instead of reversing their bridges.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the encoding/decoding design should transfer to other fixed-form cultural expressions such as proverbs, two-part allegorical sayings, rebuses, or memes, because it only needs exact targets, imageable substitutes, and reviewed bridges.
  • Editorial inference: the large candidate lift predicts a measurable retrieval gradient, so running T3 with candidate sets of intermediate size between the full 84-answer inventory and the 4-way set should raise accuracy monotonically if open search is the bottleneck; that is a direct test a follow-up run could perform.
  • Editorial inference: the annotated relation families enable a controlled ablation in which matched items differ only in whether their bridges are phonetic or semantic, testing which relation type is hardest to reverse; the current four difficulty levels do not separate relation families.
  • Editorial inference: a human-decoder baseline on the same 221 images would calibrate benchmark difficulty and separate model limitations from items that are underdetermined by construction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces C4, a benchmark for evaluating multimodal large language models (MLLMs) on cross-concept understanding using Chinese chengyu idioms. Items are constructed by replacing target idiom slots with imageable substitute concepts along manually annotated bridge paths, which provides exact answers and explicit difficulty levels defined by bridge count and depth. The benchmark contains 221 items instantiated in five task settings, yielding 884 primary answer-bearing cases. Ten MLLMs are evaluated under a fixed protocol; the strongest closed model reaches 50.7% primary accuracy, the best open model reaches 18.1%, candidate recognition is much higher than open recovery, bridge hints and explanation prompts give small gains, and slot count affects difficulty more than bridge depth. The paper concludes that open answer-space search, rather than visual perception or relation inference, is the dominant decoding bottleneck.

Significance. If the central claims hold, C4 is a useful structured benchmark for creative decoding: it offers exact scoring, inspectable relation paths, a clean separation of task forms (T1–T5), and a documented construction pipeline with planned code and data release. The benchmark-level observation that candidate constraints sharply improve MLLM accuracy is a reproducible finding independent of the bottleneck interpretation. However, the paper's main interpretive claim about the location of the decoding bottleneck is currently over-reached relative to the experimental design: the candidate-recognition condition (T3) is confounded with lexical/prior cues and a 25% chance baseline, and no image-ablated control or human baseline is provided. The benchmark itself is a solid contribution; the bottleneck conclusion needs additional evidence.

major comments (4)
  1. [4.3, Figure 4, Appendix E] The conclusion that 'open answer-space search is the dominant bottleneck' is not established because T3 is not a clean manipulation of search-space size. T3 supplies a four-way candidate set drawn from the 84-answer inventory with distractors deliberately ranked by character overlap with the gold target, and it has a 25% chance baseline. Correct T3 answers can therefore be produced by lexical-overlap heuristics or prior popularity without using the image or inferring relations. The claim that T3 'keeps perception and relation inference unchanged' is incorrect: the candidate strings themselves carry information. The authors should report a no-image or mismatched-image T3 control and a chance-subtracted lift. For the weakest models the current data do not even show positive recognition: Qwen3.5-9B's T3 accuracy is 22.2%, below the 25% chance level, so the subgroup claim that visual perception is intact is especially fragile.
  2. [4.3] The sentence 'Visual perception is not the limiting stage for most items, since invisible cues could not let a collapsed answer space raise accuracy by tens of points' presupposes the point at issue. Without an image-ablated T3 condition, one cannot determine whether the T3 lift is driven by preserved visual evidence or by candidate-set surface cues. The authors should add a control such as T3 with a randomly paired image, or an isolated object-recognition probe on the same images, and report human performance on at least T1 and T3 to calibrate what 'solved' means. This is a load-bearing point for RQ2 and for the paper's central claim.
  3. [3.2, Appendix D] The ground-truth validity of the generated images is asserted but not verified. Scene descriptions were manually reviewed to prevent leakage, but there is no evidence that GPT Image 2 actually renders the intended substitute concepts in most items. If images fail to depict the landing concepts, accuracy scores measure perception failures or rendering artifacts rather than cross-concept decoding. The RQ2 conclusion depends on the assumption that visual cues are present and recognizable, so the authors should include a verification study (e.g., human annotation or an independent detector on a sample) and report agreement. A small human sanity check would substantially strengthen the benchmark's construct validity.
  4. [4.1, Appendix F] All reported results are single-pass point estimates with temperature set to zero, no repeated runs, no confidence intervals, and no human baseline. Given the relatively small N (221 items, 884 primary cases) and the paper's comparative claims (e.g., 'open models remain substantially lower', 'candidate lift 17.3–56.0 points'), the authors should report binomial confidence intervals or repeated passes, and include a human baseline on at least T1 and T3. Without this, the strength of the claims about the 'substantial gap' and the bottleneck is hard to calibrate.
minor comments (5)
  1. [Appendix G, Table S5] The T5 column in Table S5 reports 'exact recovery of its supplied answer field'; since T5 asks for a rationale rather than an answer, clarify what is scored in this column and why it is reported alongside the primary tasks.
  2. [Figure 4] Figure 4 would be more informative with a dashed line at the 25% chance level and error bars; the current caption does not state that the chance baseline is not subtracted from the reported lift.
  3. [4.4] The claim that the L1-to-L2 slot transition produces 'roughly three times' the mean accuracy drop of the L2-to-L3 depth transition is not supported by any table or per-level numbers in the main text; add the underlying numbers or a reference to the appendix.
  4. [Equation (3)] The notation M_i = {1, >=2} and M_i = {>=2, >=2} is informal for a multiset; define what '>=2' denotes (e.g., each element is at least 2) to avoid ambiguity.
  5. [Figure 3] The L3 point is placed in a column labeled '1 mixed'; this axis label is unclear. Consider a clearer coordinate annotation that distinguishes the two paths of different depths.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: C4-Eval scores are external measurements; difficulty levels are explicit construction parameters, not fitted or self-referential outputs.

full rationale

The paper's load-bearing claims are accuracy measurements of ten MLLMs on a fixed, manually annotated benchmark. No parameter is fitted to the target result: the primary score in Equation 7 is simply the average of exact-match indicators over T1–T4, and the task definitions in Equation 5 vary only the information supplied to the model. The RQ2 bottleneck conclusion is an empirical inference from the observed gap between T3 candidate recognition and the open settings; even if that inference is confounded by the 25% chance baseline or by T3 distractors ranked on character overlap (Appendix E), a confound is an experimental-validity concern, not a circular reduction. RQ3 compares accuracy across L1–L4, which are defined by bridge count and depth in Equation 3, but the paper explicitly disclaims that these labels assert an independent psychological scale: Appendix A states the L1–L4 levels 'describe construction parameters. They do not assert an intrinsic psychological scale independent of targets and images,' and Section 4.4 cautions that adjacent level comparisons 'describe associations with the annotated encoding rather than controlled causal effects.' Measuring accuracy over independently defined construction levels is therefore an empirical finding, not a tautology. There are no load-bearing self-citations, no fitted inputs renamed as predictions, and no prior-work uniqueness theorem imported to force a choice. The paper's explicit limitation statements further reduce any appearance of circularity rather than hiding it.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim rests on assumptions about the validity of the manually constructed bridge paths, the fidelity of generated images, and the benchmark's ability to isolate cross-concept decoding. No numeric parameters were fitted to the reported accuracies; the only hand-chosen elements are the difficulty thresholds and annotation decisions, which the paper explicitly labels as construction parameters.

free parameters (1)
  • Difficulty level thresholds (L1-L4) = L1: 1 slot, depth 1; L2: 2 slots, depths 1,1; L3: 2 slots, depths 1 and >=2; L4: 2 slots, depths >=2,>=2
    Hand-chosen boundaries for synthetic difficulty. They are construction parameters that define the levels; the paper explicitly notes they do not assert an intrinsic psychological scale.
assumptions (5)
  • domain assumption Chengyu have fixed written forms that yield exact answer strings.
    Section 3.5 and scoring use exact match against canonical chengyu strings; this assumes the fixed form is unambiguous.
  • domain assumption The manually annotated and reviewed bridge paths correctly capture the intended cross-concept relations for each image.
    Section 3.1 describes independent double annotation and third-party review, but the validity of the ground truth is not independently verifiable from the paper alone.
  • domain assumption Generated images faithfully render the substitute landing concepts described in the scene prompts.
    Section 3.2 states images were generated with GPT Image 2 and prompts were reviewed to prevent leakage, but there is no systematic check that every rendered image depicts the intended substitutes.
  • domain assumption The visible concepts in each image differ from a literal depiction of the target chengyu.
    Equation 4 imposes V(I_i) does not imply literal y_i; this is enforced by prompt review but is an assumption about the final images.
  • domain assumption Model accuracy on these items is a meaningful measure of cross-concept decoding ability.
    This is the benchmark validity assumption; other factors like image understanding, language, and cultural knowledge also affect scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding." pith.science (2026). https://pith.science/paper/J4XXOOEO

@misc{pith2026260806501,
  author       = {Pith},
  title        = {Pith review of: Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J4XXOOEO}},
  note         = {Machine review of arXiv:2608.06501}
}
read the original abstract

Creative capabilities of MLLMs matter in design, communication, education, and human--AI collaboration, yet remain difficult to evaluate because explicit targets and reward signals are scarce compared with accuracy-oriented tasks. Cross-concept understanding is a core cognitive capacity underlying receptive creativity. It enables a perceiver to recover intended meaning from non-obvious but meaningful conceptual relations. We operationalize item construction as cross-concept encoding and model inference as cross-concept decoding. We introduce C4, a cognition-inspired evaluation framework for Chengyu (Chinese idiom)-based Cross-Concept Creativity. Its encoding component maps target slots to imageable substitute concepts along bridge paths in a manually annotated and third-party-reviewed cross-concept network, enabling batch generation with explicit structure, difficulty indexed by bridge count and depth, and exact answers. Using this framework, we instantiate the C4 Evaluation Set (C4-Eval), comprising 184 synthetic items and 37 human-created cross-concept chengyu figures collected from online sources. We manually construct and review cross-concept relations, bridge paths, and reasoning processes for the collected figures. Each C4-Eval item is instantiated in five task settings, yielding 884 primary answer-recovery cases. Across ten evaluated MLLMs, the strongest closed models reach 50.7% and 48.0% primary accuracy, while open-source models remain substantially lower. Candidate constraints improve accuracy sharply, but bridge hints and explanation requests provide only modest gains. These results expose a substantial gap in how current MLLMs decode creatively encoded meaning through cross-concept relations. The code is in the supplementary material.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 10 canonical work pages

  1. [1]

    Asurveyonmultimodal large language models.National Science Review, 11(12):nwae403, 2024

    ShukangYin,ChaoyouFu,SiruiZhao,KeLi,XingSun,TongXu,andEnhongChen. Asurveyonmultimodal large language models.National Science Review, 11(12):nwae403, 2024. doi: 10.1093/nsr/nwae403

    doi:10.1093/nsr/nwae403 2024

  2. [2]

    Generative artificial intelligence, human creativity, and art.PNAS Nexus, 3(3): pgae052, 2024

    Eric Zhou and Dokyun Lee. Generative artificial intelligence, human creativity, and art.PNAS Nexus, 3(3): pgae052, 2024. doi: 10.1093/pnasnexus/pgae052

    doi:10.1093/pnasnexus/pgae052 2024

  3. [3]

    Doshi and Oliver P

    Anil R. Doshi and Oliver P. Hauser. Generative AI enhances individual creativity but reduces the collective diversity of novel content.Science Advances, 10(28):eadn5290, 2024. doi: 10.1126/sciadv.adn5290. 12 KinaMind C4 for Cross-Concept Understanding

    doi:10.1126/sciadv.adn5290 2024

  4. [4]

    A causality-awareparadigmforevaluatingcreativityofmultimodallargelanguagemodels.IEEETransactions onPatternAnalysisandMachineIntelligence,47(5):3830–3846,2025

    Zhongzhan Huang, Shanshan Zhong, Pan Zhou, Shanghua Gao, Marinka Zitnik, and Liang Lin. A causality-awareparadigmforevaluatingcreativityofmultimodallargelanguagemodels.IEEETransactions onPatternAnalysisandMachineIntelligence,47(5):3830–3846,2025. doi: 10.1109/TPAMI.2025.3539433

    work page arXiv 2025

  5. [5]

    Art or artifice? large language models and the false promise of creativity

    Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. Art or artifice? large language models and the false promise of creativity. InProceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–34, 2024. doi: 10.1145/3613904.3642731

    arXiv 2024

  6. [6]

    Runco and Garrett J

    Mark A. Runco and Garrett J. Jaeger. The standard definition of creativity.Creativity Research Journal, 24 (1):92–96, 2012. doi: 10.1080/10400419.2012.650092

    arXiv 2012

  7. [7]

    Finke, Thomas B

    Ronald A. Finke, Thomas B. Ward, and Steven M. Smith.Creative Cognition: Theory, Research, and Applications. The MIT Press, 1992. ISBN 9780262272650. doi: 10.7551/mitpress/7722.001.0001

    doi:10.7551/mitpress/7722.001.0001 1992

  8. [8]

    The associative basis of the creative process.Psychological Review, 69(3):220–232,

    Sarnoff Mednick. The associative basis of the creative process.Psychological Review, 69(3):220–232,

    work page

Show all 41 references
  1. [9]

    Structure-mapping: A theoretical framework for analogy.Cognitive Science, 7(2):155–170,

    Dedre Gentner. Structure-mapping: A theoretical framework for analogy.Cognitive Science, 7(2):155–170,

  2. [10]

    Conceptual integration networks.Cognitive Science, 22(2):133–187,

    Gilles Fauconnier and Mark Turner. Conceptual integration networks.Cognitive Science, 22(2):133–187,

  3. [11]

    Lawrence Zitnick, and Devi Parikh

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. VQA: Visual question answering. InProceedings of the IEEE International Conference on Computer Vision, pages 2425–2433, 2015. doi: 10.1109/ICCV.2015.279

    2015 doi

  4. [12]

    Making the V in VQA matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6325–6334, 201...

    2017 doi

  5. [13]

    Lawrence Zitnick, and Ross Girshick

    Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogniti...

    1988 doi

  6. [14]

    Hudson and Christopher D

    Drew A. Hudson and Christopher D. Manning. GQA: A new dataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6693–6702, 2019. doi: 10.1109/CVPR.2019.00686

    2019

  7. [15]

    OK-VQA: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. OK-VQA: A visual question answering benchmark requiring external knowledge. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3190–3199, 2019. doi: 10.1109/CVPR.2019.00331

    2019

  8. [16]

    A corpus for reasoning about natural language grounded in photographs

    Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. A corpus for reasoning about natural language grounded in photographs. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6418–6428, 2019. doi: 10.1865...

    2019 doi

  9. [17]

    Winoground:Probingvisionandlanguagemodelsforvisio-linguisticcompositionality

    Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground:Probingvisionandlanguagemodelsforvisio-linguisticcompositionality. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 522...

    2022

  10. [18]

    MME: A comprehensive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394, 2025

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, and Ran He. MME: A comprehensive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.133...

    2025 doi arXiv

  11. [19]

    MM-Vet: Evaluating large multimodal models for integrated capabilities

    WeihaoYu,ZhengyuanYang,LinjieLi,JianfengWang,KevinLin,ZichengLiu,XinchaoWang,andLijuan Wang. MM-Vet: Evaluating large multimodal models for integrated capabilities. InProceedings of the International Conference on Machine Learning, 2024. doi: 10.48550/arXiv.2308.02490

    2024 doi arXiv

  12. [20]

    MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MMMU: A m...

    2024 doi arXiv

  13. [21]

    CMMMU: A chinese massive multi-discipline multimodal understanding benchmark.arXiv preprint arXiv:2401.11944,

    Ge Zhang, Xinrun Du, Bei Chen, Yiming Liang, Tongxu Luo, Tianyu Zheng, Kang Zhu, Yuyang Cheng, Chunpu Xu, Shuyue Guo, Haoran Zhang, Xingwei Qu, Junjie Wang, Ruibin Yuan, Yizhi Li, Zekun Wang, Yudong Liu, Yu-Hsuan Tsai, Fengji Zhang, Chenghua Lin, Wenhao Huang, and Jie Fu. CMMM...

    arXiv

  14. [22]

    doi: 10.48550/arXiv.2401.05604

    AndrewGritsevskiy,ArjunPanickssery,AaronKirtland,DerikKauffman,HansGundlach,IrinaGritsevskaya, JoeCavanagh,JonathanChiang,LydiaLaRoux,andMichelleHung.REBUS:Arobustevaluationbenchmark of understanding symbols.arXiv preprint arXiv:2401.05604, 2024. doi: 10.48550/arXiv.2401.05604

    2024 doi arXiv

  15. [23]

    Beaty, Mathias Benedek, Paul J

    Roger E. Beaty, Mathias Benedek, Paul J. Silvia, and Daniel L. Schacter. Creative cognition and brain network dynamics.Trends in Cognitive Sciences, 20(2):87–95, 2016. doi: 10.1016/j.tics.2015.10.004

    2016 doi

  16. [24]

    Divergent creativity in humans and large language models.Scientific Reports, 16(1), 2026

    AntoineBellemare-Pepin,FrançoisLespinasse,PhilippThölke,YannHarel,KoryMathewson,JayA.Olson, Yoshua Bengio, and Karim Jerbi. Divergent creativity in humans and large language models.Scientific Reports, 16(1), 2026. doi: 10.1038/s41598-025-25157-3

    2026 doi

  17. [25]

    BRAINTEASER: Lateral thinking puzzles for large language models

    Yifan Jiang, Filip Ilievski, Kaixin Ma, and Zhivar Sourati. BRAINTEASER: Lateral thinking puzzles for large language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14317–14332, 2023. doi: 10.18653/v1/2023.emnlp-main.885

    2023 doi

  18. [26]

    LatEval: An interactive LLMs evaluation benchmark with incomplete information from lateral thinking puzzles

    Shulin Huang, Shirong Ma, Yinghui Li, Mengzuo Huang, Wuhe Zou, Weidong Zhang, and Haitao Zheng. LatEval: An interactive LLMs evaluation benchmark with incomplete information from lateral thinking puzzles. InProceedingsofthe2024JointInternationalConferenceonComputationalLinguis...

    2024 doi

  19. [27]

    HumorinAI:Massivescale crowd-sourced preferences and benchmarks for cartoon captioning

    Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert,TimothyRogers,KevinJamieson,RobertMankoff,andRobertNowak. HumorinAI:Massivescale crowd-sourced preferences and benchmarks for cartoon captioning. InAdvances in Neural I...

    2024 doi

  20. [28]

    Creation-MMBench: Assessing context-aware creative intelligence in MLLM

    Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen, and Dahua Lin. Creation-MMBench: Assessing context-aware creative intelligence in MLLM. InProceedings of the IEEE/CVF Inte...

    2025

  21. [29]

    OpenAI GPT-5 System Card.arXiv preprint arXiv:2601.03267, 2025

    OpenAI. OpenAI GPT-5 System Card.arXiv preprint arXiv:2601.03267, 2025. doi: 10.48550/arXiv.2601. 03267

    2025 doi arXiv

  22. [30]

    Kimi K2.5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026

    Kimi Team. Kimi K2.5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026. doi: 10.48550/arXiv.2602.02276

    2026 doi arXiv

  23. [31]

    MiMo-V2-Flash technical report.arXiv preprint arXiv:2601.02780, 2026

    Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, et al. MiMo-V2-Flash technical report.arXiv preprint arXiv:2601.02780, 2026. doi: 10.48550/arXiv.2601.02780

    2026 doi arXiv

  24. [32]

    GLM-4.5V and GLM-4.1V-Thinking: Towards versatile multimodal reasoning with scalable reinforcement learning.arXiv preprint arXiv:2507.01006,

    Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, et al. GLM-4.5V and GLM-4.1V-Thinking: Towards versatile multimodal reasoning with scalable reinforcement learning.arXiv preprint arXiv:2507.01006,

    arXiv

  25. [33]

    InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, et al. InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025. doi: 10.48550/arXiv.2508.18265

    2025 doi arXiv

  26. [34]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025. doi: 10.48550/arXiv.2503.19786

    2025 doi arXiv

  27. [35]

    NVIDIA Nemotron Nano V2 VL.arXiv preprint arXiv:2511.03929, 2025

    NVIDIA, Amala Sanjay Deshmukh, Kateryna Chumachenko, Tuomas Rintamaki, et al. NVIDIA Nemotron Nano V2 VL.arXiv preprint arXiv:2511.03929, 2025. doi: 10.48550/arXiv.2511.03929

    2025 doi

  28. [36]

    call a deer a horse

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, et al. Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025. doi: 10.48550/arXiv.2511.21631. 14 KinaMind C4 for Cross-Concept Understanding A Theoretical Scope and Operational Assumptions The theoretical contribution o...

    2025 doi arXiv

  29. [1962]

    doi: 10.1037/h0048850

    doi

  30. [1983]

    doi: 10.1016/S0364-0213(83)80009-3

    doi

  31. [1998]

    doi: 10.1207/s15516709cog2202_1

    doi

  32. [2024]

    doi: 10.48550/arXiv.2401.11944

    doi arXiv

  33. [2025]

    doi: 10.48550/arXiv.2507.01006

    doi arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.