Pith. sign in

REVIEW 4 major objections 7 minor 86 references

GuessBench: Sensemaking Multimodal Creativity in the Wild

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces GuessBench, a 2,000-problem benchmark built from real Minecraft 'Guess the Build' games, and claims that even the strongest tested vision-language model fails on about a third of attempts to say what a player built.

desk verdict GuessBench is a genuine new benchmark idea with mostly transparent execution, but the synthetic letter-mask hints and missing human baseline make the 'creativity sensemaking' label provisional, and the 15.36% transfer number is oversold. read the letter →

arxiv 2506.00814 v2 pith:75WO6TYM submitted 2025-06-01 cs.CL

Zifeng Zhu , Shangbin Feng , Herun Wan , Ningnan Wang , Minnan Luo , Yulia Tsvetkov This is my paper
classification cs.CL
keywords GuessBenchvision-languagemodelscreativityevaluationMinecraftmultimodalbenchmarksensemakingconceptfrequencymultilingualrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to prove that 'creativity in the wild'—the imperfect, everyday creativity of ordinary people rather than trained artists—can be measured by asking vision-language models to guess what a player's Minecraft build is meant to represent. It curates 500 real in-game builds into 2,000 static and dynamic problems, where both the image and the hint become progressively more complete. The central empirical claim is that current models largely fail this task: the strongest tested model reaches 57.8% accuracy in the static setting and 66.0% on the final dynamic attempt, while open models average only about 14% on that attempt. The paper argues these failures reveal a real capability gap, and supports that reading by showing that success tracks concept frequency in training data, drops for low-resource languages and non-Western cultural symbols, and is not fixed by standard reasoning strategies. It further claims the benchmark is useful for training, since fine-tuning on its reasoning traces improves downstream visual-perception tasks by 15.36% on average.

What carries the argument

The load-bearing object is the GuessBench problem set: 500 curated build sets, each providing three progressively complete screenshots of the same Minecraft construction and a synthetic hint that encodes word lengths and letter positions. On the first dynamic attempt the hint shows only the answer's word/letter structure; later attempts reveal $\lceil N/8 \rceil$ and then $\lceil N/4 \rceil$ randomly selected letters for an $N$-letter answer, with the static setting using the final image and hint from this progression. This design turns an open-ended interpretive act into a repeatable measurement: models must integrate visual evidence with partial text across attempts, and correctness is scored by extracting the guessed concept from the model's free-form response. The same response traces serve as fine-tuning data, which is the mechanism behind the reported transfer to visual-perception tasks.

What would settle it

Have human players from the same 'Guess the Build' community solve the same 500 static problems and the same 500 three-attempt dynamic problems with the same synthetic hints; if human accuracy is no better than the best model's 57.8% and 66.0%, the difficulty claim could be an artifact of underdetermined puzzles rather than a VLM creativity deficit.

Watch

Extended reading notes

Core claim

GuessBench's central claim is that decoding another person's creative construction—figuring out what a block-based build represents from its appearance and a sparse hint—is a distinct, testable skill that vision-language models do not yet have. In the static setting the model sees one finished build plus a hint revealing the answer's word and letter structure; the best model tested is correct 57.8% of the time. In the dynamic setting it sees three progressively complete builds and increasingly revealing hints; on the third attempt the best model reaches 66.0%, meaning 34% of instances remain wrong. The paper also reports a stark open-versus-API gap, transfer gains from fine-tuning on the benchmark's reasoning traces, strong correlations between accuracy and concept frequency in pretraining data, and a tendency for models to abandon correct answers when a user falsely insists they are wrong.

Load-bearing premise

The benchmark's load-bearing premise is that its image-plus-letter-hint problems measure creative sensemaking rather than object recognition combined with a spelling puzzle.

Editorial extensions

If this is right

  • If GuessBench measures what it claims, current vision-language models are much weaker at decoding creative intent than their strong scores on conventional visual question-answering benchmarks suggest.
  • The large gap between open and API models implies that strong performance on earlier benchmarks may come partly from memorization of similar data rather than from general visual interpretation.
  • Because accuracy tracks concept frequency and language resources, improving creative sensemaking will require targeted data or retrieval support for long-tail concepts, non-Western symbols, and low-resource languages, not just larger models.
  • Fine-tuning on GuessBench reasoning traces should transfer to perception-heavy tasks, while tasks requiring precise logical or mathematical reasoning may degrade, matching the paper's reported MathVista decline.
  • The contradiction experiments imply that interactive uses—where users push back on a model's answer—are risky in creative settings unless the model can check whether its own guess is right.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not include a human baseline; recruiting players from the same community to solve the identical problems would show how much of the 34% error floor is task underdetermination rather than model limitation.
  • Because the hints are synthetic letter patterns rather than the semantic, often playful hints real players give, part of what GuessBench measures may be constraint satisfaction over spelling; a version using recorded human hints would separate these abilities.
  • If the transfer result is robust, the dynamic setting could serve as a cheap automatic curriculum: models could be trained on progressively informative image-hint pairs, with difficulty controlled by how many letters are revealed.
  • The cultural-bias finding suggests a concrete fairness test: supplementing training data with builds and answers from low-resource languages and non-Western contexts should shrink the reported accuracy gap if the benchmark is sensitive to representation rather than to some artifact of translation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. GuessBench collects 500 Minecraft build sets (1500 screenshots) from the Hypixel minigame "Guess the Build" and defines static and dynamic VLM guessing tasks with progressively revealed hints that are synthetic letter masks generated from the answer word. The paper evaluates six VLMs, reports that GPT-4o achieves 57.8% static and 66.0% dynamic accuracy while open models lag far behind, and claims that fine-tuning on GuessBench reasoning traces improves visual perception tasks by 15.36% on average. Additional analyses examine multilingual robustness, concept frequency, multi-view inputs, sycophancy, in-context learning, and generation-versus-discrimination performance. The authors position the benchmark as a testbed for sensemaking creativity in the wild.

Significance. If the construct-validity concerns about the synthetic hints, curation, and scoring policy are resolved, GuessBench would be a valuable resource: the images come from real multiplayer gameplay with anonymization, the static/dynamic design allows measuring iterative refinement, the single-modality ablations in Table 2 cleanly support the claim that multimodal integration is required, and the experimental details in Appendix C are sufficiently concrete to reproduce. The paper also makes several useful empirical observations (concept-frequency correlation, language correlation, sycophancy, in-context learning) that stand independently of the "creativity" framing. Its current significance, however, is capped by the lack of evidence that the task measures creative sensemaking rather than masked-word solving plus Minecraft object recognition; the headline difficulty and transfer numbers cannot be fully interpreted without a human baseline and a scoring policy for multiple valid answers.

major comments (4)
  1. [§2.2, Hint Generation] The hints used in GuessBench are synthetic letter masks generated from the answer word (word lengths, then ⌈N/8⌉ random letters, then ⌈N/4⌉ random letters), not the natural-language hints that real players exchange in "Guess the Build". The abstract and Figure 1 describe "natural language hints of varying completeness", but the actual prompts are templated statements such as "The 5th letter is r". A real hint such as "it's a green bug" has a fundamentally different information structure from a positional letter constraint, and the letter mask gives exact positional evidence that turns the problem into a word search. Since the central claim is that GuessBench measures sensemaking creativity in the wild, the paper needs either to collect or replicate real player hints, or to justify with human experiments that the synthetic masks preserve the difficulty and cognitive operation of human hinting; otherwise the headline difficulty numbers reflect a word-puzzle-plus-recognition task rather than decoding another person's creative intent.
  2. [§2.2, Build Collection] The curation explicitly removes builds where players spell out the answer with blocks or where "no meaningful construction is present". These are precisely the noisy, low-effort, and idiosyncratic player behaviors that "creativity in the wild" is supposed to include, yet no statistics are given for how many builds were removed or how the remaining 500 sets represent the game's creative space. The paper also provides no human baseline on the curated 500 sets, so the interpretation of GPT-4o's 57.8% static accuracy as "uniquely challenging" is underdetermined: without human accuracy, the reader cannot tell whether the benchmark is hard because it measures creative decoding or because the synthetic hints and curation make the task artificially constrained.
  3. [§2.3 and §4 (evaluation of multiple valid answers)] The paper acknowledges in §4 and Appendix H.2 that some GuessBench questions have multiple valid answers, but the evaluation protocol appears to be exact-match after GPT-4o answer extraction (Appendix C.2) and no policy is stated for counting alternative valid answers as correct. If a model outputs a different but valid construction (e.g., "Finger of God" for a build that is also "Modern Art"), it is presumably scored incorrect. This directly affects all reported accuracy numbers in Table 2 and the open-versus-API gap, and it is not an edge case: the paper's own case study shows a correct initial answer being abandoned during image retrieval precisely because of this ambiguity. The authors should specify the scoring rule, provide a set of accepted answers per problem (or measure inter-annotator agreement on answer validity), and report accuracy under that rule.
  4. [§5.5, Figure 6] The abstract and introduction claim that fine-tuning on GuessBench reasoning traces "improves visual perception tasks by 15.36% on average", but §5.5 reports a decline on MathVista and gains only on MultiChartQA and BLINK, and the 15.36% figure does not appear in §5.5 or Figure 6. The average presumably covers only the two positive benchmarks; as stated, the claim is misleading because a reader would expect an average over all evaluated target tasks. Please report per-benchmark deltas, the exact set of tasks included in the average, variance or repeated seeds, and clarify whether MathVista's decline is included. With 289 training samples, one base model, and one seed, the transfer result also needs additional conditions to support the general claim.
minor comments (7)
  1. [Abstract and Introduction] The phrase "start-of-the-art" should be "state-of-the-art", and Table 2 contains the typo "Alldenotes" for "All denotes".
  2. [§2.3] The evaluation section defines accuracy but does not state how near-synonyms or paraphrases (e.g., "graveyard" versus "cemetery") are handled; this should be specified alongside the exact-match rule.
  3. [§5.1, Figure 2] The Figure 2 caption says "higher is better" for all three AutoBencher metrics, but Difficulty is defined as the lowest error rate; the direction of each metric should be made explicit to avoid confusion.
  4. [Figures 2–5] In the provided manuscript, the text inside Figures 2, 3, 4, and 5 is rendered as Unicode escape sequences, making the plots unreadable; please ensure the figures are legible in the submission.
  5. [Appendix C.2] Using GPT-4o as the answer extractor for responses generated by GPT-4o introduces a possible scoring bias; report extraction accuracy on a human-annotated sample to show that the extraction step is not systematically favoring or penalizing particular models.
  6. [Appendix E] The manual evaluation that yields 80% generative accuracy reports no inter-annotator agreement or detailed rubric for "plausibly constructed within the Minecraft environment", so the comparison to the 40% discriminative accuracy should be interpreted cautiously.
  7. [Data Availability] The paper does not state where the 1500 images and annotations will be released; a public data link or a clear availability statement is needed for the benchmark to be usable by the community.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GuessBench is an empirical benchmark whose difficulty and transfer results are self-contained measurements, not derivations from fitted inputs.

full rationale

The paper contains no derivation chain in which a predicted quantity is defined in terms of the same quantity or in which a fitted parameter is later relabeled as a prediction. The central results are measured accuracies of VLMs on a fixed, manually curated set of 500 Minecraft builds with 2000 problems; these numbers come from held-out model inference, not from any equation that constructs the outcome. The synthetic hints in Section 2.2 are generated from the answer word lengths and randomly selected letters, but this is dataset design rather than circularity: the benchmark then evaluates whether models can use those hints, and the difficulty numbers are contingent empirical findings. The use of GPT-4o as the answer extractor (Section 2.3, Appendix C.2) is a scoring convenience applied uniformly to all models; it does not fit a parameter to the data and then report that fit as a prediction, and it does not by construction force the reported open-vs-API gap or the 34% GPT-4o error rate. The fine-tuning experiment in Section 5.5 uses GPT-4o's correctly answered GuessBench traces as training data and evaluates on separate benchmarks (MathVista, MultiChartQA, BLINK); this is a transfer-learning measurement with held-out test sets, not a circular reuse of the target metric. Self-citations to the authors' prior work (e.g., pluralistic alignment [17], MultiChartQA [88]) are contextual references and are not load-bearing for the benchmark's difficulty claims; no uniqueness theorem or forced-choice argument is imported from those works. The strongest interpretive risk is construct validity, namely whether the synthetic letter-mask hints and the curation of low-quality builds preserve 'creativity in the wild' as claimed, but construct validity concerns are distinct from circularity and do not make any reported result equivalent to its inputs by construction. Therefore the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces GuessBench as a dataset and task, but no new physical, theoretical, or ontological entity is postulated. The benchmark is an invented artifact, not an invented entity in the sense used here.

free parameters (3)
  • Hint reveal proportions (1/8, 1/4) = ceil(N/8) and ceil(N/4) letters per attempt
    Section 2.2: the divisors 8 and 4 are hand-picked to mimic the game and balance difficulty. They shape how easy the word puzzle is and therefore directly influence reported accuracies.
  • Number of dynamic attempts = T=3
    Section 2.2 sets the dynamic task to three attempts; the headline dynamic accuracies (e.g., GPT-4o 66.0% on attempt 3) are defined by this choice.
  • Short/long answer cutoff = 8 letters
    Section 2.3 splits answers at length 8 using observed median/mean 8/8.15; this creates the short/long reporting groups but is a descriptive choice.
assumptions (5)
  • domain assumption The curated screenshots of Guess the Build builds are a valid operationalization of 'creativity in the wild' after removing spelling builds and empty builds.
    Section 2.2 Build Collection filters out low-quality builds, yet the paper describes the remaining set as a pristine testbed for pervasive, noisy, pluralistic creativity.
  • domain assumption Formula-generated letter hints are informationally equivalent to natural-language hints from real players.
    Section 2.2 Hint Generation synthesizes hints from answer length and random letters. The abstract calls the resulting problems 'natural language hints of varying completeness', so the paper's framing depends on this equivalence.
  • domain assumption GPT-4o-based answer extraction plus exact-match comparison is a valid correctness metric even though the paper states some problems have multiple valid answers.
    Section 2.3 adopts MathVista's extraction method; Section 4 acknowledges many-to-one mappings but no alternate answer list is provided, so valid alternative answers can be counted as wrong.
  • domain assumption Screenshots may be academically reused and anonymized reshared under Minecraft and Hypixel terms of service.
    Appendix B ethics review asserts player consent and ToS compliance; this legal/ethical premise underlies the dataset's legitimacy.
  • domain assumption Dolma-v1.7 n-gram frequencies in Infini-gram are a valid proxy for VLM training-data exposure to the target concepts.
    Section 5.3 uses concept frequency in Dolma to conclude that accuracy correlates with training-data frequency; training data of GPT-4o is not publicly known.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GuessBench: Sensemaking Multimodal Creativity in the Wild." pith.science (2026). https://pith.science/paper/75WO6TYM

@misc{pith2026250600814,
  author       = {Pith},
  title        = {Pith review of: GuessBench: Sensemaking Multimodal Creativity in the Wild},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/75WO6TYM}},
  note         = {Machine review of arXiv:2506.00814}
}
read the original abstract

We propose GuessBench, a novel benchmark that evaluates Vision Language Models (VLMs) on modeling the pervasive, noisy, and pluralistic human creativity. GuessBench sources data from "Guess the Build", an online multiplayer Minecraft minigame where one player constructs a Minecraft build given a concept (e.g. caterpillar) and others try to guess it with natural language hints, presenting a pristine testbed for sensemaking creativity in the wild with VLMs acting as guessers. We curate 1500 images from the actual gameplay and design 2000 problems spanning static and dynamic image settings, natural language hints of varying completeness, and more. Extensive experiments with six open/API VLMs and five reasoning enhancement approaches demonstrate that GuessBench presents a uniquely challenging task in creativity modeling: even the start-of-the-art GPT-4o is incorrect on 34% of instances, while we observe a huge performance gap (13.87% vs. 53.93% on average) between open and API models. When used as a resource to improve VLMs, fine-tuning on the reasoning traces for GuessBench problems improves visual perception tasks by 15.36% on average. Further analysis reveals that VLM performance in creativity sensemaking correlates with the frequency of the concept in training data, while the accuracy drops sharply for concepts in underrepresented cultural contexts and low-resource languages.

Figures

Figures reproduced from arXiv: 2506.00814 by the authors.

Figure 1
Figure 1. GUESSBENCH consists of player-constructed Minecraft builds representing real-world entities and concepts. In each problem, VLMs are required to infer what the build represents based on the provided image(s) and hint(s). We curate two settings: the static setting where only one image and hint are provided, and the dynamic setting where two sequences of progressively refined builds and corresponding hints are presente… view at source ↗
Figure 2
Figure 2. The AutoBencher scores of GUESS￾BENCH, MathVista, and HallusionBench, higher is better. GUESSBENCH demonstrates higher levels of difficulty, separability, and novelty compared to the other two benchmarks. We adopt the evaluation metrics proposed by the AutoBencher [41] framework to quantitatively assess the quality of our dataset. Specifically, we compute three key metrics: Difficulty, Sepa￾rability, and Novelty. Th… view at source ↗
Figure 3
Figure 3. Performance of GPT-4o across eight languages with varying levels of resource availability. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (13 more)
Figure 5
Figure 5. Figure 5: GPT-4o performance under Single-View and Multi-View settings. In three successive [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 4
Figure 4. Figure 4: GPT-4o performance across different lev [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: The performance of various fine-tuning methods. Fine-tuning improves performance on both MultiChartQA and BLINK, whereas it re￾duces performance on MathVista. Since GUESSBENCH is designed to emphasize perceptual and interpretive capabilities, fine￾tuning on it signific…
Figure 7
Figure 7. Figure 7: A Minecraft build depicting the China Post logo. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Evaluation results of Single-Question and Multi-Question settings. All denotes the overall [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: An example illustrates the first attempt in the dynamic task setting: GPT-4o responds based [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: An example illustrates the second attempt in the dynamic task setting: GPT-4o, with [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: An example illustrates the third attempt in the dynamic task setting: GPT-4o, with access [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]
Figure 12
Figure 12. Figure 12: An example of the static task setting: GPT-4o, based on a complete Minecraft build and a [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: One example from the static task. Initially, GPT-4o provides the correct answer based on [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]
Figure 14
Figure 14. Figure 14: Following the results of the first image retrieval, GPT-4o revises its initially accurate [PITH_FULL_IMAGE:figures/full_fig_p028_14.png]
Figure 15
Figure 15. Figure 15: Subsequently, GPT-4o performs a second round of image retrieval. After retrieving [PITH_FULL_IMAGE:figures/full_fig_p029_15.png]
Figure 16
Figure 16. Figure 16: Finally, GPT-4o conducts a third round of image retrieval. After retrieving and comparing [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

86 extracted references · 52 canonical work pages

  1. [1]

    Saeid Alavi Naeini, Raeid Saqur, Mozhgan Saeidi, John Giorgi, and Babak Taati. Large language models are fixated by red herrings: Exploring creative problem solving and einstellung effect using the only connect wall dataset.Advances in Neural Information Processing Systems, 36:5631–5652, 2023

    2023

  2. [2]

    Repurposing in ai: A distinct approach or an extension of cre- ative problem solving? InICLR Blogposts 2025, 2025

    Anonymous. Repurposing in ai: A distinct approach or an extension of cre- ative problem solving? InICLR Blogposts 2025, 2025. URL https: //d2jud02ci9yv69.cloudfront.net/2025-04-28-repurposing-85/blog/ repurposing/. https://d2jud02ci9yv69.cloudfront.net/2025-04-28-repurposing- 85/blog/repurposing/

    2025

  3. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  4. [5]

    The arts and literacy: What does it mean to be arts literate?International Journal of Education & the Arts, 14(18), 2013

    Georgina Barton. The arts and literacy: What does it mean to be arts literate?International Journal of Education & the Arts, 14(18), 2013

    2013

  5. [6]

    Art or artifice? large language models and the false promise of creativity

    Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. Art or artifice? large language models and the false promise of creativity. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1–34, 2024

    2024

  6. [7]

    Nebula: A discourse aware Minecraft builder

    Akshay Chaturvedi, Kate Thompson, and Nicholas Asher. Nebula: A discourse aware Minecraft builder. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6431–6443, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024. findin...

    doi:10.18653/v1/2024 2024

  7. [9]

    HoLLMwood: Unleashing the creativity of large language models in screenwriting via role playing

    Jing Chen, Xinyu Zhu, Cheng Yang, Chufan Shi, Yadong Xi, Yuxiang Zhang, Junjie Wang, Jiashu Pu, Tian Feng, Yujiu Yang, and Rongsheng Zhang. HoLLMwood: Unleashing the creativity of large language models in screenwriting via role playing. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics:...

    work page doi:10.18653/v1/2024.findings-emn 2024

  8. [10]

    Creative text-to-audio generation via synthe- sizer programming

    Manuel Cherep, Nikhil Singh, and Jessica Shand. Creative text-to-audio generation via synthe- sizer programming. InProceedings of the 41st International Conference on Machine Learning, pages 8270–8285, 2024

    2024

Show all 86 references
  1. [11]

    Unsloth, 2023

    Michael Han Daniel Han and Unsloth team. Unsloth, 2023. URL http://github.com/ unslothai/unsloth

    2023

  2. [12]

    VillagerAgent: A graph- based multi-agent framework for coordinating complex task dependencies in Minecraft

    Yubo Dong, Xukun Zhu, Zhengzhe Pan, Linchao Zhu, and Yi Yang. VillagerAgent: A graph- based multi-agent framework for coordinating complex task dependencies in Minecraft. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Li...

    2024 doi

  3. [13]

    Creation-mmbench: Assessing context-aware creative intelligence in mllm.arXiv preprint arXiv:2503.14478, 2025

    Xinyu Fang, Zhijian Chen, Kai Lan, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, et al. Creation-mmbench: Assessing context-aware creative intelligence in mllm.arXiv preprint arXiv:2503.14478, 2025

    2025 arXiv

  4. [14]

    How compositional generalization and creativity improve as diffusion models are trained

    Alessandro Favero, Antonio Sclocchi, Francesco Cagnetta, Pascal Frossard, and Matthieu Wyart. How compositional generalization and creativity improve as diffusion models are trained. In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy

    2025

  5. [15]

    Redefining< creative> in dic- tionary: Towards an enhanced semantic understanding of creative generation.arXiv preprint arXiv:2410.24160, 2024

    Fu Feng, Yucheng Xie, Xu Yang, Jing Wang, and Xin Geng. Redefining< creative> in dic- tionary: Towards an enhanced semantic understanding of creative generation.arXiv preprint arXiv:2410.24160, 2024

    2024 arXiv

  6. [16]

    Ccedit: Creative and controllable video editing via diffusion models

    Ruoyu Feng, Wenming Weng, Yanhui Wang, Yuhui Yuan, Jianmin Bao, Chong Luo, Zhibo Chen, and Baining Guo. Ccedit: Creative and controllable video editing via diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6712–6722, 2024

    2024

  7. [17]

    Modular pluralism: Pluralistic alignment via multi-llm collaboration

    Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, and Yulia Tsvetkov. Modular pluralism: Pluralistic alignment via multi-llm collaboration. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 41...

    2024

  8. [18]

    Blink: Multimodal large language models can see but not perceive

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. InEuropean Conference on Computer Vision, pages 148–166. Springer, 2024

    2024

  9. [19]

    Discovering creative behaviors through duplex: Diverse universal features for policy exploration

    Borja G Leon, Francesco Riccio, Kaushik Subramanian, Peter Wurman, and Peter Stone. Discovering creative behaviors through duplex: Diverse universal features for policy exploration. Advances in Neural Information Processing Systems, 37:49625–49648, 2024

    2024

  10. [20]

    Commoncanvas: Open diffusion models trained on creative-commons images

    Aaron Gokaslan, A Feder Cooper, Jasmine Collins, Landan Seguin, Austin Jacobson, Mihir Patel, Jonathan Frankle, Cory Stephenson, and V olodymyr Kuleshov. Commoncanvas: Open diffusion models trained on creative-commons images. InProceedings of the IEEE/CVF Conference on Compute...

    2024

  11. [21]

    A confederacy of models: a comprehensive evaluation of LLMs on creative writing

    Carlos Gómez-Rodríguez and Paul Williams. A confederacy of models: a comprehensive evaluation of LLMs on creative writing. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14504–14528, Singapore...

    2023 doi

  12. [22]

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. InProcee...

    2024

  13. [23]

    Using ai to generate short videos as stimuli for supporting design creativity

    Ji Han, Chijioke C Obieke, Haosong Zhao, and Pingfei Jiang. Using ai to generate short videos as stimuli for supporting design creativity. InDS 136: Proceedings of the Asia Design and Innovation Conference (ADIC) 2024. The Design Society, 2024

    2024

  14. [24]

    Enhancing creative generation on stable diffusion-based models.arXiv preprint arXiv:2503.23538, 2025

    Jiyeon Han, Dahee Kwon, Gayoung Lee, Junho Kim, and Jaesik Choi. Enhancing creative generation on stable diffusion-based models.arXiv preprint arXiv:2503.23538, 2025

    2025 arXiv

  15. [25]

    Scaling creative inspiration with fine-grained functional aspects of ideas

    Tom Hope, Ronen Tamari, Daniel Hershcovich, Hyeonsu B Kang, Joel Chan, Aniket Kittur, and Dafna Shahaf. Scaling creative inspiration with fine-grained functional aspects of ideas. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–15, 2022

    2022

  16. [26]

    Composer: creative and controllable image synthesis with composable conditions

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: creative and controllable image synthesis with composable conditions. InProceedings of the 40th International Conference on Machine Learning, pages 13753–13773, 2023. 11

    2023

  17. [27]

    Social reward: Evaluating and enhancing generative ai through million-user feedback from an online creative community

    Arman Isajanyan, Artur Shatveryan, David Kocharian, Zhangyang Wang, and Humphrey Shi. Social reward: Evaluating and enhancing generative ai through million-user feedback from an online creative community. InThe Twelfth International Conference on Learning Representations

  18. [28]

    SimulBench: Evaluating language models with creative simulation tasks

    Qi Jia, Xiang Yue, Tuney Zheng, Jie Huang, and Bill Yuchen Lin. SimulBench: Evaluating language models with creative simulation tasks. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Findings of the Association for Computational Linguistics: NAACL 2025, pages 8118– 8131, A...

    2025

  19. [29]

    Self-[in] correct: Llms struggle with discriminating self-generated responses

    Dongwei Jiang, Jingyu Zhang, Orion Weller, Nathaniel Weir, Benjamin Van Durme, and Daniel Khashabi. Self-[in] correct: Llms struggle with discriminating self-generated responses. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24266–24275, 2025

    2025

  20. [30]

    Reinforcement learning friendly vision-language model for minecraft

    Haobin Jiang, Junpeng Yue, Hao Luo, Ziluo Ding, and Zongqing Lu. Reinforcement learning friendly vision-language model for minecraft. InEuropean Conference on Computer Vision, pages 1–17. Springer, 2024

    2024

  21. [31]

    An analytic theory of creativity in convolutional diffusion models.arXiv preprint arXiv:2412.20292, 2024

    Mason Kamb and Surya Ganguli. An analytic theory of creativity in convolutional diffusion models.arXiv preprint arXiv:2412.20292, 2024

    2024 arXiv

  22. [32]

    Aug- menting scientific creativity with an analogical search engine.ACM Transactions on Computer- Human Interaction, 29(6):1–36, 2022

    Hyeonsu B Kang, Xin Qian, Tom Hope, Dafna Shahaf, Joel Chan, and Aniket Kittur. Aug- menting scientific creativity with an analogical search engine.ACM Transactions on Computer- Human Interaction, 29(6):1–36, 2022

    2022

  23. [33]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 235–251. S...

    2016

  24. [34]

    Bridging generations using ai-supported co-creative activities

    Callie Y Kim, Arissa J Sato, Nathan Thomas White, Hui-Ru Ho, Christine P Lee, Yuna Hwang, and Bilge Mutlu. Bridging generations using ai-supported co-creative activities. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–15, 2025

    2025

  25. [35]

    Representation-to-creativity (R2C): Automated holistic scoring model for essay creativity

    Deokgi Kim, Joonyoung Jo, Byung-Won On, and Ingyu Lee. Representation-to-creativity (R2C): Automated holistic scoring model for essay creativity. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Findings of the Association for Computational Linguistics: NAACL 2025, pages 52...

    2025

  26. [36]

    Retrieval-augmented code generation for situated action generation: A case study on Minecraft

    Chalamalasetti Kranti, Sherzod Hakimov, and David Schlangen. Retrieval-augmented code generation for situated action generation: A case study on Minecraft. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: ...

    2024 doi

  27. [37]

    A comparative analysis of the effectiveness of rare tokens on creative expression using rambert

    Youbin Lee, Deokgi Kim, Byung-Won On, and Ingyu Lee. A comparative analysis of the effectiveness of rare tokens on creative expression using rambert. InFindings of the Association for Computational Linguistics: ACL 2023, pages 10063–10077, 2023

    2023

  28. [38]

    Seed- bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125, 2023

    Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed- bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125, 2023

    2023 arXiv

  29. [39]

    Auto mc-reward: Automated dense reward design with large language models for minecraft

    Hao Li, Xue Yang, Zhaokai Wang, Xizhou Zhu, Jie Zhou, Yu Qiao, Xiaogang Wang, Hongsheng Li, Lewei Lu, and Jifeng Dai. Auto mc-reward: Automated dense reward design with large language models for minecraft. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  30. [40]

    Tp2o: Creative text pair-to-object generation using balance swap-sampling

    Jun Li, Zedong Zhang, and Jian Yang. Tp2o: Creative text pair-to-object generation using balance swap-sampling. InEuropean Conference on Computer Vision, pages 92–111. Springer, 2024

    2024

  31. [41]

    Autobencher: Towards declarative benchmark construction

    Xiang Lisa Li, Farzaan Kaiyom, Evan Zheran Liu, Yifan Mai, Percy Liang, and Tatsunori Hashimoto. Autobencher: Towards declarative benchmark construction. InThe Thirteenth International Conference on Learning Representations

  32. [42]

    Optimus-2: Multimodal minecraft agent with goal-observation-action conditioned policy.arXiv preprint arXiv:2502.19902, 2025

    Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, and Liqiang Nie. Optimus-2: Multimodal minecraft agent with goal-observation-action conditioned policy.arXiv preprint arXiv:2502.19902, 2025

    2025 arXiv

  33. [43]

    Steve-1: A generative model for text-to-behavior in minecraft.Advances in Neural Information Processing Systems, 36:69900–69929, 2023

    Shalev Lifshitz, Keiran Paster, Harris Chan, Jimmy Ba, and Sheila McIlraith. Steve-1: A generative model for text-to-behavior in minecraft.Advances in Neural Information Processing Systems, 36:69900–69929, 2023

    2023

  34. [44]

    Infini- gram: Scaling unbounded n-gram language models to a trillion tokens

    Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, and Hannaneh Hajishirzi. Infini- gram: Scaling unbounded n-gram language models to a trillion tokens. InFirst Conference on Language Modeling, 2024

    2024

  35. [45]

    Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233. Springer, 2024

    2024

  36. [46]

    Ocrbench: on the hidden mystery of ocr in large multimodal models.Science China Information Sciences, 67(12):220102, 2024

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models.Science China Information Sciences, 67(12):220102, 2024

    2024

  37. [47]

    Procreate, don’t reproduce! propulsive energy diffusion for creative generation

    Jack Lu, Ryan Teehan, and Mengye Ren. Procreate, don’t reproduce! propulsive energy diffusion for creative generation. InEuropean Conference on Computer Vision, pages 397–414. Springer, 2024

    2024

  38. [48]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InThe Twelfth International Conference on Learning ...

    2024

  39. [49]

    Ai as humanity’s salieri: Quantifying linguistic creativity of language models via systematic attribution of machine text against web text.arXiv preprint arXiv:2410.04265, 2024

    Ximing Lu, Melanie Sclar, Skyler Hallinan, Niloofar Mireshghallah, Jiacheng Liu, Seungju Han, Allyson Ettinger, Liwei Jiang, Khyathi Chandu, Nouha Dziri, et al. Ai as humanity’s salieri: Quantifying linguistic creativity of language models via systematic attribution of machine...

    arXiv

  40. [50]

    Benchmarking language model creativity: A case study on code generation

    Yining Lu, Dixuan Wang, Tianjian Li, Dongwei Jiang, Sanjeev Khudanpur, Meng Jiang, and Daniel Khashabi. Benchmarking language model creativity: A case study on code generation. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Proceedings of the 2025 Conference of the Nation...

    2025

  41. [51]

    Subjective topic meets LLMs: Unleashing comprehensive, reflective and creative thinking through the negation of negation

    Fangrui Lv, Kaixiong Gong, Jian Liang, Xinyu Pang, and Changshui Zhang. Subjective topic meets LLMs: Unleashing comprehensive, reflective and creative thinking through the negation of negation. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2...

    2024 doi

  42. [52]

    Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems, 36:46534–46594, 2023

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems, 36:46534–46594, 2023. 13

    2023

  43. [53]

    Guillermo Marco, Julio Gonzalo, M.Teresa Mateo-Girona, and Ramón Del Castillo Santos. Pron vs prompt: Can large language models already challenge a world-class fiction author at creative text writing? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings o...

    2024 doi

  44. [54]

    Enhanced creativity and ideation through stable video synthesis.arXiv preprint arXiv:2405.13357, 2024

    Elijah Miller, Thomas Dupont, and Mingming Wang. Enhanced creativity and ideation through stable video synthesis.arXiv preprint arXiv:2405.13357, 2024

    2024 arXiv

  45. [55]

    Turning up the heat: Min-p sampling for creative and coherent llm outputs

    Nguyen Nhat Minh, Andrew Baker, Clement Neo, Allen G Roush, Andreas Kirsch, and Ravid Shwartz-Ziv. Turning up the heat: Min-p sampling for creative and coherent llm outputs. In The Thirteenth International Conference on Learning Representations

  46. [56]

    Ocr-vqa: Visual question answering by reading text in images

    Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In2019 international conference on document analysis and recognition (ICDAR), pages 947–952. IEEE, 2019

    2019

  47. [57]

    Multi-token prediction boosts creativity in algorithmic tasks

    Vaishnavh Nagarajan, Chen Henry Wu, Charles Ding, and Aditi Raghunathan. Multi-token prediction boosts creativity in algorithmic tasks. InWorkshop on Spurious Correlation and Shortcut Learning: Foundations and Solutions

  48. [58]

    Lakshmi Nair, Evana Gizzi, and Jivko Sinapov. Creative problem solving in large language and vision models - what would it take? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11978–119...

    2024 doi

  49. [59]

    Having beer after prayer? measuring cultural bias in large language models

    Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. Having beer after prayer? measuring cultural bias in large language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16366–16393, 2024

    2024

  50. [60]

    Partcraft: Crafting creative objects by parts

    Kam Woh Ng, Xiatian Zhu, Yi-Zhe Song, and Tao Xiang. Partcraft: Crafting creative objects by parts. InEuropean Conference on Computer Vision, pages 420–437. Springer, 2024

    2024

  51. [61]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. News announcement by OpenAI, 2024. URL https://openai.com/ index/hello-gpt-4o/

    2024

  52. [62]

    Mrsteve: Instruction-following agents in minecraft with what-where-when memory

    Junyeong Park, Junmo Cho, and Sungjin Ahn. Mrsteve: Instruction-following agents in minecraft with what-where-when memory. InThe Thirteenth International Conference on Learning Representations, 2024

    2024

  53. [63]

    Probing and inducing combinational creativity in vision-language models.arXiv preprint arXiv:2504.13120, 2025

    Yongqian Peng, Yuxi Ma, Mengmeng Wang, Yuxuan Wang, Yizhou Wang, Chi Zhang, Yixin Zhu, and Zilong Zheng. Probing and inducing combinational creativity in vision-language models.arXiv preprint arXiv:2504.13120, 2025

    2025 arXiv

  54. [64]

    Mp5: A multi-modal open-ended embodied system in minecraft via active perception

    Yiran Qin, Enshen Zhou, Qichang Liu, Zhenfei Yin, Lu Sheng, Ruimao Zhang, Yu Qiao, and Jing Shao. Mp5: A multi-modal open-ended embodied system in minecraft via active perception. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16307–...

    2024

  55. [65]

    Minecraft benchmark for large language models, 02 2025

    Hunter Senft-grupp, Isaac Gemal, Janna Lu, Florian Cäsar, and Keith Webber. Minecraft benchmark for large language models, 02 2025. Manuscript in preparation

    2025

  56. [66]

    Does generation require mem- orization? creative diffusion models using ambient diffusion.arXiv preprint arXiv:2502.21278, 2025

    Kulin Shah, Alkis Kalavasis, Adam R Klivans, and Giannis Daras. Does generation require mem- orization? creative diffusion models using ambient diffusion.arXiv preprint arXiv:2502.21278, 2025

    2025

  57. [67]

    Societal impacts research requires benchmarks for creative composition tasks

    Judy Hanwen Shen and Carlos Guestrin. Societal impacts research requires benchmarks for creative composition tasks. InICLR 2025 Workshop on Bidirectional Human-AI Alignment. 14

    2025

  58. [68]

    Math-llava: Bootstrapping mathematical reasoning for multimodal large language models

    Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See Kiong Ng, Lidong Bing, and Roy Lee. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 4663–4680, 2024

    2024

  59. [69]

    Position: a roadmap to pluralistic alignment

    Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christo- pher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, et al. Position: a roadmap to pluralistic alignment. InProceedings of the 41st International Conference on Machi...

    2024

  60. [70]

    Creative and context-aware transla- tion of East Asian idioms with GPT-4

    Kenan Tang, Peiyang Song, Yao Qin, and Xifeng Yan. Creative and context-aware transla- tion of East Asian idioms with GPT-4. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9285–9305, Mi...

    2024 doi

  61. [71]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  62. [72]

    Yufei Tian, Tenghao Huang, Miri Liu, Derek Jiang, Alexander Spangher, Muhao Chen, Jonathan May, and Nanyun Peng. Are large language models capable of generating human-level nar- ratives? InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, ...

    2024

  63. [73]

    Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas L Griffiths, and Faeze Brahman. Macgyver: Are large language models creative problem solvers? InProceedings of the 2024 Conference of the North American Chapter of the ...

    2024

  64. [74]

    Zola: Zero-shot creative long animation generation with short video model

    Fu-Yun Wang, Zhaoyang Huang, Qiang Ma, Guanglu Song, Xudong Lu, Weikang Bian, Yijin Li, Yu Liu, and Hongsheng Li. Zola: Zero-shot creative long animation generation with short video model. InEuropean Conference on Computer Vision, pages 329–345. Springer, 2024

    2024

  65. [75]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  66. [76]

    Create! don’t repeat: A paradigm shift in multi- label augmentation through label creative generation

    Letian Wang, Xianggen Liu, and Jiancheng Lv. Create! don’t repeat: A paradigm shift in multi- label augmentation through label creative generation. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua...

    2024

  67. [77]

    Creative birds: self-supervised single-view 3d style transfer

    Renke Wang, Guimin Que, Shuo Chen, Xiang Li, Jun Li, and Jian Yang. Creative birds: self-supervised single-view 3d style transfer. InProceedings of the IEEE/CVF international conference on computer vision, pages 8775–8784, 2023

    2023

  68. [78]

    Enhancing the reasoning ability of multimodal large language models via mixed preference optimization.arXiv preprint arXiv:2411.10442, 2024

    Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization.arXiv preprint arXiv:2411.10442, 2024

    2024 arXiv

  69. [79]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V . Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali,...

    2023

  70. [80]

    Charxiv: Charting gaps in realistic chart understanding in multimodal llms.Advances in Neural Information Processing Systems, 37: 113569–113697, 2024

    Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal llms.Advances in Neural Information Processing Systems, 37: 113569–113697, 2024. 15

    2024

  71. [81]

    Collaborating action by action: A multi-agent llm framework for embodied reasoning.arXiv preprint arXiv:2504.17950, 2025

    Isadora White, Kolby Nottingham, Ayush Maniar, Max Robinson, Hansen Lillemark, Mehul Maheshwari, Lianhui Qin, and Prithviraj Ammanabrolu. Collaborating action by action: A multi-agent llm framework for embodied reasoning.arXiv preprint arXiv:2504.17950, 2025

    2025 arXiv

  72. [82]

    Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  73. [83]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF Conference on Co...

    2024

  74. [84]

    Hidiffu- sion: Unlocking higher-resolution creativity and efficiency in pretrained diffusion models

    Shen Zhang, Zhaowei Chen, Zhenyu Zhao, Yuhao Chen, Yao Tang, and Jiajun Liang. Hidiffu- sion: Unlocking higher-resolution creativity and efficiency in pretrained diffusion models. In European Conference on Computer Vision, pages 145–161. Springer, 2024

    2024

  75. [85]

    Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities

    Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities. InThe Thirteenth International Conference on Learning Representations,...

    2025

  76. [86]

    Let’s think outside the box: Exploring leap-of-thought in large language models with creative humor generation

    Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, and Pan Zhou. Let’s think outside the box: Exploring leap-of-thought in large language models with creative humor generation. InProceedings of the IEEE/CVF Conference on Computer Vision and P...

    2024

  77. [87]

    Enhancing interactive image retrieval with query rewriting using large language models and vision language models

    Hongyi Zhu, Jia-Hong Huang, Stevan Rudinac, and Evangelos Kanoulas. Enhancing interactive image retrieval with query rewriting using large language models and vision language models. InProceedings of the 2024 International Conference on Multimedia Retrieval, pages 978–987, 2024

    2024

  78. [88]

    Guess the Build

    Zifeng Zhu, Mengzhao Jia, Zhihan Zhang, Lang Li, and Meng Jiang. MultiChartQA: Bench- marking vision-language models on multi-chart problems. InProceedings of the 2025 Confer- ence of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human L...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.