Pith. sign in

REVIEW 5 major objections 8 minor 102 references

Where Models Converge and Humans Diverge: A Coverage Framework for Distributional Pluralism in Open-Ended Generation

T0 review · 5 major / 8 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs produce plausible text but cover only a narrow slice of the human response space, a new coverage framework shows.

desk verdict A useful two-metric coverage framework with a defensible central finding, but the key human-baseline comparison is only tested at one hyperparameter setting. read the letter →

arxiv 2608.05576 v1 pith:6VS44NDV submitted 2026-08-06 cs.CL cs.CY

Zini Yang , Emily Wenger , Richard So This is my paper · ORCID
classification cs.CLcs.CY
keywords distributionalpluralismculturalreachopen-endedgenerationLLMcoveragein-boundaryratesemanticembeddingboundaryfanfictiondivergentthinking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a human-grounded, category-free way to measure how broadly an LLM's open-ended outputs spread across the space of legitimate human responses. It embeds human and model responses in the same semantic space, draws an empirical boundary around the human responses, and reports two numbers: In-Boundary Rate (IBR), the share of model outputs inside that boundary, and LLM Coverage (LLM-Cov), the share of human responses reached by at least one model output. Across the Alternative Uses Task, the Divergent Association Task, and Harry Potter fanfiction, current open-source instruction-tuned models score high on IBR (roughly 0.92-0.96) but far lower on LLM-Cov (roughly 0.29-0.46) than a Human-to-Human reference (roughly 0.90). The framework gives researchers a concrete target: a model with cultural reach should not only sound human but also reach the periphery of human expression, which current models systematically miss.

What carries the argument

The central mechanism is a geometry-based coverage framework built on an empirical human response boundary. All responses are embedded with a sentence transformer (all-mpnet-base-v2), PCA is fit only on the reference human responses, and the boundary is defined as the union of equal-radius balls centered at reference responses, where the radius is the median distance to the 15th nearest reference neighbor. LLM-Cov counts the fraction of reference responses reached by at least one model output, while IBR counts the fraction of model outputs inside the boundary; an equally sized held-out human sample evaluated through the same pipeline provides the Human-to-Human reference that calibrates expected coverage.

What would settle it

Re-run the pipeline with a representation built from human similarity judgments or discourse-aware embeddings instead of all-mpnet-base-v2. If a model that human raters judge to span varied plots, voices, and pairings shows roughly the same low LLM-Cov as in the paper, the metric is robust; if coverage jumps toward the Human-to-Human level, the reported gap is an artifact of the embedding's blind spots.

Watch

Extended reading notes

Core claim

The paper's central claim is that current LLMs are "human-plausible but not broadly pluralistic": they reliably generate outputs that fall inside an empirical boundary around human writing while covering only a narrow central region of that space. Using the union of local neighborhoods around reference human responses as the boundary, the authors find that LLM-Cov is far below Human-to-Human coverage across ideation and narrative tasks, and that the gap is largest at the periphery of the human response distribution. In Harry Potter fanfiction, the under-coverage is structured rather than random: covered texts are more canon-anchored, use more dialogue, and more often include mixed-gender relationships, while uncovered texts more often introduce non-Harry-Potter characters. The framework operationalizes cultural reach as measurable distributional breadth and provides an auditing tool for locating which cultural possibilities models omit.

Load-bearing premise

The results rest on the assumption that the embedding space preserves the dimensions of variation that make human responses culturally different; if narrative structure, voice, genre convention, or community-specific meaning collapse in that space, then coverage measures proximity in a flattened projection rather than true distributional breadth.

Editorial extensions

If this is right

  • Evaluation of open-ended generation can now separate plausibility from distributional breadth without requiring predefined response categories or dimensions of variation.
  • Higher decoding temperature raises LLM-Cov on every task while leaving IBR largely unchanged, so sampling strategy can expand model reach within the human response space.
  • Ensembling multiple models does not close the coverage gap: union coverage rises sharply with the first model and then plateaus below the Human-to-Human reference.
  • Coverage audits can locate systematic omissions; in Harry Potter fanfiction, canon-anchored, dialogue-heavy, and mixed-gender-content stories are covered more, while stories featuring non-Harry-Potter characters are covered less.
  • The Human-to-Human reference provides a ceiling for what coverage is achievable by an independent human sample, making the reported under-coverage interpretable as a property of the models rather than of the boundary.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extending the paper's logic, the same framework could audit coverage in multimodal open-ended generation, such as image or music generation, wherever a reference distribution of human artifacts can be embedded in a shared space.
  • If future work replaces the sentence encoder with discourse-aware or human-similarity-based representations, absolute coverage values may shift substantially; the paper's robustness checks only compare encoders from the same family, so the true size of the cultural-reach gap remains an open question.
  • The metric could be turned into an alignment objective: training or prompting toward higher LLM-Cov at the periphery, rather than merely higher IBR, would directly target the documented narrowing of cultural output.
  • Because even Human-to-Human coverage is not perfect, the framework implies that larger or more diverse human reference samples will be needed before coverage ceilings can be treated as stable for any given cultural domain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. This paper proposes a human-grounded coverage framework for evaluating open-ended LLM generation. The authors embed human and model responses in a shared Sentence-BERT space, define an empirical human response boundary as a union of kNN balls around a reference human sample, and propose two metrics: In-Boundary Rate (IBR, the fraction of model outputs inside the boundary) and LLM Coverage (LLM-Cov, the fraction of reference human responses within the boundary of at least one model output). They also construct a Human-to-Human (H2H) baseline by evaluating a held-out human sample against the same reference. Applying the framework to AUT, DAT, and Harry Potter fanfiction with five open-source instruction-tuned LLMs at two temperatures, they report that models achieve consistently high IBR (averages 0.918, 0.937, 0.964 across tasks) but substantially lower LLM-Cov than H2H-Cov (averages 0.305 vs 0.896, 0.286 vs 0.904, 0.457 vs 0.885; Table 1). They further show that coverage declines sharply toward the periphery of the human distribution and, for fanfiction, that covered texts are more canon-anchored, dialogue-heavy, and often contain mixed-gender relationships. The paper includes robustness sweeps on AUT for encoder, PCA threshold, boundary percentile, k, and model sample size, plus detailed reproducibility documentation for prompts, models, and feature operationalizations.

Significance. The framework addresses a real gap in LLM evaluation: internal diversity of model outputs does not imply coverage of the human response distribution. The human-to-human control is a well-chosen reference that anchors the comparison, and the distinction between plausibility (IBR) and distributional breadth (LLM-Cov) is conceptually useful. The paper's strengths include a clearly specified methodology, a broad set of robustness checks for the model-side metrics, detailed reproducibility (prompts, model identifiers, decoding settings, generation counts, and feature operationalizations), and a diagnostic analysis that goes beyond aggregate numbers to characterize what is missed. If the central finding is robust, the framework provides a reusable, category-free measure of distributional breadth ('cultural reach') and a method for auditing systematic omissions in open-ended generation. However, as detailed below, the evidence base currently has gaps in the robustness of the human-to-human reference and the dependence of the headline claims on hyperparameter choices.

major comments (5)
  1. [Appendix B.1, Table 4, Table 1] The central claim in §5.1 is comparative: LLM-Cov is 'substantially lower' than H2H-Cov (Table 1, e.g., 0.305 vs 0.896 on AUT). However, the robustness appendix never re-estimates the H2H baseline. Table 4 reports only model-side IBR and LLM-Cov across encoder, PCA, and boundary settings, and Table 6 only varies model output size; H2H-Cov and H2H-IBR remain single point estimates from one fixed split per task. Since the boundary percentile changes LLM-Cov by an order of magnitude (Table 4: AUT LLM-Cov 0.049 at p25 to 0.680 at p90), the H2H values could plausibly move as well; without recomputing them, the gap that motivates the paper's headline may depend on the chosen hyperparameter configuration. Please report H2H-Cov and H2H-IBR under the same sweeps (at least boundary percentiles and encoder choice), and, if a single split is used, provide confidence intervals across multiple random splits.
  2. [Abstract, §5.1, Table 4] The abstract and §5.1 describe LLMs as producing 'plausible but narrow' content, where plausibility is operationalized as high IBR. Table 4 shows that this is a boundary-choice-dependent conclusion: at p25 on AUT, mean IBR is 0.272, meaning more than 70% of model outputs fall outside the human boundary, while at p50 IBR is 0.918. The qualitative ordering IBR > LLM-Cov is preserved, but the absolute claim of high plausibility is not. The paper should either provide a principled justification for the p50 boundary (for example, by calibrating against human out-of-boundary rates) or explicitly restrict the 'plausible' characterization to the p50 geometry.
  3. [§7, Appendix B.1] The robustness check for the embedding space is limited to two Sentence-BERT encoders (MPNet and MiniLM) from the same BERT-based architecture family. Section 7 concedes that these embeddings 'may miss dimensions important for open-ended cultural generation, such as narrative structure, voice, genre convention, pragmatic nuance, and community-specific meaning,' yet no experiment uses a substantially different representation (e.g., a decoder LLM embedding, tf-idf, or topic-model projection). Because every IBR, LLM-Cov, and H2H value in the paper is computed in this single family of spaces, the finding that models concentrate near the center could be an artifact of the chosen geometry. Please include at least one representation outside the Sentence-BERT family, or frame the conclusions as covering in this specific embedding space rather than 'cultural reach.'
  4. [§A.1, §A.4, Table 7] In the HP task, model outputs are filtered to 200–250 words while human excerpts span 150–300 words (Appendix A.1). The paper does not control for this length mismatch. If embedding distance correlates with length, the model may fail to cover very short or very long human references simply because it never produces outputs at those lengths, contributing to the LLM-Cov gap independent of distributional breadth. The covered-vs-uncovered diagnostic (Table 7) omits word count as a variable. Please add length as a covariate, generate a length-matched model sample, or otherwise demonstrate that the coverage gap is not driven by length differences.
  5. [Appendix B] All hyperparameter robustness experiments are run on AUT only (Tables 4–6). The HP fanfiction analysis, which carries the strongest interpretive claims about structured omissions (§5.3, Table 2), is never subjected to these sweeps; the covered/uncovered split is defined by a single model configuration (Qwen2.5-7B at t=1.0) and a single p50 boundary. The features reported in Table 2 might shift under a different boundary percentile or encoder, and the uncovered set would become very small at p90. Please report at least boundary-percentile and encoder robustness for the HP diagnostic, or provide an explicit argument for why the AUT sweeps suffice.
minor comments (8)
  1. [§5.2] The text says that AUT and HP fanfiction 'exhibit the same qualitative pattern' as the DAT example shown in Figure 4, but no supporting figure or table for those tasks is included in the provided text; please add the corresponding plots or tables in the main text or appendix.
  2. [Table 8] The M/F relationship direction is not statistically significant for Mistral-24B (p=0.331), so the statement 'the direction is consistent across model-specific splits' should be qualified to note that one of five splits is non-significant.
  3. [Table 2] Table 2 and Table 7 report raw p-values for many comparisons (around 15 variables); consider adjusting for multiple comparisons (e.g., FDR) or at least noting that the M/F p=0.002 result would not survive a stringent correction.
  4. [§A.3] Appendix A.3 states 'We do not balance the sample by author,' which is relevant because a few prolific authors could dominate the reference distribution; please discuss the potential impact on the boundary estimates.
  5. [§A.3] The exact sizes of the reference and held-out sets for AUT and DAT after the split into two non-overlapping equally sized samples are not stated; please report these sizes.
  6. [§3] The notation uses both 'H heldout' and 'H_{heldout}' inconsistently; please unify the notation for clarity.
  7. [§C.4] The relationship labels are produced by a Mistral-Small-24B judge; since this is the same model family as one of the evaluated generators, a brief discussion of potential judge bias would strengthen the diagnostic.
  8. [Abstract] The term 'cultural reach' is used without an explicit definition before the abstract; define it at first occurrence in the introduction.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the coverage gap is an empirical comparison against an external human baseline, not an artifact of the metric definitions.

full rationale

The derivation chain is self-contained: the reference space is constructed from human responses, and both model outputs and held-out human outputs are projected with the same reference-fitted PCA and evaluated against the same boundary (Section 3). LLM-Cov and H2H-Cov use identical formulas and differ only in the generator set, so the reported gap is an empirical comparison against an external human baseline rather than an identity forced by the definitions. The parametric choices (encoder, PCA threshold, k, boundary percentile) are not fitted to model outputs, and Appendix B sweeps show the qualitative ordering persists across settings; the fact that the H2H baseline is not re-swept is a robustness limitation, not circularity. The self-citation to Wenger & Kenett (2025) is used only to interpret an ensemble plateau that is directly measured in Figure 3, so it is not load-bearing. No equation reduces to its own input, and no reported quantity is a fitted parameter renamed as a prediction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on four main supports: (1) the sentence-embedding space preserves the dimensions that matter for cultural reach, which the paper itself flags as uncertain in Section 7; (2) Euclidean distance in the PCA-projected space is a meaningful proxy for cultural distance; (3) the finite human reference samples are representative, with AUT/DAT provenance uncited and the HP sample drawn from unspecified URL lists; and (4) the hand-chosen boundary parameters (k=15, p50, 90% variance) set the metric scale, though robustness checks show the qualitative ordering survives. Secondary diagnostics add a fifth support: the Mistral-Small-24B judge is assumed accurate without human validation. No invented entities are introduced.

free parameters (4)
  • kNN neighborhood size k = 15 (main setting)
    Chosen by hand in Section 4.3 ('k=15, which provides a robust notion of local semantic similarity without being dominated by near-duplicate responses'); Appendix B.2 sweeps k in {5,10,15,20,30} and shows IBR varying from 0.587 to 0.962, so the metric scale depends on this choice.
  • Boundary quantile q = p50, the median (main setting)
    The global radius epsilon is the q-th quantile of kNN distances; Appendix B.1 shows AUT LLM-Cov ranging from 0.049 at p25 to 0.680 at p90, so the absolute coverage numbers are strongly set by this hand-chosen percentile even though the qualitative ordering survives.
  • PCA variance threshold = 90%, capped at 200 components (159 dims for HP)
    Retain the smallest number of components explaining at least 90% of reference-human variance; Appendix B.1 sweeps 85/90/95% with modest metric shifts, and Appendix A.4 reports the HP space as 159 dimensions at 0.9010 variance.
  • HP length filtering window = Model outputs 200-250 words; human excerpts 150-300 words
    Appendix A.1 and A.3 discard model outputs outside 200-250 words, accept human excerpts of 150-300 words ('at most 50 words outside the target interval'), and remove files matching 'a predefined list of obvious non-story or crossover indicators', which shapes which human texts form the reference distribution.
assumptions (5)
  • domain assumption all-mpnet-base-v2 sentence embeddings preserve the dimensions of variation relevant to cultural reach
    Invoked in Section 3 (Data Embedding and Projection); the paper's own Limitations admit the encoder 'may miss dimensions important for open-ended cultural generation, such as narrative structure, voice, genre convention, pragmatic nuance, and community-specific meaning'. Only MiniLM and PCA thresholds are tested as robustness.
  • domain assumption Euclidean distance in the PCA-projected space is a valid proxy for semantic or cultural distance
    The empirical boundary is a union of Euclidean balls B(hi, epsilon) defined in Section 3; if the true variation is manifold-structured or hierarchical, ball coverage mis-measures reach.
  • domain assumption The finite human reference samples are representative of the human response distributions
    Appendix A.3: 4,000 AUT responses, 1,000 DAT responses, 4,435 HP excerpts. AUT and DAT provenance is uncited; the HP sample is a convenience download from unspecified URL lists, filtered by length and crossover indicators, with no author balancing.
  • ad hoc to paper A union of equal-radius balls around reference points adequately estimates the empirical human response region
    Section 3 defines B_H(epsilon) with a single global scalar epsilon applied at every reference point, which will over-cover dense regions and under-cover sparse ones; this is a modeling choice specific to the paper.
  • domain assumption The Mistral-Small-24B greedy-decoding judge produces accurate relationship and character labels
    Appendix C.4: 'Relationship labels are produced for this same corrected reference-human split by a deterministic (greedy-decoding) Mistral-Small-24B judge.' No human agreement or error analysis is reported, and the M/F relationship result is the diagnostically weakest claim (p=0.331 for one model).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Where Models Converge and Humans Diverge: A Coverage Framework for Distributional Pluralism in Open-Ended Generation." pith.science (2026). https://pith.science/paper/6VS44NDV

@misc{pith2026260805576,
  author       = {Pith},
  title        = {Pith review of: Where Models Converge and Humans Diverge: A Coverage Framework for Distributional Pluralism in Open-Ended Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6VS44NDV}},
  note         = {Machine review of arXiv:2608.05576}
}
read the original abstract

When a large language model (LLM) writes Harry Potter fanfiction, it reliably produces fundamental elements of the Hogwarts universe, such as recognizable places and characters. Human-written Harry Potter fanfictions, however, typically include these fundamentals and much more, incorporating stylistically irregular content and relationship-diverse plotlines. This gap between LLM and human writing has been noted across a variety of domains. LLMs tend to produce "average" writing, while human writing contains more diverse content that covers a broader distribution. Existing work has shown the existence of this distributional "gap", but no work has proposed a systematic way to measure it. Our paper proposes a human-grounded framework that uses the empirical distribution of human writing on a topic to measure the distributional breadth of LLM-generated content on that same topic. We propose two metrics, LLM Coverage (LLM-Cov) and In-Boundary Rate (IBR), that separate the plausibility of LLM content from its distributional breadth. Across ideation and narrative tasks, we find that current LLMs produce plausible but narrow content that concentrates near the center of the human response space. Our framework can enable researchers to better assess the distributional breadth of LLM-authored content, which we term its "cultural reach".

Figures

Figures reproduced from arXiv: 2608.05576 by the authors.

Figure 1
Figure 1. Overview of our human-grounded coverage framework. vides an independent Human-to-Human reference. Model responses M and held-out human responses Hheldout are evaluated against the same reference set H using the same projection, distance threshold, and metrics. The two com￾parisons therefore differ only in whether the generated re￾sponses come from a model or another human sample. We treat H as a finite, task-specifi… view at source ↗
Figure 2
Figure 2. Geometric interpretation of LLM-Cov and IBR. LLM￾Cov measures the fraction of reference human responses reached by model outputs, while IBR measures the fraction of model out￾puts that remain inside the empirical human boundary. ideation with no community-specific context. DAT elic￾its semantically distant concepts and is grounded in asso￾ciative theories of creativity (Mednick, 1962; Olson et al., 2021). As an abst… view at source ↗
Figure 3
Figure 3. Sequential model addition yields rapidly diminishing gains in LLM-Cov and remains below the Human-to-Human reference on every task. response space (Wenger & Kenett, 2025). 5.2. Coverage by Human Centrality [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Models under-cover peripheral regions of the human response distribution. LLM-Cov drops sharply toward less cen￾tral human responses. We next ask how LLM coverage is distributed across the hu￾man response space. To do this, we divide human responses into ten centrality…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

102 extracted references · 53 canonical work pages

  1. [1]

    Machine Culture , journal =

    Brinkmann, Levin and Baumann, Fabian and Bonnefon, Jean-Fran. Machine Culture , journal =. 2023 , volume =

    2023

  2. [2]

    Provocations from the Humanities for Generative

    Klein, Lauren and Martin, Meredith and Brock, Andr. Provocations from the Humanities for Generative. 2025 , doi =

    2025

  3. [3]

    and Wilkens, Matthew , title =

    Underwood, Ted and Nelson, Laura K. and Wilkens, Matthew , title =. 2025 , doi =

    2025

  4. [4]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    Position: A Roadmap to Pluralistic Alignment , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , volume =

    2024

  5. [5]

    and Yu, Tong and Kumar, Sachin and Majumder, Bodhisattwa Prasad and Shang, Jingbo and Ammanabrolu, Prithviraj and McAuley, Julian J

    Xie, Zhouhang and Wu, Junda and Shen, Yiran and Jain, Raghav and Xia, Yu and Li, Xintong and Chang, Aaron and Rossi, Ryan A. and Yu, Tong and Kumar, Sachin and Majumder, Bodhisattwa Prasad and Shang, Jingbo and Ammanabrolu, Prithviraj and McAuley, Julian J. , title =. Proceedings of the Conference on Language Modeling , year =

  6. [6]

    Kirk, Hannah Rose and Whitefield, Alexander and R. The. Advances in Neural Information Processing Systems 37: Datasets and Benchmarks Track , year =

  7. [7]

    Proceedings of the 40th International Conference on Machine Learning , publisher =

    Santurkar, Shibani and Durmus, Esin and Ladhak, Faisal and Lee, Cinoo and Liang, Percy and Hashimoto, Tatsunori , title =. Proceedings of the 40th International Conference on Machine Learning , publisher =. 2023 , volume =

    2023

  8. [9]

    AI & Society , year =

    Novis-Deutsch, Nurit and Elyoseph, Tal and Elyoseph, Zohar , title =. AI & Society , year =

Show all 102 references
  1. [10]

    , title =

    Poole-Dayan, Elinor and Wu, Jiayi and Sorensen, Taylor and Pei, Jiaxin and Bakker, Michiel A. , title =. Proceedings of the International Conference on Learning Representations , year =

  2. [11]

    Proceedings of the International Conference on Learning Representations , year =

    Nie, Shangrui and Omoomi, Kian and Flek, Lucie and Zhao, Zhixue and Welch, Charles , title =. Proceedings of the International Conference on Learning Representations , year =

  3. [12]

    Zhang, Lily H. and Milli, Smitha and Jusko, Karen Long and Smith, Jonathan and Amos, Brandon and Bouaziz, Wassim and Revel, Manon and Kussman, Jack and Sheynin, Yasha and Titus, Lisa and Radharapu, Bhaktipriya and Yu, Jane and Sarma, Vidya and Rose, Kristopher and Nickel, Maxi...

  4. [13]

    Collective Constitutional AI: Aligning a Language Model with Public Input , year =

  5. [15]

    and Liu, Ziquan and Ferianc, Martin and Treleaven, Philip and Rodrigues, Miguel , title =

    Masoud, Reem I. and Liu, Ziquan and Ferianc, Martin and Treleaven, Philip and Rodrigues, Miguel , title =. Proceedings of the 31st International Conference on Computational Linguistics , year =

  6. [17]

    2026 , doi =

    Fu, Yu and Son, Seongho and Bogunovic, Ilija , title =. 2026 , doi =

    2026

  7. [18]

    Xu, Shaoyang and Leng, Yongqi and Yu, Linhao and Xiong, Deyi , title =. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , publisher =. 2025 , pages =

    2025

  8. [19]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , publisher =

    Plank, Barbara , title =. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , publisher =. 2022 , pages =

    2022

  9. [22]

    Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future , publisher =

    Basile, Valerio and Fell, Michael and Fornaciari, Tommaso and Hovy, Dirk and Paun, Silviu and Plank, Barbara and Poesio, Massimo and Uma, Alexandra , title =. Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future , publisher =. 2021 , pages =

    2021

  10. [24]

    2024 , doi =

    Atmakuru, Anirudh and Nainani, Jatin and Bheemreddy, Rohith Siddhartha Reddy and Lakkaraju, Anirudh and Yao, Zonghai and Zamani, Hamed and Chang, Haw-Shiuan , title =. 2024 , doi =

    2024

  11. [30]

    2025 , doi =

    Hou, Zhaoyi Joey and Zhang, Bowei Alvin and Lu, Yining and Baghel, Bhiman Kumar and Brei, Anneliese and Lu, Ximing and Jiang, Meng and Brahman, Faeze and Chaturvedi, Snigdha and Chang, Haw-Shiuan and Khashabi, Daniel and Li, Xiang Lorraine , title =. 2025 , doi =

    2025

  12. [31]

    2025 , doi =

    Ismayilzada, Mete and Stevenson, Claire and Plas, Lonneke van der , title =. 2025 , doi =

    2025

  13. [32]

    Proceedings of the Conference on Language Modeling , year =

    Zhang, Yiming and Diddee, Harshita and Holm, Susan and Liu, Hanchen and Liu, Xinyue and Samuel, Vinay and Wang, Barry and Ippolito, Daphne , title =. Proceedings of the Conference on Language Modeling , year =

  14. [33]

    Machine Intelligence Research , year =

    Zhao, Yunpu and Zhang, Rui and Li, Wenyi and Huang, Di and Guo, Jiaming and Shaohui Peng and Yifan Hao and Yuanbo Wen and Xing Hu and Zidong Du and Qi Guo and Ling Li and Yunji Chen , title =. Machine Intelligence Research , year =. doi:10.1007/s11633-025-1546-4 , url =

    doi

  15. [34]

    The Twelfth International Conference on Learning Representations , year =

    Padmakumar, Vishakh and He, He , title =. The Twelfth International Conference on Learning Representations , year =

  16. [36]

    and Jaeger, Garrett J

    Runco, Mark A. and Jaeger, Garrett J. , title =. Creativity Research Journal , year =

  17. [37]

    , title =

    Amabile, Teresa M. , title =. Research in Organizational Behavior , year =

  18. [38]

    Creativity Research Journal , publisher =

    Simonton, Dean Keith , title =. Creativity Research Journal , publisher =. 2018 , volume =

    2018

  19. [39]

    Creativity Research Journal , publisher =

    Corazza, Giovanni Emanuele , title =. Creativity Research Journal , publisher =. 2016 , volume =

    2016

  20. [40]

    , title =

    Diedrich, Jennifer and Benedek, Mathias and Jauk, Emanuel and Neubauer, Aljoscha C. , title =. Psychology of Aesthetics, Creativity, and the Arts , publisher =. 2015 , volume =

    2015

  21. [41]

    2025 , doi =

    He, Zicong and Zhang, Boxuan and Liu, Weihao and Tang, Ruixiang and Cheng, Lu , title =. 2025 , doi =

    2025

  22. [42]

    Guilford, J. P. , title =

  23. [43]

    Mednick, S. A. , title =. Psychological Review , year =

  24. [45]

    The Thirteenth International Conference on Learning Representations , year =

    Si, Chenglei and Yang, Diyi and Hashimoto, Tatsunori , title =. The Thirteenth International Conference on Learning Representations , year =

  25. [47]

    Creativity Research Journal , year =

    Baer, John , title =. Creativity Research Journal , year =

  26. [48]

    , title =

    Baer, John and Kaufman, James C. , title =. Roeper Review , year =

  27. [49]

    2025 , doi =

    Jain, Shomik and Lanchantin, Jack and Nickel, Maximilian and Ullrich, Karen and Wilson, Ashia and Watson-Daniels, Jamelle , title =. 2025 , doi =

    2025

  28. [50]

    2025 , doi =

    Lai, Clin and Luchini, Simone and Lauharatanahirun, Nina and Beaty, Roger , title =. 2025 , doi =

    2025

  29. [52]

    2025 , doi =

    Wenger, Emily and Kenett, Yoed , title =. 2025 , doi =

    2025

  30. [53]

    Boden, Margaret A , title =

  31. [54]

    , title =

    Boden, Margaret A. , title =

  32. [56]

    Csikszentmihalyi, Mihaly , title =

  33. [59]

    Advances in Neural Information Processing Systems , publisher =

    Song, Kaitao and Tan, Xu and Qin, Tao and Lu, Jianfeng and Liu, Tie-Yan , title =. Advances in Neural Information Processing Systems , publisher =. 2020 , doi =

    2020

  34. [61]

    Science , year =

    Farrell, Henry and Gopnik, Alison and Shalizi, Cosma and Evans, James , title =. Science , year =

  35. [62]

    Fanfiction in the age of ai: Community perspectives on creativity, authenticity and adoption

    Alfassi, R., Cooper, A., Mitchell, Z., Calabro, M., Shaer, O., and Mokryn, O. Fanfiction in the age of ai: Community perspectives on creativity, authenticity and adoption. International Journal of Human--Computer Interaction, 42 0 (5): 0 3062--3094, 2026. doi:10.1080/10447318....

    2026

  36. [63]

    Investigating cultural alignment of large language models

    AlKhamissi, B., ElNokrashy, M., AlKhamissi, M., and Diab, M. Investigating cultural alignment of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 12404--12422, Bangkok, Thailand, 202...

    2024 doi

  37. [64]

    Amabile, T. M. A model of creativity and innovation in organizations. Research in Organizational Behavior, 10: 0 123--167, 1988

    1988

  38. [65]

    R., Shah, J

    Anderson, B. R., Shah, J. H., and Kreminski, M. Homogenization effects of large language models on human creative ideation. In Proceedings of the 16th Conference on Creativity & Cognition, pp.\ 413--425, 2024. doi:10.1145/3635636.3656204. URL https://doi.org/10.1145/3635636.3656204

    2024

  39. [66]

    Collective constitutional ai: Aligning a language model with public input

    Anthropic . Collective constitutional ai: Aligning a language model with public input. https://www.anthropic.com/research/collective-constitutional-ai-aligning-a-language-model-with-public-input, 2023. Anthropic research post

    2023

  40. [67]

    Atmakuru, A., Nainani, J., Bheemreddy, R. S. R., Lakkaraju, A., Yao, Z., Zamani, H., and Chang, H.-S. CS4 : Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story - Writing Constraints , 2024. URL http://arxiv.org/abs/2410.04197. arX...

    2024 arXiv

  41. [68]

    The case for domain specificity of creativity

    Baer, J. The case for domain specificity of creativity. Creativity Research Journal, 11 0 (2): 0 173--177, 1998

    1998

  42. [69]

    and Kaufman, J

    Baer, J. and Kaufman, J. C. Bridging generality and specificity: The amusement park theoretical (apt) model of creativity. Roeper Review, 27 0 (3): 0 158--163, 2005

    2005

  43. [70]

    We need to consider disagreement in evaluation

    Basile, V., Fell, M., Fornaciari, T., Hovy, D., Paun, S., Plank, B., Poesio, M., and Uma, A. We need to consider disagreement in evaluation. In Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future, pp.\ 15--21, Online, 2021. Association for Computational L...

    2021

  44. [71]

    A., Bengio, Y., and Jerbi, K

    Bellemare-Pepin, A., Lespinasse, F., Th \"o lke, P., Harel, Y., Mathewson, K., Olson, J. A., Bengio, Y., and Jerbi, K. Divergent creativity in humans and large language models. Scientific Reports, 16 0 (1): 0 1279, 2026. doi:10.1038/s41598-025-25157-3. URL https://doi.org/10.1...

    2026 doi

  45. [72]

    Boden, M. A. The creative mind: Myths and mechanisms. Basic Books, 1990

    1990

  46. [73]

    Boden, M. A. Creativity and artificial intelligence. Artificial Intelligence, 103 0 (1): 0 347--356, 1998. doi:10.1016/S0004-3702(98)00055-1. URL https://www.sciencedirect.com/science/article/pii/S0004370298000551

    1998 doi

  47. [74]

    Boden, M. A. The Creative Mind: Myths and Mechanisms. Routledge, 2004

    2004

  48. [75]

    F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T

    Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., M \"u ller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., Leibo, J. Z., McElreath, R., Oudeyer, P.-Y., Stray, J., and Rahwan, I. Machine culture. Nature Human Behaviour, 7: 0 1855--18...

    2023 doi

  49. [76]

    Art or artifice? large language models and the false promise of creativity

    Chakrabarty, T., Laban, P., Agarwal, D., Muresan, S., and Wu, C.-S. Art or artifice? large language models and the false promise of creativity. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pp.\ 1--34. Association for Computing Machinery, 202...

    2024

  50. [77]

    and Ding, N

    Chen, H. and Ding, N. Probing the ``creativity'' of large language models: Can models produce divergent semantic association? In Findings of the Association for Computational Linguistics: EMNLP 2023, pp.\ 12881--12888, Singapore, 2023. Association for Computational Linguistics...

    2023 doi

  51. [78]

    Corazza, G. E. Potential originality and effectiveness: The dynamic definition of creativity. Creativity Research Journal, 28 0 (3): 0 258--267, 2016. doi:10.1080/10400419.2016.1195627

    2016

  52. [79]

    Creativity: Flow and the psychology of discovery and invention

    Csikszentmihalyi, M. Creativity: Flow and the psychology of discovery and invention. HarperCollins, 1996

    1996

  53. [80]

    Diedrich, J., Benedek, M., Jauk, E., and Neubauer, A. C. The novelty--usefulness tension: Toward a dialectical model of creativity. Psychology of Aesthetics, Creativity, and the Arts, 9 0 (4): 0 320--337, 2015. doi:10.1037/aca0000021

    2015 doi

  54. [81]

    A comparative approach to assessing linguistic creativity of large language models and humans

    Dinu, A., Florescu, A.-M., and Resceanu, A. A comparative approach to assessing linguistic creativity of large language models and humans. Procedia Computer Science, 270: 0 1292--1301, 2025. doi:10.1016/j.procs.2025.09.250. URL https://doi.org/10.1016/j.procs.2025.09.250

    2025 doi

  55. [82]

    Doshi, A. R. and Hauser, O. P. Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10 0 (28): 0 eadn5290, 2024. doi:10.1126/sciadv.adn5290. URL https://www.science.org/doi/10.1126/sciadv.adn5290

    2024 doi

  56. [83]

    Large ai models are cultural and social technologies

    Farrell, H., Gopnik, A., Shalizi, C., and Evans, J. Large ai models are cultural and social technologies. Science, 387 0 (6739): 0 1153--1156, 2025. doi:10.1126/science.adt9819

    2025 doi

  57. [84]

    Y., Choi, Y., and Tsvetkov, Y

    Feng, S., Sorensen, T., Liu, Y., Fisher, J., Park, C. Y., Choi, Y., and Tsvetkov, Y. Modular pluralism: Pluralistic alignment via multi- LLM collaboration. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 4151--4171, Miami, Florid...

    2024 doi

  58. [85]

    Overton pluralistic reinforcement learning for large language models, 2026

    Fu, Y., Son, S., and Bogunovic, I. Overton pluralistic reinforcement learning for large language models, 2026. URL https://arxiv.org/abs/2602.20759. arXiv:2602.20759 [cs.LG]

    2026

  59. [86]

    Guilford, J. P. The Nature of Human Intelligence. McGraw-Hill, New York, 1967

    1967

  60. [87]

    What Shapes a Creative Machine Mind ? Comprehensively Benchmarking Creativity in Foundation Models , 2025

    He, Z., Zhang, B., Liu, W., Tang, R., and Cheng, L. What Shapes a Creative Machine Mind ? Comprehensively Benchmarking Creativity in Foundation Models , 2025. URL http://arxiv.org/abs/2510.04009. arXiv:2510.04009 [cs]

    2025

  61. [88]

    J., Zhang, B

    Hou, Z. J., Zhang, B. A., Lu, Y., Baghel, B. K., Brei, A., Lu, X., Jiang, M., Brahman, F., Chaturvedi, S., Chang, H.-S., Khashabi, D., and Li, X. L. CreativityPrism : A Holistic Benchmark for Large Language Model Creativity , 2025. URL http://arxiv.org/abs/2510.20091. arXiv:25...

    2025 arXiv

  62. [89]

    Ismayilzada, M., Stevenson, C., and Plas, L. v. d. Evaluating Creative Short Story Generation in Humans and Large Language Models , 2025. URL http://arxiv.org/abs/2411.02316. arXiv:2411.02316 [cs]

    2025 arXiv

  63. [90]

    LLM Output Homogenization is Task Dependent , 2025

    Jain, S., Lanchantin, J., Nickel, M., Ullrich, K., Wilson, A., and Watson-Daniels, J. LLM Output Homogenization is Task Dependent , 2025. URL http://arxiv.org/abs/2509.21267. arXiv:2509.21267 [cs]

    2025 arXiv

  64. [91]

    R., Whitefield, A., R \"o ttger, P., Bean, A., Margatina, K., Ciro, J., Mosquera-Gomez, R., Bartolo, M., Williams, A., He, H., Vidgen, B., and Hale, S

    Kirk, H. R., Whitefield, A., R \"o ttger, P., Bean, A., Margatina, K., Ciro, J., Mosquera-Gomez, R., Bartolo, M., Williams, A., He, H., Vidgen, B., and Hale, S. A. The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about t...

    2024

  65. [92]

    M., Tilton, L., and Mimno, D

    Klein, L., Martin, M., Brock, A., Antoniak, M., Walsh, M., Johnson, J. M., Tilton, L., and Mimno, D. Provocations from the humanities for generative AI research, 2025. URL https://arxiv.org/abs/2502.19190. arXiv:2502.19190

    2025

  66. [93]

    Kommers, C., Ahnert, R., Antoniak, M., Benetos, E., Benford, S., Bunz, M., Caramiaux, B., Concannon, S., Disley, M., Dobson, J., Du, Y., Due \ n ez-Guzm \'a n, E., Francksen, K., Gius, E., Gray, J. W. Y., Heuser, R., Immel, S., So, R. J., Leigh, S., Livingston, D., Long, H., M...

    2026

  67. [94]

    Human Creativity in the Age of LLMs : Randomized Experiments on Divergent and Convergent Thinking

    Kumar, H., Vincentius, J., Jordan, E., and Anderson, A. Human Creativity in the Age of LLMs : Randomized Experiments on Divergent and Convergent Thinking . In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems , pp.\ 1--18, 2025. doi:10.1145/3706598.3...

    2025

  68. [95]

    Creative or Uncreative Partner : Comparing Humans and AI in Collaborative Creative Tasks , 2025

    Lai, C., Luchini, S., Lauharatanahirun, N., and Beaty, R. Creative or Uncreative Partner : Comparing Humans and AI in Collaborative Creative Tasks , 2025. URL https://osf.io/ey7u4_v1

    2025

  69. [96]

    Lee, B. C. and Chung, J. J. An empirical investigation of the impact of ChatGPT on creativity. Nature Human Behaviour, 8 0 (10): 0 1906--1914, 2024. doi:10.1038/s41562-024-01953-1. URL https://www.nature.com/articles/s41562-024-01953-1. Publisher: Nature Publishing Group

    1906 doi

  70. [97]

    I., Liu, Z., Ferianc, M., Treleaven, P., and Rodrigues, M

    Masoud, R. I., Liu, Z., Ferianc, M., Treleaven, P., and Rodrigues, M. Cultural alignment in large language models: An explanatory analysis based on hofstede's cultural dimensions. In Proceedings of the 31st International Conference on Computational Linguistics, 2025. URL https...

    2025

  71. [98]

    Mednick, S. A. The associative basis of the creative process. Psychological Review, 69 0 (3): 0 220--232, 1962. doi:10.1037/h0048850

    1962 doi

  72. [99]

    Benchmarking distributional alignment of large language models

    Meister, N., Guestrin, C., and Hashimoto, T. Benchmarking distributional alignment of large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.\ 24--49....

    2025 doi

  73. [100]

    Dealing with disagreements: Looking beyond the majority vote in subjective annotations

    Mostafazadeh Davani, A., D \'i az, M., and Prabhakaran, V. Dealing with disagreements: Looking beyond the majority vote in subjective annotations. Transactions of the Association for Computational Linguistics, 10: 0 92--110, 2022. doi:10.1162/tacl_a_00449. URL https://aclantho...

    2022 doi

  74. [101]

    PerSpectra : A scalable and configurable pluralist benchmark of perspectives from arguments

    Nie, S., Omoomi, K., Flek, L., Zhao, Z., and Welch, C. PerSpectra : A scalable and configurable pluralist benchmark of perspectives from arguments. In Proceedings of the International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=dyooGJcKJg....

    2026

  75. [102]

    How much of a pluralist is chatgpt? a comparative study of value pluralism in generative ai chatbots

    Novis-Deutsch, N., Elyoseph, T., and Elyoseph, Z. How much of a pluralist is chatgpt? a comparative study of value pluralism in generative ai chatbots. AI & Society, 2025. URL https://link.springer.com/article/10.1007/s00146-025-02450-3

    2025 doi

  76. [103]

    A., Nahas, J., Chmoulevitch, D., Cropper, S

    Olson, J. A., Nahas, J., Chmoulevitch, D., Cropper, S. J., and Webb, M. E. Naming unrelated words predicts creativity. Proceedings of the National Academy of Sciences, 118 0 (25): 0 e2022340118, 2021. doi:10.1073/pnas.2022340118. URL https://www.pnas.org/doi/10.1073/pnas.2022340118

    2021 doi

  77. [104]

    and He, H

    Padmakumar, V. and He, H. Does writing with language models reduce content diversity? In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Feiz5HtCD0

    2024

  78. [105]

    The ``problem'' of human label variation: On ground truth in data, modeling and evaluation

    Plank, B. The ``problem'' of human label variation: On ground truth in data, modeling and evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp.\ 10671--10682, Abu Dhabi, United Arab Emirates, 2022. Association for Computatio...

    2022

  79. [106]

    Poole-Dayan, E., Wu, J., Sorensen, T., Pei, J., and Bakker, M. A. Benchmarking overton pluralism in llms. In Proceedings of the International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=f2VxF4QIx1. ICLR 2026 poster

    2026

  80. [107]

    and Gurevych, I

    Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp.\...

    2019 doi

  81. [108]

    Runco, M. A. and Jaeger, G. J. The standard definition of creativity. Creativity Research Journal, 24 0 (1): 0 92--96, 2012. doi:10.1080/10400419.2012.650092

    2012

  82. [109]

    Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, volume 202, pp.\ 29971--30004

    Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., and Hashimoto, T. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, volume 202, pp.\ 29971--30004. PMLR, 2023. URL https://proceedings.mlr.press/v202/sa...

    2023

  83. [110]

    Simonton, D. K. Defining creativity: Don't we also need to define the opposite? Creativity Research Journal, 30 0 (3): 0 291--294, 2018. doi:10.1080/10400419.2018.1488195

    2018

  84. [111]

    Mpnet: Masked and permuted pre-training for language understanding

    Song, K., Tan, X., Qin, T., Lu, J., and Liu, T.-Y. Mpnet: Masked and permuted pre-training for language understanding. In Advances in Neural Information Processing Systems. arXiv, 2020. doi:10.48550/arXiv.2004.09297. URL http://arxiv.org/abs/2004.09297. arXiv:2004.09297 [cs.CL]

    2020 doi arXiv

  85. [112]

    L., Mireshghallah, N., Rytting, C

    Sorensen, T., Moore, J., Fisher, J., Gordon, M. L., Mireshghallah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dziri, N., Althoff, T., and Choi, Y. Position: A roadmap to pluralistic alignment. In Proceedings of the 41st International Conference on Machine Learning, volume ...

    2024

  86. [113]

    L., and Brahman, F

    Tian, Y., Ravichander, A., Qin, L., Le Bras, R., Marjieh, R., Peng, N., Choi, Y., Griffiths, T. L., and Brahman, F. Macgyver: Are large language models creative problem solvers? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computat...

    2024 doi

  87. [114]

    N., Fornaciari, T., Hovy, D., Paun, S., Plank, B., and Poesio, M

    Uma, A. N., Fornaciari, T., Hovy, D., Paun, S., Plank, B., and Poesio, M. Learning from disagreement: A survey. Journal of Artificial Intelligence Research, 72: 0 1385--1470, 2021. doi:10.1613/jair.1.12752. URL https://www.jair.org/index.php/jair/article/view/12752

    2021 doi

  88. [115]

    K., and Wilkens, M

    Underwood, T., Nelson, L. K., and Wilkens, M. Can language models represent the past without anachronism?, 2025. URL https://arxiv.org/abs/2505.00030. arXiv:2505.00030

    2025 arXiv

  89. [116]

    and Kenett, Y

    Wenger, E. and Kenett, Y. We're Different , We 're the Same : Creative Homogeneity Across LLMs , 2025. URL http://arxiv.org/abs/2501.19361. arXiv:2501.19361 [cs]

    2025 arXiv

  90. [117]

    A., Yu, T., Kumar, S., Majumder, B

    Xie, Z., Wu, J., Shen, Y., Jain, R., Xia, Y., Li, X., Chang, A., Rossi, R. A., Yu, T., Kumar, S., Majumder, B. P., Shang, J., Ammanabrolu, P., and McAuley, J. J. A survey on personalized and pluralistic preference alignment in large language models. In Proceedings of the Confe...

    2025

  91. [118]

    Self-pluralising culture alignment for large language models

    Xu, S., Leng, Y., Yu, L., and Xiong, D. Self-pluralising culture alignment for large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pape...

    2025

  92. [119]

    H., Milli, S., Jusko, K

    Zhang, L. H., Milli, S., Jusko, K. L., Smith, J., Amos, B., Bouaziz, W., Revel, M., Kussman, J., Sheynin, Y., Titus, L., Radharapu, B., Yu, J., Sarma, V., Rose, K., and Nickel, M. Cultivating pluralism in algorithmic monoculture: The community alignment dataset. In Proceedings...

    2026

  93. [120]

    Noveltybench: Evaluating language models for humanlike diversity

    Zhang, Y., Diddee, H., Holm, S., Liu, H., Liu, X., Samuel, V., Wang, B., and Ippolito, D. Noveltybench: Evaluating language models for humanlike diversity. In Proceedings of the Conference on Language Modeling, 2025. URL https://openreview.net/forum?id=XZm1ekzERf

    2025

  94. [121]

    Assessing and Understanding Creativity in Large Language Models

    Zhao, Y., Zhang, R., Li, W., Huang, D., Guo, J., Peng, S., Hao, Y., Wen, Y., Hu, X., Du, Z., Guo, Q., Li, L., and Chen, Y. Assessing and Understanding Creativity in Large Language Models . Machine Intelligence Research, 22 0 (3): 0 417--436, 2025. doi:10.1007/s11633-025-1546-4...

    2025 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.