Pith. sign in

REVIEW 3 major objections 6 minor 59 references

CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CreativeInstruct claims a single trained LLM can recover the diversity that post-training removes, by learning to emit its own [StartCreativity] spans, and that the result is a better substrate for reinforcement learning.

desk verdict A practical single-model diversity recipe with real gains, but the quality and structural-diversity headlines lean harder on unvalidated LLM judges than the data supports. read the letter →

arxiv 2608.07460 v1 pith:G43TPGEM submitted 2026-08-07 cs.CL cs.AI

Ananya Sahu , Mohit Bansal , Elias Stengel-Eskin This is my paper · ORCID
classification cs.CLcs.AI
keywords creativityoutputdiversityinstructiontuningreinforcementlearninggrapheditdistancenarrativegenerationLLMevaluationpost-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Alignment makes LLMs better at following instructions but also makes their outputs more repetitive and homogeneous. CreativeInstruct's claim is that a single model can recover much of that lost diversity by learning where to be creative: the model is trained on outputs whose creative, base-model-written spans are wrapped in special [StartCreativity] tokens, and at test time it injects those tokens itself. Across five model families and sizes, the paper reports that CreativeInstruct matches or beats two-model decoding (BACo) and distilled variants on semantic, lexical, and structural diversity, while keeping quality scores competitive. It also reports that applying GRPO to a CreativeInstruct checkpoint improves math reasoning by about 4 points on AMC and about 5 points on MATH over the same RL applied to the plain post-trained checkpoint. The reason to care is that diversity is usually treated as a separate objective or a test-time cost; this paper argues it can be taught once, inside a single checkpoint, and that doing so pays off downstream.

What carries the argument

The mechanism is a pair of learned control tokens, [StartCreativity] and [EndCreativity], placed around contiguous spans that a base model contributed to a routed generation during data creation, plus the instruction-tuning corpus built that way. Training data comes from routing each token of an output between base and aligned models by entropy and punctuation, so high-entropy tokens go to the base model; after fine-tuning, the aligned model learns to emit those tokens itself, effectively switching its own style at the right moments. The second machine is LLM-GED, an evaluation method that has an LLM turn each story into a canonical abstract event graph of entities, events, and temporal edges, and then reports the average normalized graph edit distance between pairs; this is the metric that claims to see narrative-level differences that lexical and semantic metrics miss.

What would settle it

Run a human study with enough annotators to get trustworthy quality ratings (for instance, at least five independent ratings per story with pre-agreed rubrics) comparing CreativeInstruct against the Instruct baseline on matched outputs. If human-rated quality is significantly lower for CreativeInstruct, or if human-rated structural diversity does not favor it in line with LLM-GED, the paper's central 'better and more diverse, no quality loss' claim would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that post-training's loss of diversity is not irreversible and does not require a second model at inference. CreativeInstruct says a single aligned LLM can be taught to act as its own router: it learns to insert [StartCreativity]...[/EndCreativity] spans that mark where the base model's more varied, less constrained style should take over, and it does this at generation time on its own. The paper further claims this is not just distillation of the routing teacher: the trained model outperforms the two-model BACo baseline that generated its training data on most diversity metrics. The same mechanism, the paper argues, makes a better substrate for RL, because more diverse rollouts give GRPO more to explore; the evidence is a roughly 4-point gain on AMC and 5-point gain on MATH when GRPO is run on CreativeInstruct rather than on the Instruct checkpoint.

Load-bearing premise

The load-bearing premise is that the automated LLM-based judge used for quality and structural diversity (plus the writing-quality reward model) measures what the paper claims it measures; the paper's own human quality ratings were too unreliable to use (kappa = -0.167), so 'without sacrificing quality' rests entirely on automatic metrics.

Editorial extensions

If this is right

  • At inference, diversity no longer costs twice the compute: a single CreativeInstruct checkpoint replaces the base-plus-aligned pair that created its training data, and the paper reports it beats that pair on most diversity metrics.
  • The training recipe is scalable from general-purpose instruction data: gains on semantic and structural diversity keep rising as the training set grows from 1,000 to 12,000 samples, with no plateau at the largest size.
  • The [StartCreativity] and [EndCreativity] tags matter for the method: the Distill baseline trained on the same data without the tags has lower quality and structural diversity, so the tags themselves carry the balance.
  • Creative instruction-tuned models give RL better exploration: GRPO on the CreativeInstruct checkpoint beats GRPO on the Instruct checkpoint by roughly 4 points on AMC and 5 points on MATH, with the AMC gain concentrated at higher difficulty.
  • Proper-noun uniqueness roughly doubles over the Instruct model (37.1% versus 18.1% per prompt group), a concrete sign that repetitive characters and locations are reduced.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension suggested by Qwen3 32B's cross-family transfer: if the creativity-tagging recipe works when no base model exists for the target family, the data-generation stage could in principle be decoupled from the model being trained, letting creativity be distilled across generations of models.
  • The [StartCreativity] tokens can be read as a discrete, interpretable switch; one could verify this by ablating the tokens at generation time and checking whether diversity collapses to the aligned model's level, which would show the model is genuinely routing through its own tags.
  • If LLM-GED is accepted as a structural-diversity measure, it could be reused as a cheap mode-collapse audit for any open-ended generation system, not just stories; the paper's validation, however, only covers 50 synthetic story variations, so that reuse would need independent testing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes CreativeInstruct, an instruction-tuning method that builds creativity-tagged training data by routing between a base model and its aligned counterpart (using BACo), then fine-tunes a single aligned model with special [StartCreativity]/[EndCreativity] tokens so that the model learns to inject creative spans at inference time. The paper also introduces an LLM-based graph-edit-distance metric (LLM-GED) intended to capture structural narrative diversity. Experiments on the Narrative Discourse dataset across five model families (LLaMA-3.1 8B, Qwen2.5 7B/32B, Qwen3 8B/32B) report diversity improvements over Instruct, BACo, a tag-ablated Distill baseline, and (for LLaMA-3.1 8B) CrPO, along with automatic quality scores and a writing-quality reward model. A human study finds a 70.3% preference for CreativeInstruct on creativity, but quality judgments were dropped due to negative inter-annotator agreement. Finally, GRPO applied to a CreativeInstruct Qwen3 8B checkpoint improves MATH and AMC accuracy relative to the same RL recipe applied to the base instruct checkpoint.

Significance. If the claims hold, CreativeInstruct would be a practically valuable single-model alternative to multi-model inference-time routing, and the RL result would demonstrate a useful connection between generation diversity and downstream reasoning. The paper has several concrete strengths: the Distill ablation isolates the contribution of the creativity tokens; the RL comparison uses an identical training recipe for both starting points; the evaluation spans multiple model families and sizes; and the code is released. The main open question is whether the two headline conclusions survive stronger measurement: the "without sacrificing quality" claim is contradicted by parts of Table 2 and rests on automatic metrics after the human quality annotation was dropped, and the structural-diversity metric LLM-GED is validated largely against synthetic transformations and a deterministic pipeline that encodes a similar structural assumption.

major comments (3)
  1. [Abstract and Section 4.1, Table 2] The abstract's claim that CreativeInstruct matches or exceeds diversity 'without sacrificing quality' is not supported by Table 2. For Qwen3 32B, CreativeInstruct is lower than the Instruct baseline on all four reported quality measures (coherence 4.36 vs. 4.45, fluency 4.14 vs. 4.85, relevance 4.01 vs. 4.76, WQRM 6.32 vs. 7.39). For Qwen2.5 7B, coherence, fluency, and relevance all decrease relative to Instruct (3.82 vs. 4.18, 4.02 vs. 4.62, 3.84 vs. 4.26). The Section 4.1 wording 'generally maintains competitive performance' is more accurate; the abstract should be revised to match the evidence, or the authors should add a human quality evaluation that supports the stronger claim.
  2. [Section 4.2 and Abstract] The abstract states that the results 'are mirrored in our human evaluation,' but the human evaluation in Section 4.2 explicitly omits quality judgments because inter-annotator agreement on quality was negative (kappa = -0.167). Table 5 reports preferences only for diversity and creativity. Consequently, the human evaluation supports the creativity-preference claim but cannot be cited as evidence for the 'without sacrificing quality' part of the central claim. Please either reword the abstract and Section 4.2 to separate the supported claims from the unsupported one, or collect human quality judgments with a revised annotation protocol.
  3. [Section 3.2 and Appendices B.1/B.2] LLM-GED is the headline metric for the structural-diversity gains, but its validation is not yet strong enough to carry that weight. Table 9 reports correlation against a hand-authored ordering of four synthetic transformation types (same, lexical, temporal, different), which is not a substitute for correlation with human judgments of structural diversity on real model outputs. Appendix B.2's equivalence check uses a paired t-test on 50 stories (mean difference -0.012, p = 0.56); failure to reject the null is not evidence of equivalence, and 50 stories is too small for high power. Moreover, the deterministic pipeline already treats substitution as free for event nodes sharing the same coarse narrative role (observation, quote, reflection), and the LLM-GED prompt instructs the judge to canonicalize synonyms and paraphrases while penalizing temporal reorderings; agreement between the two therefore shows internal consistency of assumptions, not unbiased measurement. I would ask for an independent validation of LLM-GED against human structural-diversity ratings on real generated stories, and for a confidence interval or TOST-style analysis for the pipeline comparison, before relying on the 63% relative LLM-GED gain for LLaMA-3.1 8B.
minor comments (6)
  1. [Section 4.3] The sentence 'the AM performance actually drops slightly' should read 'the AMC performance'.
  2. [Figure 6 caption] The caption contains the typo 'Anotation interface'; it should be 'Annotation interface'.
  3. [Section 4.1 and Table 1] The text says CreativeInstruct 'consistently' improves diversity, but in Table 1 the Qwen3 8B MiniLM cosine dissimilarity is lower for CreativeInstruct (0.369) than for Instruct (0.399), and for Qwen3 32B the Distill baseline outperforms CreativeInstruct on several semantic metrics. Please qualify 'consistently' as 'in most settings and on most metrics'.
  4. [Section 4.1, 'Importance of Data Diversity'] The claim that general-purpose Tülu data outperforms in-domain data 'even at the same data set size' is not quite supported: the in-domain variant uses 2,020 samples while the nearest Tülu comparison uses 2,000 samples, and the prompt sets differ. Clarify the comparison or match the datasets more carefully.
  5. [Tables 1 and 2] No significance tests or confidence intervals are reported for the diversity or automatic quality metrics. At least for the headline LLM-GED and the quality metrics that support the 'no sacrifice' claim, error bars or significance tests would help the reader assess stability across prompts and sampling.
  6. [Section 4.2, Table 5] The significance asterisk is shown only for Creativity (70.3%). Please state explicitly whether the Diversity win rate (57.4%) is not statistically significant, and report the inter-annotator agreement for the diversity and creativity judgments on the 14-prompt subset.

Circularity Check

2 steps flagged · score 4.0 of 10

LLM-GED validation is self-confirming: the metric's prompt encodes the same ranking used as 'ground truth'; the main training and RL comparisons are otherwise empirically self-contained.

  1. self definitional [Section 3.2 / Appendix B.1 / Appendix E.2 (LLM-GED prompt and validation)]
    "Temporal structure: - Events are ordered via directed next_event edges - Reordering events requires deleting and reinserting next_event edges - Temporal order cannot be changed via relabeling alone. Canonicalization: - Synonyms and paraphrases must not count as relabel operations - Only structural or semantic role changes count as edits. ... We hypothesize an ideal metric for measuring narrative diversity will rank same stories as the least diverse, lexical shifts as slightly more diverse, temporal shifts as more diverse and different stories entirely as most diverse."

    The validation ranking used as 'ground truth' in Appendix B.1 is already built into the LLM-GED prompt: the judge is instructed that paraphrases are zero-cost and that temporal reordering requires edge insertions/deletions. The four test categories are exactly Same, Lexical, Temporal, and Different, so a judge that follows its own instructions must produce the hypothesized ordering. The reported high correlation (0.889) therefore measures self-consistency with the metric's definition, not independent evidence that LLM-GED captures narrative diversity. This makes the metric's validation circular, though it does not by itself force the model-comparison results, which also rely on semantic and lexical metrics.

  2. other [Appendix B.2 (LLM-GED vs deterministic GED pipeline)]
    "Node substitution cost is zero for nodes of matching type; for event nodes, substitution is additionally free only when the two events share the same coarse narrative role (e.g., observation, quote, reflection), so that paraphrased but structurally equivalent events are not penalized. ... Overall the unified single-pass LLM-based GED metric and the pipeline GED computation produced statistically equivalent mean diversity scores across the 50 generated stories (mean difference = -0.012, paired t-test p = 0.56)."

    The deterministic pipeline used to 'validate' LLM-GED is built from the same story-graph extraction prompt (Appendix E.1) and the same coarse-role substitution rule that implements the LLM-GED prompt's instruction that paraphrases must not count as relabel operations. Both systems therefore embed the identical invariance assumption about what counts as narrative structure. Agreement between them is an internal consistency check between two implementations of the same construct, not an external validation. The equivalence claim on 50 short stories (p=0.56) cannot independently establish that LLM-GED measures true narrative diversity on real model outputs.

full rationale

Overall, the CREATIVEINSTRUCT derivation chain is not circular: training data are produced by BACo routing, the model is fine-tuned with [StartCreativity] tags, and it is compared against Instruct, BACo, Distill, and CrPO on held-out Narrative Discourse prompts; these are external empirical comparisons. The GRPO experiment is likewise an independent matched comparison. The human evaluation limitation (quality kappa = -0.167) is disclosed by the paper and shifts quality measurement to automatic proxies, which is a proxy-validity issue, not circularity. The only self-reference that could be questioned is the citation of the authors' own Cog-drift work for the RL-diversity premise, but the premise is also supported by other cited works and is tested empirically in Table 6, so it is not load-bearing. The specific reduction I find is in the validation of LLM-GED: the metric's prompt encodes the exact same narrative-diversity ordering used as 'ground truth' in Appendix B.1, and the deterministic pipeline in B.2 shares the same canonicalization and substitution-cost assumptions. Thus the claimed correlation and equivalence are partly by construction. Because semantic/lexical metrics and the human creativity judgment independently support the main claim, this is partial, not global, circularity; hence score 4.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

CreativeInstruct's claims rest on (1) BACo's routed spans being a valid creativity signal, (2) LLM-GED and GPT5-mini quality scores being valid measurements, and (3) the learned tokens, not the larger training corpus, causing the gains. The Distill ablation partially supports (3), but the metric assumptions are weakly validated and quality scores conflict with the abstract for two model families.

free parameters (2)
  • creativity_span_delta = 0.005
    Threshold used in Section 3.1 to also mark base/aligned tokens with near-equal probabilities as creativity spans; hand-set, changes which spans get tagged.
  • BACo prob+punc entropy routing threshold = not stated (from Wang et al. 2026)
    The data-generation pipeline routes high-entropy tokens to the base model and low-entropy tokens to the aligned model; the exact threshold is not given in this paper, so the training corpus cannot be reconstructed from this text alone.
assumptions (4)
  • domain assumption BACo token-level entropy and punctuation routing identifies token spans whose base-model provenance marks creativity, and these spans are the right supervision signal.
    Section 3.1; central to data construction. No analysis of whether base-model spans correspond to independently judged creative segments.
  • domain assumption Base models are more diverse and creative than aligned models of the same family.
    Section 2, citing West and Potts 2025 and Wang et al. 2026. This is an empirical premise inherited from prior work, not established in this paper.
  • domain assumption An LLM judge (GPT5-mini) can compute graph edit distances on abstract event graphs that correspond to narrative diversity.
    Section 3.2; validated only against synthetic shifts and a small deterministic pipeline, not against human structural-diversity judgments.
  • domain assumption The model will emit learned [StartCreativity] and [EndCreativity] tokens at inference in a way that causally produces the diversity gains.
    Section 3.1; no analysis of token emission rates and no ablation that removes the special tokens at inference time.
invented entities (1)
  • [StartCreativity] and [EndCreativity] special tokens
    purpose: Mark spans where the model should switch from aligned, instruction-following generation to base-model-like creative generation.
    The Distill ablation (same data without tags) is internal evidence that the tags matter, but there is no independent, external falsifiable handle on the token mechanism, and the paper does not report how often the model emits the tokens at inference.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity." pith.science (2026). https://pith.science/paper/G43TPGEM

@misc{pith2026260807460,
  author       = {Pith},
  title        = {Pith review of: CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G43TPGEM}},
  note         = {Machine review of arXiv:2608.07460}
}
read the original abstract

While post-training improves the capabilities of large language models (LLMs), it generally lowers their output diversity and creativity, negatively impacting tasks that explicitly require creativity (e.g., story generation) as well as those that require it implicitly, e.g., reinforcement learning (RL). We instead propose CreativeInstruct, a scalable instruction-tuning method that teaches LLMs to balance creative, base-model-like generations with the quality of post-trained models, by learning to inject special [StartCreativity] spans that bias generation toward creativity. Furthermore, we introduce a structural diversity metric based on graph edit distance, which captures narrative level variation missed by purely lexical and semantic metrics. On narrative generation, CreativeInstruct matches or exceeds the diversity of both multi-model baselines and distilled variants of their outputs, without sacrificing quality or requiring multiple models at inference time. These results are mirrored in our human evaluation, where we find that annotators rate CreativeInstruct generations as more creative than the post-trained LLMs' generations in 70.3% of cases. We also show the benefits of creative models as a substrate for RL: GRPO applied to a CreativeInstruct checkpoint improves by ~4% on AMC and ~5% points on MATH over the same training applied to the post-trained checkpoint.

Figures

Figures reproduced from arXiv: 2608.07460 by the authors.

Figure 1
Figure 1. Instruction-tuned models succeed in following [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. CREATIVEINSTRUCT operates by first creating a dataset of creativity-tagged instruction-tuning data. For each prompt, token generation is routed between base and aligned model variants based on token entropy and whether the token represents a sentence boundary, following Wang et al. (2026). Contiguous spans of tokens generated from the base model are then marked with new [Start-] and [EndCreativity] tokens, and the a… view at source ↗
Figure 3
Figure 3. Scaling synthetic training data improves se [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Out-of-domain (OOD) accuracy as a function [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Instructions given to annotators [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Anotation interface. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 24 canonical work pages

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  2. [2]

    Publications Manual , year = "1983", publisher =

    1983

  3. [3]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    arXiv 1981

  4. [4]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  5. [5]

    Dan Gusfield , title =. 1997

    1997

  6. [6]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  7. [7]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  8. [8]

    Adacad: Adaptively decoding to balance conflicts between contextual and parametric knowledge , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    work page 2025

Show all 59 references
  1. [9]

    arXiv preprint arXiv:2407.13248 , year=

    Are large language models capable of generating human-level narratives? , author=. arXiv preprint arXiv:2407.13248 , year=

    arXiv

  2. [10]

    Applied Sciences , volume=

    Evaluating creativity: can LLMs be good evaluators in creative writing tasks? , author=. Applied Sciences , volume=. 2025 , publisher=

    2025

  3. [11]

    arXiv preprint arXiv:2510.09913 , year=

    Don't Throw Away Your Pretrained Model , author=. arXiv preprint arXiv:2510.09913 , year=

  4. [12]

    2024 , email =

    Tülu 3: Pushing Frontiers in Open Language Model Post-Training , author =. 2024 , email =

    2024

  5. [13]

    arXiv preprint arXiv:2509.26209 , year=

    Diversity-incentivized exploration for versatile reasoning , author=. arXiv preprint arXiv:2509.26209 , year=

  6. [14]

    arXiv preprint arXiv:2505.00047 , year=

    Base models beat aligned models at randomness and creativity , author=. arXiv preprint arXiv:2505.00047 , year=

    arXiv

  7. [15]

    ICML , year=

    Optimizing Diversity and Quality through Base-Aligned Model Collaboration , author=. ICML , year=

  8. [16]

    One fish, two fish, but not the whole sea: Alignment reduces language models’ conceptual diversity, 2024 , author=

    2024

  9. [17]

    arXiv preprint arXiv:2510.22954 , year=

    Artificial hivemind: The open-ended homogeneity of language models (and beyond) , author=. arXiv preprint arXiv:2510.22954 , year=

  10. [18]

    arXiv preprint arXiv:2501.19361 , year=

    We're different, we're the same: Creative homogeneity across LLMs , author=. arXiv preprint arXiv:2501.19361 , year=

    arXiv

  11. [19]

    arXiv preprint arXiv:2106.09685 , year=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. arXiv preprint arXiv:2106.09685 , year=

    arXiv

  12. [20]

    arXiv preprint arXiv:2303.16634 , volume=

    G-eval: Nlg evaluation using gpt-4 with better human alignment, 2023 , author=. arXiv preprint arXiv:2303.16634 , volume=

    2023 arXiv

  13. [21]

    arXiv preprint arXiv:2504.07532 , year=

    Ai-slop to ai-polish? aligning language models through edit-based writing rewards and test-time computation , author=. arXiv preprint arXiv:2504.07532 , year=

    arXiv

  14. [22]

    arXiv e-prints , pages=

    Cog-DRIFT: Exploration on Adaptively Reformulated Instances Enables Learning from Hard Reasoning Problems , author=. arXiv e-prints , pages=

  15. [23]

    arXiv preprint arXiv:2507.19672 , year=

    Alignment and safety in large language models: Safety mechanisms, training paradigms, and emerging challenges , author=. arXiv preprint arXiv:2507.19672 , year=

    arXiv

  16. [24]

    arXiv preprint arXiv:2605.22817 , year=

    Vector Policy Optimization: Training for Diversity Improves Test-Time Search , author=. arXiv preprint arXiv:2605.22817 , year=. 2605.22817 , archivePrefix=

    arXiv

  17. [25]

    Advances in Neural Information Processing Systems , year=

    Diversity-aware policy optimization for large language model reasoning , author=. Advances in Neural Information Processing Systems , year=

  18. [26]

    arXiv preprint arXiv:2510.01171 , year=

    Verbalized sampling: How to mitigate mode collapse and unlock llm diversity , author=. arXiv preprint arXiv:2510.01171 , year=

    arXiv

  19. [27]

    arXiv preprint arXiv:2505.14442 , year=

    Creative preference optimization , author=. arXiv preprint arXiv:2505.14442 , year=

  20. [28]

    URL https://arxiv

    Modifying large language model post-training for diverse creative writing, 2025 , author=. URL https://arxiv. org/abs/2503.17126 , year=

    2025 arXiv

  21. [29]

    arXiv preprint arXiv:2511.09310 , year=

    LiteraryTaste: A Preference Dataset for Creative Writing Personalization , author=. arXiv preprint arXiv:2511.09310 , year=

  22. [30]

    arXiv preprint arXiv:2604.02319 , year=

    No Single Best Model for Diversity: Learning a Router for Sample Diversity , author=. arXiv preprint arXiv:2604.02319 , year=

    arXiv

  23. [31]

    Creative planning with language models: Practice, evaluation and applications , author=. Proceedings of the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 5: Tutorial Abstracts...

    2025

  24. [32]

    PNAS nexus , volume=

    Large language models are homogeneously creative , author=. PNAS nexus , volume=. 2026 , publisher=

    2026

  25. [33]

    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages=

    Universal decompositional semantics on universal dependencies , author=. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages=

    2016

  26. [34]

    2003 , publisher=

    Form and meaning in language , author=. 2003 , publisher=

    2003

  27. [35]

    Structure , volume=

    Assessing creativity: the importance of unexpected novelty , author=. Structure , volume=

  28. [36]

    Stanford Humanities Review , volume=

    Creativity and unpredictability , author=. Stanford Humanities Review , volume=

  29. [37]

    Psychosociological Issues in Human Resource Management , volume=

    Increase diversity to boost creativity and enhance problem solving , author=. Psychosociological Issues in Human Resource Management , volume=. 2016 , publisher=

    2016

  30. [38]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    arXiv

  31. [39]

    5-math technical report: Toward mathematical expert model via self-improvement , author=

    Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement , author=. arXiv preprint arXiv:2409.12122 , year=

    arXiv

  32. [40]

    International Conference on Learning Representations , volume=

    Understanding the effects of rlhf on llm generalisation and diversity , author=. International Conference on Learning Representations , volume=

  33. [41]

    arXiv preprint arXiv:2501.18101 , year=

    Diverse preference optimization , author=. arXiv preprint arXiv:2501.18101 , year=

    arXiv

  34. [42]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  35. [43]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    arXiv

  36. [44]

    ArXiv , year=

    Qwen2.5 Technical Report , author=. ArXiv , year=

  37. [45]

    Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    A diversity-promoting objective function for neural conversation models , author=. Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    2016

  38. [46]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    Rethinking and refining the distinct metric , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

  39. [47]

    Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation , pages=

    Jointly measuring diversity and quality in text generation models , author=. Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation , pages=

  40. [48]

    ROUGE : A Package for Automatic Evaluation of Summaries

    Lin, Chin-Yew. ROUGE : A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out. 2004

    2004

  41. [49]

    arXiv preprint arXiv:2210.02410 , year=

    The vendi score: A diversity evaluation metric for machine learning , author=. arXiv preprint arXiv:2210.02410 , year=

    arXiv

  42. [50]

    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    Semantic diversity in dialogue with natural language inference , author=. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    2022

  43. [51]

    arXiv preprint arXiv:2302.09664 , year=

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation , author=. arXiv preprint arXiv:2302.09664 , year=

    arXiv

  44. [52]

    von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin , license =

  45. [53]

    arXiv preprint arXiv:2605.30021 , year=

    Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs , author=. arXiv preprint arXiv:2605.30021 , year=

    arXiv

  46. [54]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Understanding the repeat curse in large language models from a feature perspective , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  47. [55]

    Intelligence , volume=

    Creative reasoning across developmental levels: Convergence and divergence in problem creation , author=. Intelligence , volume=. 2012 , publisher=

    2012

  48. [56]

    1998 , publisher=

    Reinforcement learning: An introduction , author=. 1998 , publisher=

    1998

  49. [57]

    URL https://arxiv

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024 , author=. URL https://arxiv. org/abs/2402.03300 , volume=

    2024 arXiv

  50. [58]

    4th International Conference on Pattern Recognition Applications and Methods 2015 , year=

    An exact graph edit distance algorithm for solving pattern recognition problems , author=. 4th International Conference on Pattern Recognition Applications and Methods 2015 , year=

    2015

  51. [59]

    2020 , publisher=

    spaCy: Industrial-strength natural language processing in Python , author=. 2020 , publisher=

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.