Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories
Pith reviewed 2026-06-29 18:50 UTC · model grok-4.3
The pith
Small preference datasets cause 88.3 percent of LLM stories to reuse the same eleven words.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Sampling 20,000 stories from four current models using five prompts reveals that 11 words occur in 88.3% of generated stories, with little difference between models. These words include names (Elias, Mara, Elara), settings (lighthouses), and professions (clockmaker, librarian). These tokens do not often occur in published literature nor pre-training data, but they are found in preference data that is likely to have been used by all current models. Surprisingly, these "lighthouse" stories are infrequent when compared with the average post-training story, much of which contains references to copyrighted characters or adult content. This result demonstrates the potentially disproportionate impa
What carries the argument
The mechanism by which small preference datasets introduce repeated tokens into post-trained models, overriding patterns from pre-training data.
If this is right
- Models from different providers converge on the same repeated tokens because they draw from overlapping preference data.
- Alignment algorithms amplify the frequency of items that are rare in pre-training but present in small preference sets.
- Post-training outputs exhibit lower diversity than would be predicted from pre-training data alone.
- The lighthouse pattern appears less often than copyrighted or adult content in typical aligned model outputs.
Where Pith is reading between the lines
- Curating larger and more varied preference datasets could reduce repetition in open-ended generation tasks.
- The same mechanism may produce repeated tropes in other creative outputs such as poetry or role-play dialogue.
- Controlled experiments that swap preference datasets between models would isolate the contribution of alignment data.
- Monitoring for similar low-diversity clusters could help identify unintended effects of alignment in non-story domains.
Load-bearing premise
The five prompts and four models produce outputs representative of typical LLM story generation, and the identified words are verifiably rare in pre-training data while common in the relevant preference datasets.
What would settle it
Counting the eleven words in the actual preference datasets used for these models and finding them absent or rare, or training a model on preference data that excludes those words and observing repetition rates below 20 percent.
Figures
read the original abstract
LLM-generated stories are a popular use case, but they show very low variability. We sample 20,000 total stories from four current models using five prompts. We find that 11 words occur in 88.3% of generated stories, with little difference between models. These words include names (Elias, Mara, Elara), settings (lighthouses), and professions (clockmaker, librarian). These tokens do not often occur in published literature nor pre-training data, but they are found in preference data that is likely to have been used by all current models. Surprisingly, these "lighthouse" stories are infrequent when compared with the average post-training story, much of which contains references to copyrighted characters or adult content. This result demonstrates the potentially disproportionate impact of small datasets combined with powerful alignment algorithms.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that LLM-generated stories exhibit strikingly low diversity: across 20,000 stories sampled from four current models using five prompts, 11 specific tokens (names such as Elias, Mara, Elara; the setting 'lighthouse'; professions such as clockmaker and librarian) appear in 88.3% of outputs, with little model-to-model variation. The authors argue these tokens are rare in published literature and pre-training corpora yet common in preference data used for alignment, and conclude that small preference datasets combined with powerful alignment algorithms can produce disproportionate effects on creative output.
Significance. If the frequency counts and the attribution to preference data hold after verification, the result would be significant for post-training research: it supplies concrete evidence that alignment can systematically suppress lexical diversity even when the underlying base models differ. The 20,000-story sample size is a clear strength, providing reliable empirical support for the reported 88.3% figure and enabling direct comparison across models.
major comments (2)
- [Abstract] Abstract: the central causal claim—that the 11 tokens 'do not often occur in published literature nor pre-training data, but they are found in preference data'—is asserted without any frequency counts, dataset citations, or verification procedure. This assertion is load-bearing for the conclusion that the phenomenon results from 'small datasets combined with powerful alignment algorithms'; absent the supporting data, the link between observation and explanation remains unestablished.
- [Abstract] Abstract / implied methods: the five prompts and four models are presented as representative of 'typical LLM story generation,' yet no selection criteria, prompt wording, model versions, or sampling protocol are supplied. If the prompts are atypical or the models share undisclosed training overlap, the 88.3% dominance cannot be generalized, undermining the claim that the pattern is a general feature of current LLMs.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback, which highlights areas where additional empirical support and methodological transparency are needed. We address each major comment below and commit to revisions that strengthen the manuscript without altering its core findings.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central causal claim—that the 11 tokens 'do not often occur in published literature nor pre-training data, but they are found in preference data'—is asserted without any frequency counts, dataset citations, or verification procedure. This assertion is load-bearing for the conclusion that the phenomenon results from 'small datasets combined with powerful alignment algorithms'; absent the supporting data, the link between observation and explanation remains unestablished.
Authors: We agree that the abstract presents this attribution without accompanying quantitative evidence or citations, leaving the causal link to preference data insufficiently supported. The full manuscript contains qualitative discussion of token rarity but lacks the explicit frequency tables, dataset references, and verification steps required to substantiate the claim. In the revision we will add a concise methods subsection (and update the abstract) that reports relative frequencies drawn from samples of published literature (e.g., Project Gutenberg), pre-training corpora (e.g., The Pile), and publicly documented preference datasets, together with the exact procedure used to identify the 11 tokens in those sources. This will make the link between observation and explanation verifiable. revision: yes
-
Referee: [Abstract] Abstract / implied methods: the five prompts and four models are presented as representative of 'typical LLM story generation,' yet no selection criteria, prompt wording, model versions, or sampling protocol are supplied. If the prompts are atypical or the models share undisclosed training overlap, the 88.3% dominance cannot be generalized, undermining the claim that the pattern is a general feature of current LLMs.
Authors: We concur that the abstract and methods description omit the information needed to evaluate representativeness. The manuscript identifies four current models and five prompts but supplies neither the precise wording, version numbers, selection rationale, nor sampling hyperparameters. In the revised version we will expand the methods section to list the exact prompt texts, model identifiers and release versions, temperature/top-p settings, and the criteria used to select these models and prompts as representative of frontier LLM story-generation practice. This addition will allow readers to assess potential overlap and generalizability directly. revision: yes
Circularity Check
Empirical frequency counts from sampled outputs; no derivation or self-referential reduction
full rationale
The paper reports direct empirical observations: sampling 20,000 stories from four models with five prompts, then counting token frequencies (11 words in 88.3% of outputs). No equations, fitted parameters, predictions derived from inputs, or load-bearing self-citations appear in the provided text. The claim that the tokens are rare in pre-training data but present in preference data is asserted without reduction to the paper's own fitted quantities or prior self-citations; it is an external empirical assertion (even if verification details are limited). The central result is therefore self-contained as a frequency measurement and does not reduce to its inputs by construction.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
John Joon Young Chung, Vishakh Padmakumar, Melissa Roemmele, Yuqian Sun, and Max Kreminski
Latent dirichlet allocation.Journal of machine Learning research, 3(Jan):993–1022. John Joon Young Chung, Vishakh Padmakumar, Melissa Roemmele, Yuqian Sun, and Max Kreminski. 2025. Modifying Large Language Model Post-Training for Diverse Creative Writing. Anil R. Doshi and Oliver P. Hauser. 2024. Generative AI enhances individual creativity but reduces th...
-
[2]
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu
Optimal detection of changepoints with a lin- ear computational cost.Journal of the American Statistical Association, 107(500):1590–1598. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2024. Understanding the Ef- fects of RLHF on LLM Generalisation and Diversity. InICLR 2024...
2024
-
[3]
Training language models to follow instructions with human feedback
Multi-Novelty: Improve the Diversity and Novelty of Contents Generated by Large Language Models via inference-time Multi-Views Brainstorm- ing. Franco Moretti. 2000. The Slaughterhouse of Literature. Modern Language Quarterly, 61(1):207–228. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarw...
work page internal anchor Pith review Pith/arXiv arXiv 2000
-
[4]
Who are the characters in the text?
-
[5]
What is the character's role in the text?,→
-
[6]
What is the setting? Return JSON only in this exact schema: {{ "character_names": ["first name only", ...],,→ "settings": ["place or location noun phrase", ...],,→ "professions": ["profession or stable role noun phrase", ...],→ }} Additional rules: - For`character_names`, include only first names for named human or human-like characters. ,→ ,→ - For`profe...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.