Pith. sign in

REVIEW 3 major objections 5 minor 28 references

Improving Neural Story Generation by Targeted Common Sense Grounding

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By ranking sensible completions above spurious ones, a GPT-2 pipeline improves common sense and reaches the best-reported perplexity on WritingPrompts.

desk verdict A simple no-extra-parameter common-sense grounding objective for GPT-2 story generation; the transfer evidence is real but thinner than the abstract suggests. read the letter →

arxiv 1908.09451 v2 pith:3I4QO72L submitted 2019-08-26 cs.LG cs.CLstat.ML

Huanru Henry Mao , Bodhisattwa Prasad Majumder , Julian McAuley , Garrison W. Cottrell This is my paper · ORCID
classification cs.LGcs.CLstat.ML
keywords storygenerationcommonsensereasoningmulti-tasklearningtransferperplexityrankingGPT-2WritingPrompts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that next-token perplexity alone is not enough to stop neural story generators from writing nonsense such as "a sunny, warm summer night," and that a direct ranking signal for sensible text can close part of the gap. The authors propose a multi-task fine-tuning scheme: a pretrained GPT-2 is first adapted to BookCorpus, then trained on WritingPrompts while alternating with a perplexity-ranking objective on SWAG and on synthetic pairs of human-written versus GPT-2-generated text. The ranking objective makes the model assign higher length-normalized probability to the sensible completion than to spurious alternatives, reusing the language model's own softmax and adding no parameters. On WritingPrompts the pipeline reaches the lowest reported perplexity, and on zero-shot Story Cloze it improves from 59.1% to 64.1% accuracy, which the paper reads as evidence that common-sense grounding transfers to story completion. If correct, this gives a cheap, task-agnostic way to bias generative models toward more sensible outputs without sacrificing fluency.

What carries the argument

The central object is a perplexity-ranking objective: for candidate sequences $S_1,\ldots,S_N$, with $S_1$ the sensible continuation, the model maximizes the softmax probability of $S_1$ over the length-normalized log probabilities $\frac{1}{T_i}\log P(S_i)$ (Equation 2). This reuses the language model's softmax layer and adds no parameters. The objective is wrapped in a two-stage pipeline: intermediate fine-tuning on BookCorpus adapts a pretrained GPT-2 to narrative text, then multi-task fine-tuning alternates the WritingPrompts language-model loss with ranking on SWAG and on 250K synthetic pairs of human WebText excerpts and GPT-2 samples. The ranking mechanism is what carries the common-sense improvement, and the synthetic pairs expand it beyond the four-choice SWAG format.

What would settle it

Run the full pipeline with the synthetic pairs relabeled so the model is trained to prefer GPT-2-written text over human WebText excerpts; if zero-shot SWAG or Story Cloze accuracy still rises by the same margin, the common-sense gain is not caused by the sensible/spurious distinction the paper claims. Since the reported Story Cloze numbers have no error bars, repeating the comparison over several seeds would also settle whether the 5-point gain is signal or noise.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is that common sense can be operationalized for a generative model as a ranking constraint: among several plausible continuations, the model should assign the highest probability to the one a human would call sensible. Training with this constraint—a softmax over length-normalized log probabilities of candidate sequences—biases a pretrained GPT-2 toward text with fewer logical inconsistencies. Combined with an intermediate BookCorpus fine-tuning stage, the method yields sub-word perplexity 20.78 on WritingPrompts, estimated word-level perplexity 29.63, prompt-ranking accuracy 80.1%, and zero-shot common-sense accuracies of 76.3% on SWAG and 64.1% on Story Cloze, improving over the same model without the auxiliary objectives. The paper's own caveat is that this supplies short-term common-sense grounding, not story-level coherence.

Load-bearing premise

The load-bearing assumption is that human-written text is more sensible than text sampled from a neural language model; if that ordering fails for the specific WebText and GPT-2 samples used, the ranking objective will optimize style or domain differences instead of common sense.

Editorial extensions

If this is right

  • Two-stage fine-tuning (BookCorpus, then WritingPrompts) alone beats the Fusion Model's perplexity and prompt-ranking accuracy, so task-specific story-generation architectures are unnecessary once a pretrained model is available.
  • Adding multi-task perplexity ranking raises zero-shot Story Cloze accuracy from 59.1% to 64.1% and SWAG from 51.4% to 76.3%, while WritingPrompts perplexity stays effectively unchanged.
  • The auxiliary loss reuses the existing softmax layer, so the common-sense gain costs no additional parameters and only extra training epochs on auxiliary data.
  • The synthetic human-versus-GPT-2 pairs provide a further common-sense boost beyond SWAG alone, and the paper's qualitative samples show more grounded story completions with both auxiliary tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same parameter-free ranking loss could be applied to dialogue, summarization, or instruction-following models by sampling degraded outputs from the model itself and ranking preferred outputs above them; the paper does not test these settings.
  • Inference: because the paper's own conclusion notes that SWAG offers only short-term grounding, story-level coherence is unlikely to come from this method alone; building ranking pairs from full alternative story endings would be a direct next test.
  • Inference: the reported Story Cloze gain has no error bars, and the SWAG accuracies remain below dedicated reasoning models, so the mechanism may be best understood as a regularizer that steers an already strong next-token model rather than a general common-sense engine.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a multi-task fine-tuning scheme for GPT-2-based story generation. The primary task is language modeling on WritingPrompts; auxiliary tasks are 'perplexity ranking' on SWAG multiple-choice questions and on a synthetic dataset of 250K pairs of human WebText versus GPT-2-1.5B samples. The training pipeline first adapts GPT-2 to BookCorpus, then fine-tunes on WritingPrompts while alternating with the auxiliary objectives. The paper reports improvements in prompt ranking, SWAG and zero-shot Story Cloze accuracy, and claims state-of-the-art word-level perplexity on WritingPrompts.

Significance. If the claims hold, the work offers a simple, parameter-free way to inject common-sense bias into a generative language model: perplexity ranking reuses the LM head and does not add parameters. The zero-shot Story Cloze evaluation is a genuinely held-out signal, and the release of code supports reproducibility. The main value would be to show that ranking-based auxiliary objectives can improve the likelihood of sensible completions, beyond discriminative fine-tuning. However, the independent evidence is thin, as detailed below.

major comments (3)
  1. [Section 5, Table 3] The central claim of improved common sense reasoning rests on the Story Cloze column, which shows 59.1 for GPT2 to BC to WP, 63.2 with SWAG, and 64.1 with SWAG plus SYNTH. Because SWAG is used in training, the large SWAG validation improvement from 51.4 to 75.3 to 76.3 is expected and cannot by itself support the claim. The Story Cloze numbers are the independent signal, but they come from a single run with no error bars or significance testing, and the SYNTH increment is only 0.9 points. Please report results over multiple seeds with standard deviations and a significance test (or bootstrap) for the Story Cloze comparison; without this, the as-stated claim that the method achieves 'improved common sense reasoning' is not statistically supported.
  2. [Section 2.2, Synthetic] The synthetic task relies on the assumption that human-written text is more sensible than text written by neural language models, but the paper neither validates this premise on the actual 250K pairs nor analyzes what the ranking model learns. The WebText and GPT-2 samples are not matched for topic, style, or prompt, so the model could exploit surface-level or domain cues rather than common sense. This is load-bearing because the synthetic dataset is the component that makes the grounding 'targeted' beyond SWAG itself. I ask for a validation of the premise (e.g., human ratings on a sample of pairs) and an analysis of the learned ranking signal, or at least an ablation that isolates SYNTH from SWAG with error bars.
  3. [Section 4, Table 3] The claimed state-of-the-art word-level perplexity is an estimate obtained by taking the product of subword probabilities, marked with an asterisk. The paper does not give the exact word-boundary algorithm or validate the conversion against a tokenizer-independent word perplexity. Since the headline comparison against Fan et al.'s word-level perplexity of 36.6 depends on this estimate, please provide the estimation formula and an independent validation (for example, compare the estimated GPT-2 word perplexity on a corpus where word-level perplexity can be computed directly).
minor comments (5)
  1. [Section 2.2, Eq. (2)] Equation (2) defines ranking using length-normalized log probabilities, while the preceding text says the model should assign higher probability P(S1) to the sensible sequence. Please clarify whether 'higher probability' refers to the length-normalized quantity or the unnormalized sequence probability, since the two can disagree for sequences of different lengths.
  2. [Section 6, Related Work] The statement that perplexity ranking 'by definition, guarantees that the model will generate sensible text with a higher probability' is too strong: the objective only enforces the ranking on the auxiliary training pairs. It does not guarantee properties of arbitrary generated text. I suggest softening the claim or stating it as an intended inductive bias.
  3. [Table 3] Please report the number of seeds and standard deviations for all metrics, and state how many random samples were used for prompt ranking as well as the variance; Section 4 mentions 1000 random samples but no standard error is reported.
  4. [Tables 4-7] The qualitative samples are useful for intuition, but they are selected examples. A small human evaluation, or a larger random sample with inter-annotator agreement, would make the claim of improved sensible generation more convincing.
  5. [Throughout] There are several formatting artifacts such as 'V aswani', 'Y oshi', and the inconsistent spacing in 'WritingProm pts' in Table 3. These should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method's transfer claim rests on held-out Story Cloze, not on a training-signal echo.

full rationale

The paper's central claim is that adding perplexity-ranking auxiliary tasks (SWAG and a synthetic human-vs-GPT2 corpus) to a two-stage GPT-2 fine-tuning pipeline improves common sense in story generation. The only component that might look circular is the SWAG validation improvement, since the model is fine-tuned on SWAG training data and then evaluated on SWAG validation data using the same perplexity-ranking rule. The paper explicitly labels this 'unsurprisingly' and does not treat it as the main evidence; it then reports zero-shot performance on the held-out Story Cloze validation set, which was not used in training. The Story Cloze gain (59.1 to 64.1 from baseline to full model) is an external, independent transfer result, so the central claim does not reduce to the training objective. The synthetic dataset is built on an explicit assumption ('human written text is more sensible than text written by neural language models') that is not directly validated; however, an unverified assumption is a correctness risk, not a circular step, because the downstream evaluation is not constructed from the synthetic labels. There is no load-bearing self-citation, no imported uniqueness theorem, and no fitted parameter renamed as a prediction. The paper also acknowledges the limitation that SWAG provides only short-term grounding. Therefore no specific step can be exhibited where Eq. X reduces to Eq. Y by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its load-bearing assumptions are all about data semantics: that SWAG correctness captures common sense, that human text is more sensible than GPT-2 text, and that perplexity ranking transfers from sentence-level multiple choice to open-ended story generation. The free parameters are standard hyperparameters and heuristic choices, not fitted constants in a derivation.

free parameters (4)
  • Auxiliary task training frequency = SWAG every 30 iterations; synthetic every 15; primary every iteration
    Chosen by hand in Section 8 to weight the primary task over auxiliary tasks. No ablation is reported, and this ratio affects the trade-off between storytelling fluency and common sense.
  • Maximum learning rate = 5e-5
    Set following Howard and Ruder (2018) with 1,000 warm-up iterations and linear decay over 99,000 iterations. It is a standard choice, not fitted to the target data.
  • Nucleus sampling probability p = 0.9
    Used when generating the qualitative samples in Section 5 and the appendix; it affects the presented outputs but not the quantitative metrics.
  • Batch size = 16
    Set to 16 in Section 8; a practical choice for the 8 GPU setup, not independently varied.
assumptions (5)
  • standard math The language-model factorization P(X) = product of P(x_t | x_1:t-1) with cross-entropy training yields a meaningful perplexity ranking across complete text sequences.
    Invoked in Eq. 1 and used for both the primary objective and the auxiliary ranking loss. Standard in language modeling, but the auxiliary task assumes that comparing length-normalized log probabilities across sentences is a valid way to score common sense.
  • domain assumption Human-written text is more sensible than text written by neural language models.
    Stated explicitly in Section 2.2 as the basis for the synthetic dataset. This premise determines what the auxiliary objective rewards; if false, training on human-versus-GPT-2 pairs could bias the model toward style or domain rather than common sense.
  • domain assumption SWAG's correct endings are sensible and its adversarial endings are spurious, so ranking correct endings lower-perplexity is a valid proxy for common sense in stories.
    Used to construct the auxiliary loss and to evaluate CSR in Section 4. SWAG is deliberately adversarially filtered to fool language models, so the proxy is non-obvious and is itself a claim about the dataset.
  • domain assumption Pre-trained GPT-2 weights and BookCorpus fine-tuning transfer positively to WritingPrompts story generation.
    This is the foundation of the two-stage pipeline. It is supported empirically by Table 3, but the authors assume the transfer works and do not test alternative pretrained models or domain-adaptation corpora.
  • domain assumption Length-normalized log probabilities are comparable across sequences of different lengths in the ranking objective.
    Equation 2 divides each log-probability by sequence length to compare sensible and spurious completions of different lengths. This is an ad hoc normalization choice that the paper does not justify.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Neural Story Generation by Targeted Common Sense Grounding." pith.science (2026). https://pith.science/paper/3I4QO72L

@misc{pith2026190809451,
  author       = {Pith},
  title        = {Pith review of: Improving Neural Story Generation by Targeted Common Sense Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3I4QO72L}},
  note         = {Machine review of arXiv:1908.09451}
}
read the original abstract

Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasoning, e.g., they often contain sentences deprived of world knowledge. We propose a simple multi-task learning scheme to achieve quantitatively better common sense reasoning in language models by leveraging auxiliary training signals from datasets designed to provide common sense grounding. When combined with our two-stage fine-tuning pipeline, our method achieves improved common sense reasoning and state-of-the-art perplexity on the Writing Prompts (Fan et al., 2018) story generation dataset.

Figures

Figures reproduced from arXiv: 1908.09451 by the authors.

Figure 1
Figure 1. Our two-stage fine-tuning pipeline with aux [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 16 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

    work page

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    A neural probabilistic language model

    Yoshua Bengio, R \' e jean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137--1155

    work page

  4. [4]

    Rich Caruana. 1993. Multitask learning: A knowledge-based source of inductive bias. In ICML

    work page 1993

  5. [5]

    Dai and Quoc V

    Andrew M. Dai and Quoc V. Le. Semi-supervised sequence learning. In NIPS

    work page

  6. [6]

    BERT: pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805

    arXiv

  7. [7]

    Jeffrey L. Elman. 1990. Finding structure in time. Cognitive Science, 14(2):179--212

    work page 1990

  8. [8]

    Hierarchical neural story generation

    Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In ACL

    work page

Show all 28 references
  1. [9]

    The curious case of neural text degeneration

    Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. CoRR, abs/1904.09751

    1904 arXiv

  2. [10]

    Fine-tuned language models for text classification

    Jeremy Howard and Sebastian Ruder. Fine-tuned language models for text classification. CoRR, abs/1801.06146

    arXiv

  3. [11]

    Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler

    Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In NIPS

  4. [12]

    Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning

    2012

  5. [13]

    Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018. The natural language decathlon: Multitask learning as question answering. CoRR arXiv:1806.08730

    2018 arXiv

  6. [14]

    Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James F. Allen. A corpus and evaluation framework for deeper understanding of commonsense stories. NAACL-HLT

  7. [15]

    Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

    Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In NAACL-HLT

  8. [16]

    Jason Phang, Thibault F \' e vry, and Samuel R. Bowman. Sentence encoders on stilts: Supplementary training on intermediate labeled-data tasks. CoRR, abs/1811.01088

    arXiv

  9. [17]

    Radford, J

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. Technical report, OpenAI

  10. [18]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. Technical report, OpenAI

  11. [19]

    Writing stories with help from recurrent neural networks

    Melissa Roemmele. Writing stories with help from recurrent neural networks. In AAAI

  12. [20]

    An overview of multi-task learning in deep neural networks

    Sebastian Ruder. An overview of multi-task learning in deep neural networks. CoRR, abs/1706.05098

    arXiv

  13. [21]

    Neural machine translation of rare words with subword units

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In ACL

  14. [22]

    Identifying beneficial task relations for multi-task learning in deep neural networks

    Anders S gaard and Joachim Bingel. Identifying beneficial task relations for multi-task learning in deep neural networks. In EACL

  15. [23]

    Shane Storks, Qiaozi Gao, and Joyce Y. Chai. Commonsense reasoning for natural language understanding: A survey of benchmarks, resources, and approaches. CoRR, abs/1904.01172

    1904 arXiv

  16. [24]

    Commonsenseqa: A question answering challenge targeting commonsense knowledge

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. CoRR, abs/1811.00937

    arXiv

  17. [25]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS

  18. [26]

    Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan

    Lili Yao, Nanyun Peng, Ralph M. Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and-write: Towards better automatic storytelling. CoRR, abs/1811.05701

    arXiv

  19. [27]

    SWAG: A large-scale adversarial dataset for grounded commonsense inference

    Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. SWAG: A large-scale adversarial dataset for grounded commonsense inference. CoRR, abs/1808.05326

    arXiv

  20. [28]

    Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler

    Yukun Zhu, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In ICCV

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.