{"id":"7b648c54-58e8-4898-8b74-70215a0751c3","arxiv_id":"1908.09451","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multi-task fine-tuning pipeline with a perplexity ranking objective improves common sense and perplexity in GPT-2 based story generation.","lead":"This paper trains a story-writing AI to prefer sensible text by adding a common sense ranking task alongside ordinary language modeling. The method improves the model on common sense benchmarks and lowers perplexity on a story dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The synthetic common-sense task's premise that human text is more sensible than GPT2 samples is unverified, and the only transfer evidence is a 0.9-point Story Cloze gain from a single run.","rationale":"The paper's central claim has two components: improved common sense reasoning and state-of-the-art perplexity. The perplexity claim is weakened by the estimated word-level conversion and the single 2018 baseline, but the paper itself labels the conversion with an asterisk, and the main novelty is the auxiliary common-sense objectives. The CSR claim is supported by a SWAG validation improvement that is expected because the model trains on SWAG, and by a zero-shot Story Cloze improvement from 59.1 to 64.1, which is independent and is the most convincing evidence. The weakest link in this chain is the synthetic dataset in Section 2.2: it contributes a new training signal beyond SWAG, but its labels are only proxies ('human' vs 'GPT2') for the property ('sensible') that the paper wants to improve. The paper states the assumption explicitly and does not verify it; the OpenAI GPT2-1.5B output corpus and WebText differ in many non-semantic ways, so the learned ranking may be a text-source classifier. The reported marginal gain from adding SYNTH to Story Cloze is 0.9 points with no variance, so the empirical support for the synthetic component is thin. If the assumption is false or the gain is noise, the 'targeted common sense grounding' contribution is not established, though the SWAG-based multi-task objective could still be valuable. A human annotation study of the synthetic pairs is a direct test of the premise and would settle the concern. I therefore agree with the reader's weakest assumption and see no reason to change the CONDITIONAL verdict.","tokens_in":9288,"tokens_out":17455,"duration_ms":192713,"concrete_test":"Annotate a random sample of 500 pairs from the synthetic dataset (WebText vs GPT2-1.5B) with three crowd-workers per pair, asking which text is more sensible or whether they are equally sensible. Take the majority vote as the sensibleness label and compute the accuracy of the paper's 'human text is the sensible one' labeling. If accuracy is below 70% (or not clearly above chance), the Section 2.2 premise is not supported, and the synthetic objective cannot be claimed to ground common sense. If accuracy is high, the premise survives and the central claim is not threatened by this concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The synthetic auxiliary task in Section 2.2 is the only component of the method not derived from an existing common-sense benchmark, and its correctness rests entirely on the premise that human-written WebText is more sensible than GPT2-1.5B samples. The paper never tests this premise, and the construction (250K human-vs-machine pairs, no prompt/topic matching details) allows the model to learn surface-level stylistic or domain cues rather than common sense. The only evidence that this objective transfers to story common sense is the zero-shot Story Cloze gain in Table 3: adding SYNTH changes 63.2 to 64.1, a difference of under one point reported from a single run with no error bars or multiple seeds. If the human-vs-GPT2 label is not a faithful sensibleness signal, the synthetic objective could push the model toward stylistic mimicry rather than common sense, and the 'further boosts' conclusion in Section 5 is unsupported. This is load-bearing because the synthetic dataset is a stated contribution and is what makes the common-sense grounding 'targeted' beyond SWAG itself.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multi-task fine-tuning scheme for GPT-2-based story generation. The primary task is language modeling on WritingPrompts; auxiliary tasks are 'perplexity ranking' on SWAG multiple-choice questions and on a synthetic dataset of 250K pairs of human WebText versus GPT-2-1.5B samples. The training pipeline first adapts GPT-2 to BookCorpus, then fine-tunes on WritingPrompts while alternating with the auxiliary objectives. The paper reports improvements in prompt ranking, SWAG and zero-shot Story Cloze accuracy, and claims state-of-the-art word-level perplexity on WritingPrompts.","tokens_in":9429,"tokens_out":4680,"duration_ms":50272,"significance":"If the claims hold, the work offers a simple, parameter-free way to inject common-sense bias into a generative language model: perplexity ranking reuses the LM head and does not add parameters. The zero-shot Story Cloze evaluation is a genuinely held-out signal, and the release of code supports reproducibility. The main value would be to show that ranking-based auxiliary objectives can improve the likelihood of sensible completions, beyond discriminative fine-tuning. However, the independent evidence is thin, as detailed below.","major_comments":[{"comment":"The central claim of improved common sense reasoning rests on the Story Cloze column, which shows 59.1 for GPT2 to BC to WP, 63.2 with SWAG, and 64.1 with SWAG plus SYNTH. Because SWAG is used in training, the large SWAG validation improvement from 51.4 to 75.3 to 76.3 is expected and cannot by itself support the claim. The Story Cloze numbers are the independent signal, but they come from a single run with no error bars or significance testing, and the SYNTH increment is only 0.9 points. Please report results over multiple seeds with standard deviations and a significance test (or bootstrap) for the Story Cloze comparison; without this, the as-stated claim that the method achieves 'improved common sense reasoning' is not statistically supported.","section":"Section 5, Table 3"},{"comment":"The synthetic task relies on the assumption that human-written text is more sensible than text written by neural language models, but the paper neither validates this premise on the actual 250K pairs nor analyzes what the ranking model learns. The WebText and GPT-2 samples are not matched for topic, style, or prompt, so the model could exploit surface-level or domain cues rather than common sense. This is load-bearing because the synthetic dataset is the component that makes the grounding 'targeted' beyond SWAG itself. I ask for a validation of the premise (e.g., human ratings on a sample of pairs) and an analysis of the learned ranking signal, or at least an ablation that isolates SYNTH from SWAG with error bars.","section":"Section 2.2, Synthetic"},{"comment":"The claimed state-of-the-art word-level perplexity is an estimate obtained by taking the product of subword probabilities, marked with an asterisk. The paper does not give the exact word-boundary algorithm or validate the conversion against a tokenizer-independent word perplexity. Since the headline comparison against Fan et al.'s word-level perplexity of 36.6 depends on this estimate, please provide the estimation formula and an independent validation (for example, compare the estimated GPT-2 word perplexity on a corpus where word-level perplexity can be computed directly).","section":"Section 4, Table 3"}],"minor_comments":[{"comment":"Equation (2) defines ranking using length-normalized log probabilities, while the preceding text says the model should assign higher probability P(S1) to the sensible sequence. Please clarify whether 'higher probability' refers to the length-normalized quantity or the unnormalized sequence probability, since the two can disagree for sequences of different lengths.","section":"Section 2.2, Eq. (2)"},{"comment":"The statement that perplexity ranking 'by definition, guarantees that the model will generate sensible text with a higher probability' is too strong: the objective only enforces the ranking on the auxiliary training pairs. It does not guarantee properties of arbitrary generated text. I suggest softening the claim or stating it as an intended inductive bias.","section":"Section 6, Related Work"},{"comment":"Please report the number of seeds and standard deviations for all metrics, and state how many random samples were used for prompt ranking as well as the variance; Section 4 mentions 1000 random samples but no standard error is reported.","section":"Table 3"},{"comment":"The qualitative samples are useful for intuition, but they are selected examples. A small human evaluation, or a larger random sample with inter-annotator agreement, would make the claim of improved sensible generation more convincing.","section":"Tables 4-7"},{"comment":"There are several formatting artifacts such as 'V aswani', 'Y oshi', and the inconsistent spacing in 'WritingProm pts' in Table 3. These should be cleaned up.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is transparent about its limitations and the code release is a strength. The main risk is that the independent evidence for the central claim is a single-run Story Cloze gain of about one point, so I would like to see the authors add statistical robustness and a validation of the synthetic-data premise before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper has one genuinely nice idea: instead of adding a classification head or scoring module, define common-sense quality as the model's own length-normalized log-probability ranking over plausible and spurious completions, and train that ranking as an auxiliary objective during fine-tuning. That is parameter-free and applies directly at generation time. The two-stage pipeline (BookCorpus intermediate fine-tuning, then WritingPrompts plus SWAG and synthetic pairs) is also clean and reproducible; code is released.\n\nWhat works: the zero-shot Story Cloze improvement from 59.1 to 64.1 when adding SWAG is a genuine transfer signal, not just memorization. Prompt ranking also improves. The paper is honest that the auxiliary objective does not add parameters.\n\nThe soft spots are the ones you'd expect. The SWAG validation gain (51.4 to 75.3) is nearly circular because the model trained on SWAG. The synthetic dataset is supposed to be the more 'targeted' grounding, but its whole premise — human WebText is more sensible than GPT-2 samples — is never verified, and the dataset construction (no prompt/topic matching details) could let the model learn surface cues. The only evidence that SYNTH helps beyond SWAG is the 63.2 to 64.1 Story Cloze bump, a 0.9-point difference from a single run with no error bars. That is too thin to support the claim that synthetic pairs 'further boost' common sense. Also, the 'state-of-the-art perplexity' label compares only to Fan et al. (2018) and uses an estimated word-level perplexity; it is a limited claim.\n\nThese are fixable, not fatal. The core method works without the synthetic set; the synthetic contribution is the weakest part. I'd like to see multiple seeds and a sanity check of whether the ranking task actually separates human and GPT-2 text in a meaningful way.\n\nWho is this for? Researchers working on story generation, open-ended language models, and multi-task auxiliary objectives for commonsense behavior. It deserves a serious referee; the central transfer claim is worth engaging even if the current evidence is conditional.\n\nRecommendation: send to review, but ask for a stronger synthetic evaluation and uncertainty quantification.","headline":"A simple no-extra-parameter common-sense grounding objective for GPT-2 story generation; the transfer evidence is real but thinner than the abstract suggests.","tokens_in":10049,"tokens_out":1945,"would_cite":true,"duration_ms":19376,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By ranking sensible completions above spurious ones, a GPT-2 pipeline improves common sense and reaches the best-reported perplexity on WritingPrompts.","keywords":["story generation","common sense reasoning","multi-task learning","transfer learning","perplexity ranking","GPT-2","WritingPrompts"],"falsifier":"Run the full pipeline with the synthetic pairs relabeled so the model is trained to prefer GPT-2-written text over human WebText excerpts; if zero-shot SWAG or Story Cloze accuracy still rises by the same margin, the common-sense gain is not caused by the sensible/spurious distinction the paper claims. Since the reported Story Cloze numbers have no error bars, repeating the comparison over several seeds would also settle whether the 5-point gain is signal or noise.","tokens_in":9022,"feed_emoji":"📖","tokens_out":12432,"duration_ms":112252,"temperature":0.7,"pith_summary":"This paper argues that next-token perplexity alone is not enough to stop neural story generators from writing nonsense such as \"a sunny, warm summer night,\" and that a direct ranking signal for sensible text can close part of the gap. The authors propose a multi-task fine-tuning scheme: a pretrained GPT-2 is first adapted to BookCorpus, then trained on WritingPrompts while alternating with a perplexity-ranking objective on SWAG and on synthetic pairs of human-written versus GPT-2-generated text. The ranking objective makes the model assign higher length-normalized probability to the sensible completion than to spurious alternatives, reusing the language model's own softmax and adding no parameters. On WritingPrompts the pipeline reaches the lowest reported perplexity, and on zero-shot Story Cloze it improves from 59.1% to 64.1% accuracy, which the paper reads as evidence that common-sense grounding transfers to story completion. If correct, this gives a cheap, task-agnostic way to bias generative models toward more sensible outputs without sacrificing fluency.","feed_headline":"Common-sense ranking makes GPT-2 stories more sensible","feed_subtitle":"A two-stage fine-tune plus a sensible-text ranking rule lifts Story Cloze from 59 to 64 percent with no extra parameters.","key_machinery":"The central object is a perplexity-ranking objective: for candidate sequences $S_1,\\ldots,S_N$, with $S_1$ the sensible continuation, the model maximizes the softmax probability of $S_1$ over the length-normalized log probabilities $\\frac{1}{T_i}\\log P(S_i)$ (Equation 2). This reuses the language model's softmax layer and adds no parameters. The objective is wrapped in a two-stage pipeline: intermediate fine-tuning on BookCorpus adapts a pretrained GPT-2 to narrative text, then multi-task fine-tuning alternates the WritingPrompts language-model loss with ranking on SWAG and on 250K synthetic pairs of human WebText excerpts and GPT-2 samples. The ranking mechanism is what carries the common-sense improvement, and the synthetic pairs expand it beyond the four-choice SWAG format.","core_discovery":"On its own terms, the central discovery is that common sense can be operationalized for a generative model as a ranking constraint: among several plausible continuations, the model should assign the highest probability to the one a human would call sensible. Training with this constraint—a softmax over length-normalized log probabilities of candidate sequences—biases a pretrained GPT-2 toward text with fewer logical inconsistencies. Combined with an intermediate BookCorpus fine-tuning stage, the method yields sub-word perplexity 20.78 on WritingPrompts, estimated word-level perplexity 29.63, prompt-ranking accuracy 80.1%, and zero-shot common-sense accuracies of 76.3% on SWAG and 64.1% on Story Cloze, improving over the same model without the auxiliary objectives. The paper's own caveat is that this supplies short-term common-sense grounding, not story-level coherence.","pith_inferences":["Inference: the same parameter-free ranking loss could be applied to dialogue, summarization, or instruction-following models by sampling degraded outputs from the model itself and ranking preferred outputs above them; the paper does not test these settings.","Inference: because the paper's own conclusion notes that SWAG offers only short-term grounding, story-level coherence is unlikely to come from this method alone; building ranking pairs from full alternative story endings would be a direct next test.","Inference: the reported Story Cloze gain has no error bars, and the SWAG accuracies remain below dedicated reasoning models, so the mechanism may be best understood as a regularizer that steers an already strong next-token model rather than a general common-sense engine."],"forward_implications":["Two-stage fine-tuning (BookCorpus, then WritingPrompts) alone beats the Fusion Model's perplexity and prompt-ranking accuracy, so task-specific story-generation architectures are unnecessary once a pretrained model is available.","Adding multi-task perplexity ranking raises zero-shot Story Cloze accuracy from 59.1% to 64.1% and SWAG from 51.4% to 76.3%, while WritingPrompts perplexity stays effectively unchanged.","The auxiliary loss reuses the existing softmax layer, so the common-sense gain costs no additional parameters and only extra training epochs on auxiliary data.","The synthetic human-versus-GPT-2 pairs provide a further common-sense boost beyond SWAG alone, and the paper's qualitative samples show more grounded story completions with both auxiliary tasks."],"supporting_citations":[{"why":"It supplies the WritingPrompts dataset and the Fusion Model baseline that the proposed pipeline aims to beat.","marker":"Fan et al. (2018)"},{"why":"It provides the pretrained GPT-2 initialization and the model-written samples used as spurious text in the synthetic ranking pairs.","marker":"Radford et al. (2019)"},{"why":"It supplies SWAG, the four-choice adversarial common-sense dataset used both for the perplexity-ranking auxiliary objective and for evaluation.","marker":"Zellers et al. (2018)"},{"why":"It supplies Story Cloze, the zero-shot story-completion benchmark used to test whether common-sense gains transfer outside SWAG.","marker":"Mostafazadeh et al. (2016)"},{"why":"It provides BookCorpus for the intermediate domain-adaptation stage that lowers perplexity before multi-task training on WritingPrompts.","marker":"Zhu et al. (2015)"},{"why":"It contributes the fine-tuning recipe, including the triangular learning-rate schedule, used in both training stages.","marker":"Howard and Ruder (2018)"},{"why":"It motivates intermediate fine-tuning on a related auxiliary corpus before the target task, the first stage of the two-stage pipeline.","marker":"Phang et al. (2018)"},{"why":"It supplies nucleus sampling, the decoding procedure used for the qualitative story samples.","marker":"Holtzman et al. (2019)"}],"fun_headline_variants":["Rank by common sense, not likelihood, for GPT-2 stories","Common-sense ranking lifts Story Cloze to 64% with no extra parameters","Two-stage fine-tune plus sensible-text rule improves story sense","Targeted common-sense grounding boosts GPT-2 story plausibility"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that human-written text is more sensible than text sampled from a neural language model; if that ordering fails for the specific WebText and GPT-2 samples used, the ranking objective will optimize style or domain differences instead of common sense.","fun_headline_variants_meta":{"raw":{"variants":["Rank by common sense, not likelihood, for GPT-2 stories","Common-sense ranking lifts Story Cloze to 64% with no extra parameters","Two-stage fine-tune plus sensible-text rule improves story sense","Targeted common-sense grounding boosts GPT-2 story plausibility"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001287,"raw_usage":{"total_tokens":5192,"prompt_tokens":814,"completion_tokens":4378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":4301}},"tokens_in":430,"tokens_out":4378,"duration_ms":32770,"temperature":1.0,"reasoning_tokens":4301,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:10:28.286167+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the full pipeline with the synthetic pairs relabeled so the model is trained to prefer GPT-2-written text over human WebText excerpts; if zero-shot SWAG or Story Cloze accuracy still rises by the same margin, the common-sense gain is not caused by the sensible/spurious distinction the paper claims. Since the reported Story Cloze numbers have no error bars, repeating the comparison over several seeds would also settle whether the 5-point gain is signal or noise.","supporting_citations":[],"review_version":1}