pith. sign in

arxiv: 2512.05747 · v3 · submitted 2025-12-05 · 💻 cs.CL

Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning

Jinlong Liu , Mohammed Bahja , Venelin Kovatchev , Mark Lee This is my paper

Pith reviewed 2026-05-17 01:05 UTC · model grok-4.3

classification 💻 cs.CL
keywords long-form story generationauthorial styleGRPOauthorship verificationsentence transformerstyle transferreward modelingfine-tuning
0
0 comments X

The pith

A style-similarity judge trained on authorship verification supplies the reward for GRPO fine-tuning, letting an 8B model generate long-form stories that match the voices of Twain, Austen, Dickens, and Hardy more closely than baselines.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that authorial style can be isolated and optimized in long-form generation by first training a sentence-transformer judge on pairs that verify whether two texts share an author, then calibrating its outputs into a continuous reward between zero and one. This reward then drives Group Relative Policy Optimization on an 8B story model, sidestepping the paired preference data that Direct Preference Optimization requires. The approach matters because ad-hoc prompts usually mix style with overall quality and scale poorly to long texts, so a dedicated judge offers a more stable signal for controllable style transfer. If the claim holds, moderate-sized models and budgets become sufficient to produce extended narratives that reliably echo specific classic authors rather than generic prose.

Core claim

The paper claims that fine-tuning a sentence-transformer on authorship-verification supervision yields a calibrated style-similarity reward that, when used as the primary objective in Group Relative Policy Optimization, produces an 8B-parameter story generator whose outputs score higher on authorial style for Mark Twain, Jane Austen, Charles Dickens, and Thomas Hardy than open-weight baselines, reaching an average style score of 0.893.

What carries the argument

The authorship-verification-calibrated style-similarity judge that converts sentence-transformer outputs into a bounded reward signal for GRPO fine-tuning.

If this is right

  • Style scores rise for each of the four tested authors when the GRPO-trained model is compared with open-weight baselines.
  • GRPO eliminates the need to collect explicit accept/reject pairs that Direct Preference Optimization demands for the same task.
  • The pipeline operates at the scale of an 8B model and a moderate training budget while still delivering measurable style control.
  • Style evaluation is separated from general quality judgments through the use of a dedicated verification-trained judge.
  • The same two-stage structure of judge training followed by reward-driven optimization can be reused for additional target authors or styles.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same verification-based judge construction could be adapted to control voice consistency across multi-chapter narratives rather than single excerpts.
  • If the judge proves robust, the method offers a route to style control that reduces dependence on large volumes of human preference labels.
  • Extending the pipeline to non-English authors or contemporary writers would reveal whether the calibration step transfers beyond the four classic cases examined.
  • The approach suggests a broader pattern in which task-specific verification data can replace generic reward models in other controllable generation settings.

Load-bearing premise

The judge trained on authorship-verification pairs measures authorial style independently of overall writing quality and generalizes reliably to long-form generated text.

What would settle it

A blind human study in which readers match generated stories to target authors without knowing the source model, then compare match rates for the GRPO outputs against the open-weight baselines, would directly test whether the reported style gains are perceptible.

Figures

Figures reproduced from arXiv: 2512.05747 by Jinlong Liu, Mark Lee, Mohammed Bahja, Venelin Kovatchev.

Figure 1
Figure 1. Figure 1: Subjects & Authors Distributions women – Fiction, and Man-woman relationships – Fiction. The resulting corpus contains 489 authors and 978 books; [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Subjects performance of GTE-large-en-v1.5 original chunks to populate intermediate labels: the heuristic similarity 1−r is only directly defined for original–refilled pairs, and using originals through￾out would overuse a small set of chunks, reducing pair diversity and increasing memorisation risk. Intermediate labels are therefore constructed from refilled–refilled pairs under the same-subject, different… view at source ↗
Figure 3
Figure 3. Figure 3: Separation width vs. chunk size. Separa [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Midpoint shift by chunk size. Absolute de [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Reward and KL-divergence trajectories during training. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Result of Chunk size 500 0.0 0.2 0.4 0.6 0.8 1.0 Predicted score Nomic-Embed-v1.5 ModernBERT-embed-base Base Fine-Tuned 0.0 0.2 0.4 0.6 0.8 1.0 Predicted score GTE-Large-v1.5 Base Fine-Tuned BAAI-BGE-M3 Pair Type Same Author Cross Author [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Result of Chunk size 1000 11 [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Result of Chunk size 1500 0.0 0.2 0.4 0.6 0.8 1.0 Predicted score Nomic-Embed-v1.5 ModernBERT-embed-base Base Fine-Tuned 0.0 0.2 0.4 0.6 0.8 1.0 Predicted score GTE-Large-v1.5 Base Fine-Tuned BAAI-BGE-M3 Pair Type Same Author Cross Author [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Result of Chunk size 2000 12 [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Result of Chunk size 2500 0.0 0.2 0.4 0.6 0.8 1.0 Predicted score Nomic-Embed-v1.5 ModernBERT-embed-base Base Fine-Tuned 0.0 0.2 0.4 0.6 0.8 1.0 Predicted score GTE-Large-v1.5 Base Fine-Tuned BAAI-BGE-M3 Pair Type Same Author Cross Author [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Result of Chunk size 3000 13 [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Subject-wise performance of fine-tuned embedding judges (excluding [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗
read the original abstract

Evaluating and optimising authorial style in long-form story generation remains challenging because style is often assessed with ad hoc prompting and is frequently conflated with overall writing quality. We propose a two-stage pipeline. First, we train a dedicated style-similarity judge by fine-tuning a sentence-transformer with authorship-verification supervision, and calibrate its similarity outputs into a bounded $[0,1]$ reward. Second, we use this judge as the primary reward in Group Relative Policy Optimization (GRPO) to fine-tune an 8B story generator for style-conditioned writing, avoiding the accept/reject supervision required by Direct Preference Optimization (DPO). Across four target authors (Mark Twain, Jane Austen, Charles Dickens, Thomas Hardy), the GRPO-trained 8B model achieves higher style scores than open-weight baselines, with an average style score of 0.893 across authors. These results suggest that AV-calibrated reward modelling provides a practical mechanism for controllable style transfer in long-form generation under a moderate model size and training budget.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. This manuscript proposes a two-stage pipeline for capturing authorial style in long-form story generation. In the first stage, a sentence-transformer is fine-tuned on authorship-verification pairs to serve as a style-similarity judge, with its outputs calibrated to a [0,1] reward range. In the second stage, this judge provides the reward signal for Group Relative Policy Optimization (GRPO) to fine-tune an 8B-parameter story generation model. The authors evaluate the approach on four classic authors—Mark Twain, Jane Austen, Charles Dickens, and Thomas Hardy—reporting that the GRPO-fine-tuned model achieves higher style scores than open-weight baselines, with an average score of 0.893 across authors. The work aims to address challenges in assessing and optimizing style without conflating it with general writing quality.

Significance. Should the style judge prove to measure stylistic features independently of content, this pipeline represents a practical advance for controllable style transfer in long-form generation using moderate compute resources. The choice of GRPO over DPO is a strength, as it directly optimizes with the continuous reward without requiring paired preference data. This could enable more accessible experimentation in style-conditioned generation. The reported average score of 0.893 suggests effective style capture if the metric is reliable.

major comments (2)
  1. [§4.1] §4.1 (Style Judge Training): The authorship-verification supervision is described without details on pair sampling strategy. Positive pairs drawn from the same author's works may frequently share narrative content, themes, or settings, allowing the fine-tuned embeddings to capture content similarity rather than isolated authorial style (syntax, diction, voice). No content-controlled ablation or cross-work validation is mentioned. This directly impacts the validity of the style scores used in both training and evaluation, as the headline result of 0.893 relies on the judge rewarding stylistic fidelity.
  2. [§5] §5 (Experiments and Evaluation): The evaluation reports an average style score of 0.893 but omits critical details on the judge calibration procedure to [0,1], generated story lengths, evaluation prompts, sample counts per author, and statistical significance of improvements over baselines. These omissions prevent verification of the central quantitative claim and assessment of reliable generalization to long-form text.
minor comments (1)
  1. [§4] The methods section would benefit from an explicit equation defining the calibrated reward function to improve reproducibility of the [0,1] scaling.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. We address each major comment below and indicate the revisions we will make to improve methodological transparency and address concerns about the independence of the style metric from content.

read point-by-point responses

  1. Referee: [§4.1] §4.1 (Style Judge Training): The authorship-verification supervision is described without details on pair sampling strategy. Positive pairs drawn from the same author's works may frequently share narrative content, themes, or settings, allowing the fine-tuned embeddings to capture content similarity rather than isolated authorial style (syntax, diction, voice). No content-controlled ablation or cross-work validation is mentioned. This directly impacts the validity of the style scores used in both training and evaluation, as the headline result of 0.893 relies on the judge rewarding stylistic fidelity.

    Authors: We acknowledge that the current manuscript provides insufficient detail on the pair sampling strategy in §4.1, and that this omission leaves open the possibility that the judge captures content similarity in addition to style. We will revise §4.1 to include a complete description of how positive pairs (same author, different works) and negative pairs (different authors) were constructed. In addition, we will add a content-controlled ablation and cross-work validation experiment to the revised manuscript to quantify the extent to which the judge isolates stylistic features such as syntax and diction from shared narrative elements. revision: yes

  2. Referee: [§5] §5 (Experiments and Evaluation): The evaluation reports an average style score of 0.893 but omits critical details on the judge calibration procedure to [0,1], generated story lengths, evaluation prompts, sample counts per author, and statistical significance of improvements over baselines. These omissions prevent verification of the central quantitative claim and assessment of reliable generalization to long-form text.

    Authors: We agree that these experimental details are necessary for reproducibility and for allowing readers to assess the reliability of the 0.893 average score. We will expand §5 to report the exact calibration procedure used to map similarity scores to the [0,1] reward range, the target length of generated stories, the specific evaluation prompts employed, the number of samples generated per author, and the results of statistical significance tests comparing the GRPO model to baselines. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in the derivation chain

full rationale

The paper's core pipeline consists of two explicitly separated stages: (1) independently fine-tuning a sentence-transformer on authorship-verification pairs to produce a calibrated [0,1] style-similarity reward, and (2) using that fixed external judge as the reward signal inside GRPO to optimize the 8B generator. The reported average style score of 0.893 is simply the output of the same pre-trained judge applied to the resulting generations; it is not redefined, refitted, or derived from the GRPO objective itself. No equations reduce the final metric to the training inputs by construction, no self-citations are invoked as load-bearing uniqueness theorems, and no ansatz is smuggled through prior work. The derivation therefore remains self-contained against external benchmarks (the AV-trained judge and the open-weight baselines).

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that a sentence-level similarity model trained on authorship pairs can serve as a faithful proxy for authorial style in generated long-form text. No free parameters or invented entities are explicitly introduced in the abstract; the calibration step to [0,1] is described but not quantified.

axioms (1)
  • domain assumption Authorship-verification supervision produces a similarity metric that isolates style from content and quality.
    Invoked when the paper states the judge is trained with authorship-verification supervision and then used as the primary reward.

pith-pipeline@v0.9.0 · 5485 in / 1374 out tokens · 31076 ms · 2026-05-17T01:05:50.689868+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

5 extracted references · 5 canonical work pages · 2 internal anchors

  1. [1]

    InFind- ings of the Association for Computational Linguis- tics: EMNLP 2023, pages 14078–14084, Singapore

    Who wrote it and why? prompting large- language models for authorship verification. InFind- ings of the Association for Computational Linguis- tics: EMNLP 2023, pages 14078–14084, Singapore. Association for Computational Linguistics. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin

    work page 2023

  2. [2]

    Understanding R1-Zero-Like Training: A Critical Perspective

    Understanding r1-zero-like training: A critical perspective.Preprint, arXiv:2503.20783. Gaspard Michel, Elena Epure, Romain Hennequin, and Christophe Cerisara. 2024. Distinguishing fictional voices: a study of authorship verification models for quotation attribution. InProceedings of the 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural ...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems, 36:53728–53741. Sahana Ramnath, Kartik Pandey, Elizabeth Boschee, and Xiang Ren. 2025. CA VE: Controllable author- ship verification explanations. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of t...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    <input_prompt> (contains Author/Title, the requested plot, and constraints)

    work page

  5. [5]

    about" and the phrase

    <text> (the generated story) You must compute: A) Plot adherence (YES/NO) B) Writing BaseScore (0–9) from Grammar/Clarity/Coherence/Concision C) FinalScore with the rules below <input_prompt> {prompt} </input_prompt> <text> {response_text} </text> ### Step 1 — Extract requirements from <input_prompt> (must do internally) - Extract REQUIRED_PLOT as the tex...

    work page