Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

A dual-track pipeline using language models detects 349 biblical allusions in a corpus of modern novels and recovers over half of those previously documented by scholars.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-10 02:11 UTC

load-bearing objection The paper gives a concrete dual-track LLM pipeline that recovers 54% of known biblical allusions in McCarthy while claiming 349 total, but it reports no precision or false-positive numbers for the new detections. the 2 major comments →

arxiv 2604.19447 v1 submitted 2026-04-21 cs.CL

'The Order in the Horse's Heart': A Case Study in LLM-Assisted Stylometry for the Discovery of Biblical Allusion in Modern Literary Fiction

Ewan Cameron This is my paper
classification cs.CL
keywords biblical allusionstylometrylarge language modelsintertextualityliterary fictiontextual echoregister detection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds a system to locate subtle references to the King James Bible within contemporary fiction by combining two approaches. One track flags unusual words shared with the Bible and checks their local context with models, while the other has a model read the prose broadly to catch echoes in tone or structure. When tested on the target novels, the combined pipeline identifies hundreds of such connections and matches more than half of the allusions already noted in literary scholarship. A sympathetic reader would care because the work shows how automated assistance could make the study of intertextuality feasible across far larger collections of texts than manual review allows.

Core claim

The authors present a dual-track pipeline for detecting biblical allusions that carry a textual echo of shared phrasing or cadence. A bottom-up track uses inverse document frequency to surface rare vocabulary shared with the King James Bible, embeds the occurrences for sense disambiguation, and routes candidates through cascaded language model review. A top-down track directs a model to scan the fiction undirected by any specific biblical passage to catch register-based similarities. Long-context cross-validation holds entire novels alongside the Bible in one pass, and all findings are checked against published scholarship. Restricting to proper literary allusions rather than signposted or c

What carries the argument

Dual-track pipeline with a rarity-based bottom-up embedding track for candidate selection and an undirected top-down register track for additional detection, followed by cascaded LLM review and long-context cross-validation.

Load-bearing premise

That cascaded language model review can reliably separate genuine literary allusions from coincidental word overlaps or stylistic similarities without high rates of false positives or missed connections.

What would settle it

Expert scholars examining the 349 surfaced passages and concluding that a majority lack sufficient textual echo to qualify as allusions, or applying the same pipeline to a fresh corpus and recovering substantially less than 54 percent of its documented cases.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The pipeline can scale to statistical analysis of intertextuality across much larger literary collections than manual methods permit.
  • It distinguishes literary allusions involving reworked phrasing or cadence from overt signposted references such as direct quotes or named figures.
  • Recovery rates vary by connection type, reaching 80 percent for register collisions but only 30 percent for transformed imagery.
  • Language models function here as assistants that augment rather than replace mechanical stylometric techniques.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same dual-track structure could be repurposed to detect allusions to other fixed source texts such as Shakespeare or classical literature.
  • Repeated application across an author's full body of work might reveal whether allusion density changes with career stage or genre.
  • Integration into digital archives could let researchers query and visualize networks of biblical influence without exhaustive close reading.
  • Extending the method to non-English corpora would test whether the rarity-plus-register logic generalizes beyond the King James Bible.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript presents a dual-track LLM-assisted pipeline for detecting biblical allusions in Cormac McCarthy's novels: a bottom-up track using IDF to identify rare vocabulary shared with the King James Bible, followed by local-context embedding, sense disambiguation, and cascaded LLM review; and a top-down track in which an LLM scans McCarthy's prose for biblical register without reference to specific passages. Both tracks are cross-validated by a long-context model holding entire novels and the KJV together, with all findings checked against published scholarship. Restricting to textual-echo allusions (shared phrasing, reworked vocabulary, or cadence) and distinguishing literary allusions from signposted references, the pipeline reports 349 allusions across the corpus and independently recovers 62 of 115 previously documented allusions (54% recall), with recall varying by type (30% for transformed imagery to 80% for register collisions).

Significance. If the LLM filtering stage can be shown to maintain low false-positive rates on novel detections, the approach would offer a scalable, semi-automated method for large-scale intertextuality studies that combines mechanical stylometry with scholarly grounding. The explicit use of external published scholarship to anchor the 115 known cases provides independent validation and is a methodological strength. The type-specific recall breakdown and the distinction between allusion categories add useful nuance. However, the absence of precision or false-positive quantification for the 349 total (especially the ~287 novel items) limits the immediate utility for statistical claims about intertextuality.

major comments (2)
  1. [Abstract] Abstract: The central claim that the pipeline 'surfaces 349 allusions' and recovers 54% of known cases rests on the cascaded LLM review and long-context cross-validation reliably separating genuine textual echoes from coincidental overlaps. No precision, false-positive rate, inter-annotator agreement with literary experts, or ablation of the LLM filtering stage is reported for the novel detections, even though the initial IDF and undirected register tracks are acknowledged to generate many surface matches. This is load-bearing for the reliability of the 349-allusion corpus.
  2. [Abstract] Abstract (validation paragraph): The statement that 'every finding is checked against published scholarship' applies only to the 62 recovered known allusions; the ~287 novel detections lack an equivalent external anchor. Because the method begins with high-recall but low-precision candidate generation, the soundness of the full set depends on an unquantified assumption about the LLM review step.
minor comments (2)
  1. [Methods] The description of the cascaded LLM review and long-context cross-validation would benefit from explicit prompts, decision rules for model disagreements, and any temperature or sampling parameters used.
  2. [Results] Table or figure reporting per-novel or per-type breakdown of the 349 allusions (beyond the aggregate recall) would improve clarity and allow readers to assess variation.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and detailed report. The comments correctly identify that the reliability of the 349 novel detections hinges on the unquantified performance of the LLM filtering stages, and that the abstract phrasing regarding scholarship checks requires clarification. We address each point below and will revise the manuscript to incorporate these observations.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim that the pipeline 'surfaces 349 allusions' and recovers 54% of known cases rests on the cascaded LLM review and long-context cross-validation reliably separating genuine textual echoes from coincidental overlaps. No precision, false-positive rate, inter-annotator agreement with literary experts, or ablation of the LLM filtering stage is reported for the novel detections, even though the initial IDF and undirected register tracks are acknowledged to generate many surface matches. This is load-bearing for the reliability of the 349-allusion corpus.

    Authors: We agree that the absence of explicit precision or false-positive quantification for the novel detections is a substantive limitation. The pipeline's design uses dual-track convergence and long-context cross-validation as internal filters, and the 54% recall on the held-out known allusions provides one indirect signal, but this does not substitute for direct measurement on the novel set. In revision we will add a limitations subsection that (a) reports the rate at which the two independent tracks agree on novel candidates, (b) discusses the expected false-positive behavior inferred from the known-allusion performance, and (c) outlines how future work could obtain expert inter-annotator agreement on a sampled subset. We will also include a brief ablation note on the contribution of the cascaded LLM review versus the embedding and register stages alone. revision: yes

  2. Referee: [Abstract] Abstract (validation paragraph): The statement that 'every finding is checked against published scholarship' applies only to the 62 recovered known allusions; the ~287 novel detections lack an equivalent external anchor. Because the method begins with high-recall but low-precision candidate generation, the soundness of the full set depends on an unquantified assumption about the LLM review step.

    Authors: The referee is correct; the abstract sentence is imprecise. The scholarship check was performed only for the 115 previously documented allusions (recovering 62). Novel detections rest on the internal pipeline filters. We will revise the abstract to read: 'All recovered known allusions are checked against published scholarship; novel detections are validated through dual-track convergence and long-context cross-validation.' This change will be mirrored in the methods and results sections for consistency. revision: yes

Circularity Check

0 steps flagged

No circularity; external scholarship provides independent benchmark for recall

full rationale

The paper describes an LLM pipeline (IDF rare-word candidate generation, cascaded review, undirected register track, long-context cross-validation) applied to McCarthy novels and the KJV, then measures recall against 115 allusions previously documented in independent academic literature. This external reference set ensures the 54% recall figure and the count of 349 surfaced allusions are computed from outside the method's own outputs rather than by construction. No equations, parameter fits, self-citations, or uniqueness theorems appear as load-bearing steps in the derivation; the pipeline operates on raw texts and produces candidate lists whose validity is assessed against published scholarship.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; pipeline thresholds for rarity, context embedding, and LLM prompting criteria are not detailed.

pith-pipeline@v0.9.0 · 5555 in / 1070 out tokens · 37339 ms · 2026-05-10T02:11:58.314939+00:00 · methodology

0 comments
read the original abstract

We present a dual-track pipeline for detecting biblical allusions in literary fiction and apply it to the novels of Cormac McCarthy. A bottom-up embedding track uses inverse document frequency to identify rare vocabulary shared with the King James Bible, embeds occurrences in their local context for sense disambiguation, and passes candidate passage pairs through cascaded LLM review. A top-down register track asks an LLM to read McCarthy's prose undirected to any specific biblical passage for comparison, catching allusions not distinguished by word or phrase rarity. Both tracks are cross-validated by a long-context model that holds entire novels alongside the KJV in a single pass, and every finding is checked against published scholarship. Restricting attention to allusions that carry a textual echo--shared phrasing, reworked vocabulary, or transplanted cadence--and distinguishing literary allusions proper from signposted biblical references (similes naming biblical figures, characters overtly citing scripture), the pipeline surfaces 349 allusions across the corpus. Among a target set of 115 previously documented allusions retrieved through human review of the academic literature, the pipeline independently recovers 62 (54% recall), with recall varying by connection type from 30% (transformed imagery) to 80% (register collisions). We contextualise these results with respect to the value-add from LLMs as assistants to mechanical stylometric analyses, and their potential to facilitate the statistical study of intertextuality in massive literary corpora.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Beyond Similarity: Grounded Agentic Extraction and Expert-Adjudicated Evaluation of Intertextuality in Classical Chinese Histories

    cs.CL 2026-07 conditional novelty 8.0

    A tool-constrained LLM extracts span-grounded, typology-labeled intertextual pairs; expert-adjudicated validation and a 65,380-comparison run across the Twenty-Four Histories yield stable citation composition but decl...

Reference graph

Works this paper leans on

38 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    Alter, Robert. 2010. Pen of Iron: A merican Prose and the K ing J ames B ible . Princeton University Press

    work page 2010

  2. [2]

    Bamman, David et al. 2024. On classification with large language models in cultural analytics. In Proceedings of the Computational Humanities Research Conference

    work page 2024

  3. [3]

    Barr \'e , Jules. 2024. Latent structures in F rench fiction. In Proceedings of the Computational Humanities Research Conference 2024, volume 3834 of CEUR Workshop Proceedings, pages 21--36, Aarhus, Denmark

    work page 2024

  4. [4]

    Bloom, Harold. 2000. How to Read and Why. Scribner

    work page 2000

  5. [5]

    Broadbent, Alex. 2013. Philosophy of Epidemiology. Palgrave Macmillan, London

    work page 2013

  6. [6]

    Broncano, Manuel. 2014. Religion in Cormac M c C arthy's Fiction: Apocryphal Borderlands . Routledge

    work page 2014

  7. [7]

    B \"u chler, Marco, Gregory Crane, Maria Moritz, and Alison Babeu. 2012. Increasing recall for text re-use in historical documents to support research in the humanities. In Theory and Practice of Digital Libraries (TPDL 2012), volume 7489 of Lecture Notes in Computer Science, pages 95--100, Springer, Berlin, Heidelberg

    work page 2012

  8. [8]

    Dexter, and Timothy Bolt

    Burns, Patrick J., Pramit Chaudhuri, Joseph P. Dexter, and Timothy Bolt. 2021. Profiling of intertextuality in L atin literature using word embeddings. In Proceedings of the 2021 Conference of the North A merican Chapter of the Association for Computational Linguistics , Association for Computational Linguistics

    work page 2021

  9. [9]

    Cant, John. 2008. Cormac M c C arthy and the Myth of A merican Exceptionalism . Routledge

    work page 2008

  10. [10]

    Chaloner, Kathryn and Isabella Verdinelli. 1995. Bayesian experimental design: A review. Statistical Science, 10(3):273--304

    work page 1995

  11. [11]

    Cowart, David. 2021. The allusive art of cormac M c C arthy's The Orchard Keeper. Arizona Quarterly, 77(3):27--54

    work page 2021

  12. [12]

    Crews, Michael Lynn. 2017. Books Are Made Out of Books: A Guide to Cormac M c C arthy's Literary Influences . University of Texas Press

    work page 2017

  13. [13]

    Crystal, David. 2010. Begat: The K ing J ames B ible and the E nglish Language . Oxford University Press

    work page 2010

  14. [14]

    Cui, Lu and Lanju Zhang. 2019. On the efficiency of adaptive sample size design. Statistics in Medicine, 38(6):933--944

    work page 2019

  15. [15]

    Daugherty, Leo. 1993. Gravers false and true: Blood Meridian as gnostic tragedy. In Edwin T. Arnold and Dianne C. Luce, editors, Perspectives on Cormac M c C arthy . University Press of Mississippi, pages 157--172

    work page 1993

  16. [16]

    Dover, Kenneth James. 1971. Theocritus: Select Poems. Macmillan, London. Reprinted by Bristol Classical Press, 1991, ISBN 978-0862921477

    work page 1971

  17. [17]

    Frye, Northrop. 1982. The Great Code: The B ible and Literature . Harcourt Brace Jovanovich

    work page 1982

  18. [18]

    Greenland, Sander. 2000. Principles of multilevel modelling. International Journal of Epidemiology, 29(1):158--167

    work page 2000

  19. [19]

    Griffith, Mark. 1975. Man and the leaves: A study of mimnermos fr. 2. California Studies in Classical Antiquity, 8:73--88

    work page 1975

  20. [20]

    Hacking, Ian. 2012. `` L anguage, truth, and reason'' 30 years later. Studies in History and Philosophy of Science Part A, 43(4):599--609

    work page 2012

  21. [21]

    Gyorkos, and Louis Coupal

    Joseph, Lawrence, Theresa W. Gyorkos, and Louis Coupal. 1995. Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. American Journal of Epidemiology, 141(3):263--272

    work page 1995

  22. [22]

    Kinney, Rodney, Chloe Anastasiades, Russell Authur, et al. 2023. The S emantic S cholar O pen D ata P latform. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Association for Computational Linguistics

    work page 2023

  23. [23]

    Lewis, Brett Daniel. 2023. Word Made Flesh: Biblicality in Cormac M c C arthy's Appalachian Novels . Ph.D. thesis, University of Memphis

    work page 2023

  24. [24]

    Li, Lucian. 2024. Tracing the genealogies of ideas with sentence embeddings. In Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities

    work page 2024

  25. [25]

    Lincoln, Kenneth. 2009. Cormac M c C arthy: A merican Canticles . Palgrave Macmillan

    work page 2009

  26. [26]

    Lipsitch, Marc, Eric Tchetgen Tchetgen, and Ted Cohen. 2010. Negative controls: A tool for detecting confounding and bias in observational studies. Epidemiology, 21(3):383--388

    work page 2010

  27. [27]

    Lowes, John Livingston. 1927. The Road to X anadu: A Study in the Ways of the Imagination . Houghton Mifflin

    work page 1927

  28. [28]

    Mundik, Petra. 2017. A Bloody and Barbarous God: The Metaphysics of Cormac M c C arthy . University of New Mexico Press

    work page 2017

  29. [29]

    Noble, O. Alan. 2020. The bible. In Steven Frye, editor, Cormac M c C arthy in Context . Cambridge University Press, pages 98--106

    work page 2020

  30. [30]

    Potts, Matthew L. 2015. Cormac M c C arthy and the Signs of Sacrament: Literature, Theology, and the Moral of Stories . Bloomsbury Academic

    work page 2015

  31. [31]

    Ricks, Christopher. 2002. Allusion to the Poets. Oxford University Press

    work page 2002

  32. [32]

    Schafer, William J. 1977. Cormac mccarthy: The hard wages of original sin. Appalachian Journal, 4(2):111--113

    work page 1977

  33. [33]

    Smith, David A., Ryan Cordell, and Abby Mullen. 2014. Detecting and modeling local text reuse. In Proceedings of the IEEE/ACM Joint Conference on Digital Libraries

    work page 2014

  34. [34]

    Sp \"a rck Jones, Karen. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28(1):11--21

    work page 1972

  35. [35]

    Tennyson, Hallam. 1897. Alfred Lord Tennyson: A Memoir by His Son, volume 2. Macmillan

    work page

  36. [36]

    Umphrey, Ryan, Zachary Roberts, and Michael Roberts. 2024. Investigating expert-in-the-loop LLM discourse patterns for ancient intertextual analysis. In Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities

    work page 2024

  37. [37]

    Wu, Zhaofeng, Linlu Qiu, Alexis Ross, Ekin Aky \"u rek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. 2024. Reasoning or reciting? E xploring the capabilities and limitations of language models through counterfactual tasks. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lingui...

    work page 2024

  38. [38]

    Yang, Yue, Yinzhi Xu, Chenghao Huang, JohnMichael Jurgensen, Han Hu, and Hao Wang. 2025. I nter IDEAS : Philosophical intertextuality via LLM s. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23162--23183, Association for Computational Linguistics

    work page 2025