The \textit{Questio de aqua et terra}: A Computational Authorship Verification Study

Alejandro Moreo; Fabrizio Sebastiani; Martina Leocata

REVIEW 3 major objections 5 minor 15 references

The \textit{Questio de aqua et terra}: A Computational Authorship Verification Study

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A computational authorship-verification study attributes the disputed 'Questio de aqua et terra' to Dante with posterior probability 0.999999967.

desk verdict A serious, transparent computational effort with a genuine methodological contribution, but the headline probability for Dante's authorship is not supported by the evidence once the closed-world assumption and the model-selection protocol are examined. read the letter →

arxiv 2501.05480 v1 pith:JW6P2B6P submitted 2025-01-07 cs.CL cs.DL

Martina Leocata , Alejandro Moreo , Fabrizio Sebastiani This is my paper · ORCID

classification cs.CLcs.DL

keywords authorshipverificationQuestiodeaquaetterraDanteAlighieristylometrydistributionalrandomoversamplingmedievalLatinlogisticregressionculturalheritage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the Latin cosmological treatise 'Questio de aqua et terra' was written by Dante Alighieri, a question that has divided scholars because the work's cosmology seems to clash with Dante's other writings and no contemporary manuscript survives. To answer it, the authors assemble a corpus of 330 thirteenth- and fourteenth-century Latin texts, train authorship verifiers that combine logistic regression with stylometric features, and evaluate them by leave-one-out cross-validation. Their best verifier is correct on 329 of the 330 texts ($F_1=0.970$), and the main reason for that accuracy is Distributional Random Oversampling (DRO), a technique that synthesizes training examples for the scarce Dante class and lifts $F_1$ from 0.400 to 0.970. Applied to the Questio, the verifier attributes the text to Dante with posterior probability $\Pr(\mathrm{Dante}\mid\mathrm{Questio})=0.999999967$. The paper's wider point is that DRO makes computational authorship verification practical for cultural-heritage texts, where only a handful of texts by the candidate author usually survive.

What carries the argument

The load-bearing mechanism is Distributional Random Oversampling (DRO), a technique that generates synthetic training examples for the minority class by appending random 'latent' features to each text vector, exploiting the distributional hypothesis that words—and, here, syntactic and character-level units—with similar distributions serve similar functions. DRO is used for the first time in authorship verification, and ablating it drops $F_1$ from 0.970 to 0.400; removing character n-grams drops it to 0.455. The supporting machinery is the MedLatinQuestio corpus of 330 medieval Latin texts by 38 authors, segmented into 5,430 training instances, with the final configuration using 12,807 natural features plus 21,770 DRO-generated latent features at a 20/80 positive/negative ratio.

What would settle it

The closed-world premise would be falsified if adding even a small set of Latin works by a previously excluded fourteenth-century cosmologist to the MedLatinQuestio corpus and rerunning the leave-one-out protocol caused the verifier to reclassify the Questio as non-Dantean or materially lowered its confidence.

Watch

Extended reading notes

Core claim

The paper's central claim is that the Questio is Dantean and that computational verification can establish this with near-certainty. On the authors' own evaluation, a logistic-regression verifier using token lengths, function words, sentence lengths, POS n-grams, and character n-grams, supplemented by DRO-generated synthetic examples, makes 329 correct decisions out of 330 in leave-one-out tests on the MedLatinQuestio corpus; the sole error is Boccaccio's Epistle 23, misclassified as Dantean. Trained on all 330 texts and applied to the Questio, the verifier reports $\Pr(\mathrm{Dante}\mid\mathrm{Questio})=0.999999967$, equivalent to about 33 chances in a billion that the text is non-Dantean. An authorship-attribution system trained on the same corpus ranks Dante first among 38 authors at probability 0.737, with the nearest competitor at 0.067. The authors read this combination, together with the near-impossibility of consciously imitating character n-gram frequencies, as strong evidence against the forgery hypothesis.

Load-bearing premise

The load-bearing premise is that the true author of the Questio, if not Dante, is one of the 38 authors in the reference corpus; if the real author lies outside that closed set, the verifier's near-certain probability does not measure anything about the true authorship.

Editorial extensions

If this is right

If the verifier's result is correct, the Questio joins Dante's Latin corpus, and the philological arguments from cosmology and from the absence of contemporary references need to be squared with a Dantean attribution.
DRO's contribution implies that authorship verification on imbalanced cultural-heritage corpora can be substantially improved by distributional oversampling, not just by better feature engineering.
The feature ablation results indicate that character n-grams and DRO-generated latent features carry most of the authorship signal; future verifiers for medieval Latin should treat those as core rather than optional.
The MedLatinQuestio corpus and code, released with the paper, let other researchers reproduce the $F_1=0.970$ result and apply the same pipeline to other disputed texts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's 'Adso da Melk' caveat is left qualitative: the 0.999999967 probability is conditional on the true author being one of the 38 comparators, so adding a genuine non-Dantean author to the corpus could in principle flip the verdict even if the current estimate is stable.
The authors used DRO on non-content features and found it works, which suggests the distributional hypothesis operates below the word level; a direct test would ablate DRO per feature type to see which levels contribute most.
A natural extension is to run the same verifier on the 1509 printed edition of the Questio and on the disputed Pietro Alighieri commentary that mentions it; textual variants there could change the feature vectors that drive the attribution.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth your time, but read it with the headline claim in brackets. The paper does three things well: it builds a new medieval Latin corpus (MedLatinQuestio), it applies Distributional Random Oversampling to authorship verification for the first time, and it publishes code and data so the results can be checked. The LOO F1=0.970 on 330 heterogeneous texts is solid, and the ablation showing DRO lifts F1 from 0.400 to 0.970 is genuinely striking. The AA cross-check, with Dante ranking first among 38 authors, is a useful second opinion, not a rhetorical add-on.

The soft spots are real and tend to cluster around the strength of the final attribution. First, the closed-world premise: LOO removes one document but keeps other texts by the same author in training, so it measures recognition of a known style, not rejection of an unseen author. The paper's own "Adso da Melk" passage concedes this, then dismisses it with an empirical claim — that character n-grams are effectively impossible to imitate — that is asserted, not demonstrated. That leaves Pr(Dante|Questio)=0.999999967 as a closed-world object, not an open-world probability that the text is non-Dantean. Second, the reported F1 is not an unbiased estimate of the final system, because the same corpus was used to select the configuration (the "10 hardest texts" guided the feature ablation). A nested validation would fix this. Third, the extreme posterior is not obviously calibrated: the system's soft F1 is 0.900, so its confidence scores are only moderately reliable on average, yet the Questio receives a probability with nine nines. Reporting that number without a calibration caveat oversells the result.

These are fixable problems, not fatal ones. The paper is honest about many of its limitations, the corpus and code are public, and the DRO self-citation is legitimate because the method is theirs and it is evaluated on held-out texts. The right outcome is a revision, not a rejection: add a nested validation, run an explicit open-set or unseen-author experiment, and hedge the conclusion so that it says "consistent with Dantean authorship under a closed-world model" rather than "true author." I would send this to a serious natural-language-processing or digital-humanities venue, and I would bring it to a reading group myself.

Referee Report

3 major / 5 minor

Summary. The paper presents a computational authorship verification study of the disputed treatise 'Questio de aqua et terra', attributed by tradition to Dante Alighieri. The authors assemble a corpus of 330 medieval Latin texts (MedLatinQuestio), train a binary classifier (logistic regression with hand-crafted stylometric feature sets) to distinguish Dante from non-Dante, and report a leave-one-out F1 of 0.970. They attribute the high performance mainly to Distributional Random Oversampling (DRO), which is used here for the first time in authorship verification. When the trained verifier is applied to the Questio, it returns a posterior probability Pr(Dante|Questio)=0.999999967, which the authors interpret as strong evidence for Dantean authorship. Additional authorship attribution experiments on 38 candidate authors also rank Dante first, with a probability of 0.737. The authors make their code and dataset publicly available.

Significance. If the attribution result holds, this work would provide a strong quantitative contribution to a long-standing philological debate and would showcase DRO as a useful technique for authorship verification in low-resource historical corpora. The paper is carefully structured, the corpus is a valuable resource, and the experiments are described in sufficient detail to be reproduced. The main strength is the public release of code and data, and the explicit consideration of topic/genre confounding in feature selection. However, the central claim of near-certain Dantean authorship rests on a closed-world assumption that the paper itself partially acknowledges; this limits the significance of the strong posterior probability until an open-set evaluation is provided.

major comments (3)

[Section 5 and Section 6] The claim that Pr(Dante|Questio)=0.999999967 means '33 chances in a billion' that the Questio is non-Dantean is an overinterpretation of a model posterior. The logistic regression posterior is calibrated only on average over the 330-text corpus (F_s1=0.900); there is no evidence that probabilities at the 1e-9 level are meaningful. The paper should either report the raw confidence and explicitly state that extreme posterior values are not validated, or temper the wording in the abstract and Section 5.
[Section 3.4 and Section 4] The leave-one-out protocol leaves out one document while other texts by the same author remain in the training set, so it measures closed-set recognition of known authors, not rejection of an author entirely absent from the training set. The 'Adso da Melk' scenario in Section 6 concedes that a real non-Dantean author outside the 38-author set could, in principle, lead the verifier to a different outcome. The conclusion that Dante is the true author with near-certainty is therefore not supported by the current LOO evidence; an explicit open-set evaluation (e.g., training on a subset of authors and testing on held-out authors) is needed to substantiate the open-world claim.
[Section 4, Table 2 and ablation procedure] The reported F1=0.970 is computed on the same 330 texts that were used to guide the greedy ablation and to identify the '10 hardest texts' on which the final configuration was selected. This makes the reported accuracy an optimistic, selection-dependent estimate rather than an unbiased estimate of the deployed verifier's performance. The authors should either use nested cross-validation or explicitly describe the reported F1 as a model-selection result, not an independent accuracy estimate.

minor comments (5)

[Section 3.1] Typo: 'for the agoals of our task' should read 'for the goals of our task'.
[Section 3.3] Typo: 'morphosyntactyic verbal endings' should read 'morphosyntactic verbal endings'.
[Section 6] The vanilla accuracy value is reported as 0.996 in Section 6, while Section 4 computes it as 329/330=0.997. These values should be made consistent.
[Table 1 caption] The caption states that the 3rd column indicates whether the inference is correct and the 4th column the confidence, but the table appears to have only three columns, with the ✓/✗ symbol placed inside the 'Text' column. The caption should be updated to match the actual table layout.
[Reference list] The Kestemont (2014) reference gives the workshop year as 'CLfL 2024'; the year should be 2014 to match the publication date.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attribution is an out-of-sample application of a model evaluated on held-out texts; the self-cited DRO method is validated by the paper's own ablation, and the closed-world caveat is an evidence limitation, not a definitional shortcut.

full rationale

The paper's derivation chain is not circular. The authorship verifier is trained on the 330 labeled MedLatinQuestio texts and then applied to the Questio, which is never used in training or configuration selection. Section 3.4 specifies the LOO protocol: 'training the verifier ... on all the texts in L \ {x}' and 'applying the trained verifier on the (unsegmented) text x,' so each LOO decision is an out-of-sample prediction. Section 5 then states: 'we use the maximally accurate configuration to train an authorship verifier on the full set of 330 texts of undisputed authorship, and ... apply the resulting verifier to the Questio.' The reported Pr(Dante|Questio)=0.999999967 is the model's posterior for an unlabeled document, not a value fitted to that document. DRO, although introduced in prior work by two of the authors, is evaluated here through ablation: Table 4.1 shows F1 dropping from 0.970 to 0.400 without DRO, so its contribution is empirically measured on held-out texts rather than assumed from the citation. The paper explicitly acknowledges the closed-world limitation in Section 6 with the 'Adso da Melk' scenario: if the true author is not among the 38 authors, the verifier's answer could change. This is a genuine external-validity caveat about the comparator set, but it is not a case where the conclusion is equivalent to an input by construction; the target text remains unlabeled and the experiments are independent of the verdict. The '33 chances in a billion' phrasing is a model posterior over the two trained classes, not an open-world probability, but that is an interpretability caveat, not circularity. No equation is defined in terms of the target outcome, no fitted parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on five modeling assumptions and five tunable choices. The most scientifically consequential are the closed-world comparator premise and the topic/genre-neutrality of features. No new physical or philological entities are introduced; DRO's latent features are procedural artifacts, not independent entities.

free parameters (5)

DRO target class balance = 20/80 positive/negative
Set by hand based on the oversampling literature; affects the decision boundary, the synthetic positive examples, and the final confidence score.
Segment length threshold = 400 tokens
Chosen after preliminary experiments (300 and 500 gave worse accuracy); segments are used as training examples and shape the style model.
Feature type set after greedy ablation = 5 feature types selected from an initial pool of 8
The ablation process is guided by accuracy on the 10 hardest texts in the same corpus, making the configuration a data-dependent choice that affects the LOO estimate.
N-gram orders for character and POS features = n = 1, 2, 3
Preliminary experiments on the same corpus showed no improvement for higher n; these orders define the dimensionality and the stylistic signal.
Logistic regression regularization strength C = optimized per LOO fold
Internal hyperparameter tuned independently in each training run; standard model selection, not a scientific parameter tied to the attribution.

assumptions (5)

domain assumption Authors leave a consistent stylistic fingerprint across their texts.
Underlies all computational authorship identification; invoked in Section 2. If false, the LOO results and the Questio verdict are meaningless.
domain assumption The chosen features are mostly topic- and genre-neutral, so they capture style rather than cosmology or genre.
Stated in Section 3.3. The Questio is cosmologically and generically closer to the non-Dantean class, and the authors rely on this to argue that topic or genre confounds would bias against Dante.
domain assumption The 330-text corpus fairly represents the relevant universe of medieval Latin authors, i.e., the true author, if not Dante, is among the 38 authors.
Acknowledged in Section 6 via the 'Adso da Melk' hypothetical. If the true author is outside the corpus, the AV and AA systems cannot detect them and the closed-world verdict does not answer the open-world question.
domain assumption Segments of the same text can be treated as independent training examples.
Segmenting 330 texts into 5,430 examples creates within-author correlation that is not accounted for in training, affecting the interpretation of the LOO accuracy.
domain assumption DRO's synthetic vectors preserve the statistical regularities of the original texts.
DRO relies on the distributional hypothesis, and the authors note in Section 6 that they use it 'in an unorthodox way' on function words, POS n-grams, and other non-content features. The validity of this extension is assumed rather than demonstrated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The \textit{Questio de aqua et terra}: A Computational Authorship Verification Study." pith.science (2026). https://pith.science/paper/JW6P2B6P

@misc{pith2026250105480,
  author       = {Pith},
  title        = {Pith review of: The \textitQuestio de aqua et terra: A Computational Authorship Verification Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JW6P2B6P}},
  note         = {Machine review of arXiv:2501.05480}
}

read the original abstract

The Questio de aqua et terra is a cosmological treatise traditionally attributed to Dante Alighieri. However, the authenticity of this text is controversial, due to discrepancies with Dante's established works and to the absence of contemporary references. This study investigates the authenticity of the Questio via computational authorship verification (AV), a class of techniques which combine supervised machine learning and stylometry. We build a family of AV systems and assemble a corpus of 330 13th- and 14th-century Latin texts, which we use to comparatively evaluate the AV systems through leave-one-out cross-validation. Our best-performing system achieves high verification accuracy (F1=0.970) despite the heterogeneity of the corpus in terms of textual genre. The key contribution to the accuracy of this system is shown to come from Distributional Random Oversampling (DRO), a technique specially tailored to text classification which is here used for the first time in AV. The application of the AV system to the Questio returns a highly confident prediction concerning its authenticity. These findings contribute to the debate on the authorship of the Questio, and highlight DRO's potential in the application of AV to cultural heritage.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

[1]

Aggarwal, C. C. (2018). Machine learning for text . Springer International Publishing, Cham, CH

work page 2018
[2]

Aseervatham, S., Gaussier, \' E ., Antoniadis, A., Burlet, M., and Denneulin, Y. (2012). Logistic regression and text classification. In Gaussier, \' E . and Yvon, F., editors, Textual Information Access: Statistical Models , pages 61--84. Wiley, Chichester, UK

work page 2012
[3]

Casadei, A. (2025). Analysis of the 1509 edition and implications on the various reconstructions [in Italian] . In Casadei, A. and Pontari, P., editors, La Questio de aqua et terra: Nuove indagini e prospettive . Pisa University Press, Pisa, IT. Forthcoming

work page 2025
[4]

Corbara, S., Moreo, A., and Sebastiani, F. (2023). Syllabic quantity patterns as rhythmic features for L atin authorship attribution. Journal of the Association for Information Science and Technology , 74(1):128--141

work page 2023
[5]

Corbara, S., Moreo, A., Sebastiani, F., and Tavoni, M. (2022). MedLatinEpi and MedLatinLit : Two datasets for the computational authorship analysis of medieval Latin texts. ACM Journal of Computing and Cultural Heritage , 15(3):57:1--57:15

work page 2022
[6]

Fioravanti, G. (2017). Alberto di Sassonia, Biagio Pelacani, and the «Questio de aqua et terra» [in Italian] . Studi danteschi , LXXXII:81--97

work page 2017
[7]

D., and Madigan, D

Genkin, A., Lewis, D. D., and Madigan, D. (2007). Large-scale Bayesian logistic regression for text categorization. Technometrics , 49(3):291--304

work page 2007
[8]

Kestemont, M. (2014). Function words in authorship attribution: F rom black magic to theory? In Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLfL 2024) , pages 59--66, Gothenburg, SE

work page 2014

Show all 15 references

[9]

and Sahlgren, M

Lenci, A. and Sahlgren, M. (2023). Distributional semantics . Cambridge University Press, Cambridge, UK

2023
[10]

Moreo, A., Esuli, A., and Sebastiani, F. (2016). Distributional random oversampling for imbalanced text classification. In Proceedings of the 39th ACM Conference on Research and Development in Information Retrieval (SIGIR 2016) , pages 805--808, Pisa, IT

2016
[11]

Pearl, J. (2009). Simpson's paradox, confounding, and collapsibility. In Causality: Models, Reasoning and Inference , chapter 6. Cambridge University Press, Cambridge, UK, 2nd edition

2009
[12]

Savoy, J. (2020). Machine learning methods for stylometry: A uthorship attribution and author profiling . Springer, Cham, CH

2020
[13]

Setzu, M., Corbara, S., Monreale, A., Moreo, A., and Sebastiani, F. (2024). Explainable authorship identification in cultural heritage applications. ACM Journal of Computing and Cultural Heritage , 17(3):Article 44

2024
[14]

Stamatatos, E. (2018). Masking topic-related information to enhance authorship attribution. Journal of the Association for Information Science and Technology , 69(3):461--473

2018
[15]

Zhang, J., Jin, R., Yang, Y., and Hauptmann, A. G. (2003). Modified logistic regression: An approximation to SVM and its applications in large-scale text categorization. In Proceedings of the 12th International Conference on Machine Learning (ICML 2003) , pages 888--895, Washi...

2003

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Aggarwal, C. C. (2018). Machine learning for text . Springer International Publishing, Cham, CH

work page 2018

[2] [2]

Aseervatham, S., Gaussier, \' E ., Antoniadis, A., Burlet, M., and Denneulin, Y. (2012). Logistic regression and text classification. In Gaussier, \' E . and Yvon, F., editors, Textual Information Access: Statistical Models , pages 61--84. Wiley, Chichester, UK

work page 2012

[3] [3]

Casadei, A. (2025). Analysis of the 1509 edition and implications on the various reconstructions [in Italian] . In Casadei, A. and Pontari, P., editors, La Questio de aqua et terra: Nuove indagini e prospettive . Pisa University Press, Pisa, IT. Forthcoming

work page 2025

[4] [4]

Corbara, S., Moreo, A., and Sebastiani, F. (2023). Syllabic quantity patterns as rhythmic features for L atin authorship attribution. Journal of the Association for Information Science and Technology , 74(1):128--141

work page 2023

[5] [5]

Corbara, S., Moreo, A., Sebastiani, F., and Tavoni, M. (2022). MedLatinEpi and MedLatinLit : Two datasets for the computational authorship analysis of medieval Latin texts. ACM Journal of Computing and Cultural Heritage , 15(3):57:1--57:15

work page 2022

[6] [6]

Fioravanti, G. (2017). Alberto di Sassonia, Biagio Pelacani, and the «Questio de aqua et terra» [in Italian] . Studi danteschi , LXXXII:81--97

work page 2017

[7] [7]

D., and Madigan, D

Genkin, A., Lewis, D. D., and Madigan, D. (2007). Large-scale Bayesian logistic regression for text categorization. Technometrics , 49(3):291--304

work page 2007

[8] [8]

Kestemont, M. (2014). Function words in authorship attribution: F rom black magic to theory? In Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLfL 2024) , pages 59--66, Gothenburg, SE

work page 2014