{"id":"c0f4c716-0e59-45d2-8a4a-43f66ed82479","arxiv_id":"2601.17040","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FP-THD combines ParseNet layout analysis with an MAE-ViT line OCR to transcribe full pages of medieval Latin while preserving archaic characters, achieving 1.78% CER on a 10-page printed test set.","lead":"This paper builds a full-page transcription pipeline for 15th–16th century Latin documents by combining an existing layout-analysis model with a masked-autoencoder OCR model. It adds a newly curated printed-text dataset and reports lower character/word error rates than two open/commercial baselines on a 10-page test.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Molino test pages may overlap with training line images; no page-level holdout is stated for Table 6.","rationale":"The central claim is the full-page transcription performance on the 1585 Molino book. The only evidence is Table 6, which compares CER/WER on 10 pages against baselines. Unlike the Rodrigo and Bentham evaluations, where test sets are standard and disjoint by construction, the Molino test pages were created by the authors from the same book that supplied training data. The omission of a page-level holdout is not a stylistic detail but a necessary condition for interpreting the comparison. If the 10 test pages overlap with training pages, the model has memorized the text and the claimed 1.78% CER is meaningless. The ground-truth bootstrapping from Pero-OCR, while a concern, does not by itself invalidate the relative ranking: both FP-THD and Pero-OCR are scored against the same corrected reference, so the comparison is informative if the corrections are accurate. The lack of a layout ablation affects interpretation of component contributions but does not threaten the headline number. The data-leakage concern is the most load-bearing because it can turn a strong result into an artifact. The proposed check—verifying disjointness of test pages from training data—settles the issue directly and is feasible given the released repository.","tokens_in":12593,"tokens_out":7113,"duration_ms":74990,"concrete_test":"Inspect the released repository (https://github.com/IAAA-Lab/FP-THD) for the data split. If the split is not documented, request from the authors the page-level identifiers for (a) the 10 full test pages used in Table 6 and (b) all pages from [36] whose line crops appear in the 12,385 training / 1,377 validation images. Confirm that the two sets are disjoint. If any test page contributes line images to training or validation, the Table 6 result must be recomputed on a genuinely held-out set. As a complementary check, re-run the trained model on a new set of pages from [36] never used in any training/validation step and compare CER/WER to the reported values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that FP-THD outperforms Pero-OCR and BVPB on 10 full test pages from the 1585 Molino book (Table 6; §4.3), achieving CER 0.0178 and WER 0.0450. This is only interpretable as generalization if those 10 pages are disjoint from the pages used to construct the 12,385 training and 1,377 validation line images (Table 2). The paper never states such a holdout. Table 2's footnote only says 'testing is directly performed with full text pages in the case of the Molino dataset.' §4.1 lists 37 pages from [36] among the Molino sources, and §4.3 uses 10 pages from the same [36] book. Appendix A describes creating the line images and ground-truth text from this book using ParseNet and Pero-OCR. If any of the 10 test pages were included in the training or validation crops, the model has already seen that text and the reported CER reflects memorization, not recognition. This would invalidate the headline comparison and the conclusion that the pipeline generalizes to full pages. The line-level Rodrigo/Bentham results do not mitigate this, because those benchmarks use separate, standard test sets; the Molino full-page evaluation is the only one lacking the required holdout statement.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FP-THD, a full-page transcription pipeline for historical documents, combining ParseNet-based layout analysis with a masked-autoencoder/Vision-Transformer OCR model. The OCR component is trained and evaluated on three datasets: two handwritten corpora (Rodrigo, Bentham) and a newly curated printed medieval-Latin corpus (Molino). The authors report line-level CER/WER on Rodrigo and Bentham, and a full-page comparison on 10 pages of the Molino 1585 book, claiming the best CER (0.0178) and WER (0.0450) over Pero-OCR and the BVPB/ABBYY transcription, with better preservation of tildes and archaic characters.","tokens_in":12922,"tokens_out":3366,"duration_ms":38515,"significance":"If the full-page Molino result is valid, the paper offers a practical, open, and effective pipeline for a niche but important task: faithful transcription of medieval Latin printed documents without post-processing or a language model. The strengths include a public code/data repository, no reliance on commercial OCR, competitive line-level results on Rodrigo, and a documented effort to preserve diacritics and special characters. However, the central full-page evaluation is currently under-specified: the page-level train/test split is not stated, and the Molino ground truth was produced with the same OCR engine family used as a comparison baseline. These issues are fixable with documentation, but they currently prevent the strongest claim from being accepted as evidence of generalization.","major_comments":[{"comment":"The full-page evaluation on 10 Molino test pages lacks a page-level holdout statement. §4.1 lists 37 pages from [36] among the Molino training sources, and §4.3 tests on 10 pages from the same [36] book. The paper never states that those 10 pages are disjoint from the pages used to generate the 12,385 training and 1,377 validation line images. If there is overlap, the reported CER=0.0178 and WER=0.0450 may reflect memorization rather than recognition. Please provide page identifiers, the selection procedure, and an explicit confirmation of disjointness.","section":"§4.3, Table 6; Table 2 footnote"},{"comment":"The Molino training/validation line images and ground-truth texts were generated using Pero-OCR, with manual correction of 'part of the transcriptions'. Pero-OCR is the same engine family as the comparison baseline in Table 6. The paper does not quantify which pages or characters were manually corrected, how many annotators were involved, or whether the 10 test-page ground truths were independently transcribed from the source images. Without this information, the comparison is partially circular: the model is trained on targets derived from the baseline and then evaluated against that same baseline. Please clarify the annotation workflow and, if possible, provide an independent test transcription.","section":"Appendix A and Table 6"},{"comment":"The line-level comparisons with Granell et al. [37], Chammas et al. [38], and HTR-JAND [39] are informative but not fully head-to-head unless the test splits are identical. The paper does not state whether the Rodrigo and Bentham test partitions match those used by the cited baselines. Please specify the split definitions and, where available, report results on the exact standard partitions so that 'competitive with state-of-the-art' can be verified.","section":"§4.2, Tables 3 and 4"}],"minor_comments":[{"comment":"'ABBY' should be 'ABBYY' (the proprietary OCR engine). The same typo appears in the text of §4.3.","section":"§4.3, Table 6"},{"comment":"There is a typo in 'symbols such assemployed' — likely intended as 'such as s employed' with a formatting issue around the long-s glyph.","section":"Section 1"},{"comment":"The configuration keys appear as 'LA YOUT_PARSER' and 'LA YOUT_PARSER_2'; this is likely a copy-paste/rendering issue but should be corrected for reproducibility.","section":"Tables 1 and A.7"},{"comment":"The text refers to 'the input of our MSE model'; given the method section, this should probably be 'MAE model'.","section":"Appendix A"},{"comment":"Rodrigo is described as 'Gothic typography' but this is a handwritten corpus; consider rewording to 'Gothic script' or 'Gothic handwriting' to avoid confusion.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The central contribution is plausible and the line-level results on Rodrigo/Bentham give reasonable confidence in the OCR component. The main obstacle is that the full-page Molino claim rests on an underspecified page split and a ground-truth provenance that overlaps with the comparison baseline. If the authors can supply the page-level split and a clear account of the manual correction process, the paper would likely become acceptable. I do not see a need to reject, but the current text does not support the headline claim as written."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a legitimate engineering contribution — a full-page OCR pipeline for medieval Latin that wraps ParseNet and a MAE-ViT line recognizer, plus a new 143-page Molino dataset. If the numbers hold, it's useful for digital-humanities teams working on early modern printed texts. The pipeline itself is straightforward wiring, but it's well executed: they release code and data, and the line-level results on Rodrigo and Bentham are plausible and competitive. Those benchmarks use standard splits, so the line-level numbers carry real weight. The Molino dataset is the most valuable part — 143 pages of medieval Latin with abbreviations, manually corrected where needed. That is a genuine resource for the subfield.\n\nThe soft spot is the Molino full-page evaluation. Appendix A says the ground truth was generated with Pero-OCR, the same system used as a comparison baseline in Table 6, then partially manually corrected. The paper never states that the 10 test pages are disjoint from the training pages. It does say 37 pages from the same 1585 book were used for training; the test pages come from that same book. If there is overlap, the reported 1.78% CER is partly memorization, not recognition. The stress-test note is right to flag this — I checked the text and the only relevant statement is the footnote that testing is done on full pages, which does not address the overlap question. That is the load-bearing weakness of the headline claim. A second, minor gap: there is no ablation isolating layout errors from OCR errors, so we don't know whether ParseNet is helping or hurting on the more complex pages.\n\nTo be fair, these are reporting gaps, not evidence of sloppy methods. The line-level experiments are separate and solid. The qualitative claim about preserving tildes and archaic characters is supported by the example outputs. And the dataset release, with documentation of the curation process, is a concrete contribution. I would want the authors to state the page-level split explicitly, describe how much manual correction went into the test-page ground truth, and ideally add a few more test pages. None of that is a fundamental flaw; it is fixable experimental reporting.\n\nThis deserves a serious referee. It is not a paradigm-shift paper, but it is useful and the data is citable. I'd send it out with a request to tighten the experimental reporting, and I'd probably bring it to a reading group as an example of a well-scoped OCR integration with honest limitations stated in the discussion.","headline":"Solid integration paper with a genuinely useful new dataset; the headline full-page numbers need a clear page-level holdout statement before they can be read as generalization.","tokens_in":13374,"tokens_out":1676,"would_cite":true,"duration_ms":17813,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage pipeline transcribes full pages of 16th-century Latin print at 1.78% character error while keeping archaic symbols intact.","keywords":["historical document transcription","masked autoencoder","vision transformer","layout analysis","medieval Latin OCR","diacritic preservation","full-page OCR","OCR pipeline"],"falsifier":"Take the ten full-page test images from the 1585 book, match them against the page list used to crop the 12,385 training lines, and retrain the OCR model on all non-overlapping pages; if the disjoint-holdout CER/WER rises materially above 1.78%/4.50%, the published advantage is partly memorization rather than generalization.","tokens_in":12525,"feed_emoji":"📜","tokens_out":7293,"duration_ms":67953,"temperature":0.7,"pith_summary":"This paper attempts to show that full-page transcription of historical Latin documents can be done accurately without losing the archaic symbols that carry meaning. To do this it couples a layout-analysis model, which finds and crops text lines, with an OCR model built from a masked autoencoder and vision transformer trained on line images. On ten full pages of a 1585 printed law book the pipeline reports a character error rate of 1.78% and a word error rate of 4.50%, better than both an open OCR engine and a commercial transcription service, while preserving tildes, long-s forms, and ligatures. The same OCR component also reaches competitive error rates on two handwritten historical corpora without any post-processing or language-model correction.","feed_headline":"One pipeline reads 1585 Latin print at 1.8% character error","feed_subtitle":"Layout analysis plus a masked-autoencoder OCR preserves medieval tildes and abbreviations that modern tools drop.","key_machinery":"The load-bearing object is a two-component pipeline. The first component, ParseNet, is a convolutional encoder–decoder that detects baselines, ascenders, and descenders, outputs a PAGE-format XML layout (an XML standard for page geometry), and crops each line with geometric rectification to a standardized 50-pixel height. The second component is a masked autoencoder with vision transformer (MAE-ViT): a ResNet-18 feature extractor turns the line image into tokens, span masking (up to 8 consecutive tokens) replaces random masking as a regularizer, and CTC loss predicts the character sequence directly, without a language model. Because training data for Molino was annotated to preserve original","core_discovery":"The central claim is that the FP-THD pipeline—ParseNet for layout analysis followed by an MAE-ViT OCR model—can transcribe complete pages of medieval Latin printed material at character accuracy that outperforms existing tools. The authors created the Molino dataset from a 1585 edition of a civil-law commentary, trained the OCR on 12,385 line images, and tested on ten full pages. They report a CER of 1.78% and WER of 4.50%, with qualitative evidence that tildes and other diacritics are preserved better than by the Pero-OCR and ABBYY baselines.","pith_inferences":["Inference: The flagship Molino numbers should be read as provisional until the paper confirms that the ten full-page test images are not among the 12,385 training line images; Section 4.1, Table 2 does not state this split explicitly.","Inference: Because the Molino ground truth was generated from Pero-OCR output with manual correction (Appendix A), an independent evaluator could re-measure the pipeline against freshly hand-transcribed pages to rule out annotation bias toward any recognizer.","Inference: The same two-stage recipe—layout detection plus a span-masked ViT OCR trained on line images—could transfer to other early-modern scripts, such as German Fraktur, that also use abbreviation systems, provided enough annotated lines are created.","Inference: A direct extension would test whether preserved tildes and abbreviations actually improve downstream abbreviation-expansion and Latin-to-Spanish translation quality compared with modernized transcriptions."],"forward_implications":["If the reported numbers hold, the pipeline makes medieval Latin pages machine-searchable without modernizing their spelling, which matters for legal and historical scholarship.","The OCR component's performance on the Rodrigo and Bentham corpora suggests the same line recognizer can be reused across handwriting styles with only retraining, not a new architecture.","Because the system emits PAGE XML, Markdown, and plain text, the transcription can enter collaborative revision workflows with version control.","No post-processing or external language model is used, so the measured errors come from the visual model itself; future gains from decoding or context are additive.","At 4.5% word error rate the output is usable as input for the paper's planned downstream steps, such as automatic expansion of Latin abbreviations and translation."],"fun_headline_variants":["Transcribe full 1585 Latin pages at 1.8% CER","Medieval diacritics preserved by new full-page OCR pipeline","Layout analysis + MAE OCR reads 1585 Latin at 1.8% CER","Pipeline preserves tildes and abbreviations in historical transcription","One pipeline, full-page historic docs, 1.8% CER on 1585 Latin"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The full-page result depends on the ten test pages being disjoint from the pages used to build the 12,385 training line images, a separation the paper never explicitly states.","fun_headline_variants_meta":{"raw":{"variants":["Transcribe full 1585 Latin pages at 1.8% CER","Medieval diacritics preserved by new full-page OCR pipeline","Layout analysis + MAE OCR reads 1585 Latin at 1.8% CER","Pipeline preserves tildes and abbreviations in historical transcription","One pipeline, full-page historic docs, 1.8% CER on 1585 Latin"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000531,"raw_usage":{"total_tokens":2342,"prompt_tokens":637,"completion_tokens":1705,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":381,"completion_tokens_details":{"reasoning_tokens":1604}},"tokens_in":381,"tokens_out":1705,"duration_ms":12216,"temperature":1.0,"reasoning_tokens":1604,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T09:26:00.085443+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the ten full-page test images from the 1585 book, match them against the page list used to crop the 12,385 training lines, and retrain the OCR model on all non-overlapping pages; if the disjoint-holdout CER/WER rises materially above 1.78%/4.50%, the published advantage is partly memorization rather than generalization.","supporting_citations":[],"review_version":1}