Pith. sign in

REVIEW 3 major objections 6 minor 29 references

This paper claims that ensembling multiple test-time augmented transcriptions of the same historical document image with a Needleman–Wunsch style aligner improves field transcription accuracy by about four percentage points over a single-pa

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:55 UTC pith:RWNVR5J2

load-bearing objection A clean, useful test-time augmentation ensemble for MLLM document extraction, but the headline 4-point gain rests on an evaluation protocol that may have leaked validation data into the reported results. the 3 major comments →

arxiv 2509.09722 v1 pith:RWNVR5J2 submitted 2025-09-10 cs.CV cs.CLcs.LG

Improving MLLM Historical Record Extraction with Test-Time Image

Taylor Archibald , Tony Martinez This is my paper
classification cs.CV cs.CLcs.LG
keywords test-time augmentationensemble votingmultimodal LLMhistorical document transcriptionNeedleman–Wunsch alignmentconfidence calibrationstructured field extractiongenealogical records
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that a single multimodal LLM, queried on several mildly distorted copies of a scanned historical record and fused by a voting aligner, extracts structured fields more reliably than one clean pass—and that the degree of agreement among the copies is a usable confidence signal. On 622 Pennsylvania death records, the ensemble raises field accuracy from 71.2% to 75.2% and cuts character error rate from 9.0% to 7.2%, while recovering most of the fields the baseline missed entirely. It also shows that augmentation selection should target the consensus, not individual sample accuracy, and that different distortions serve different purposes: padding and blur help accuracy most, grid-warp helps confidence calibration most. A sympathetic reader would care because the method is black-box, needs no fine-tuning, and transfers to any closed transcription model.

Core claim

On a new corpus of 622 early-20th-century Pennsylvania death records with 3,684 nonblank name fields, the authors show that querying the multimodal LLM Gemini 2.0 Flash on 10 mildly distorted copies of each scan—padding, blur, noise, warp, resize—and aligning the returned strings with a progressive Needleman–Wunsch vote yields 75.2% field accuracy and 7.2% character error rate, against 71.2% field accuracy and 9.0% CER for a single pass. The alignment's per-column vote share doubles as a confidence score; grid-warp ensembles reach 0.642 raw correlation between confidence and correctness. The authors interpret the gain through an effective-sample-size argument: correlated augmentations give a

What carries the argument

The load-bearing object is the progressive Needleman–Wunsch consensus aligner. Each new transcription is aligned to the evolving consensus with match=+1, mismatch=-1, gap=-1; every alignment column accumulates a vote tally, the consensus character is the most-voted non-gap symbol, and confidence is that vote count divided by the number of transcriptions. This turns an unordered set of strings into one string plus a per-character, per-word confidence without any access to model internals. A secondary theoretical object is the effective sample size Neff = N/(1+(N−1)ρ), which frames the role of augmentation diversity: correlated errors shrink the number of independent voters, so the best augmen

Load-bearing premise

The reported gain is only meaningful if the records used to select the best ten augmentations were held out from the final evaluation; if the same data guided both choices, selection inflates the four-point improvement.

What would settle it

Run the full protocol on a fresh, never-touched set of historical death records and compare ensemble field accuracy against a single-pass baseline; if the gap shrinks toward zero, the reported gain was selection-driven. Separately, re-estimate the grid-warp confidence–accuracy correlation out-of-fold; a drop toward the 0.202 seen for Resize would show the calibration result is configuration-specific.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • The same one-model pipeline can be dropped onto other document collections and proprietary transcription models without retraining or logprob access.
  • Choosing augmentations by their contribution to consensus validation accuracy outperforms choosing the individually most accurate configurations and approaches an oracle selector.
  • Padding and blur/resize give the biggest accuracy gains, while grid-warp distortions give the strongest confidence–accuracy correlation (0.642), so a deployment could mix strategies: accuracy-oriented for output, warp-oriented for triage.
  • The agreement-based confidence score can flag fields for human review; unanimous grid-warp predictions are especially precise.
  • Ensembling recovers most fields the single pass omits (4–7 of 7 on this dataset), so the gain comes partly from recall, not only correction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the 'token jitter' mechanism the paper proposes for padding could be tested head-on by logging the tile grid under controlled pixel shifts; if confirmed, the method's benefit would extend to any fixed-layout document, not only historical records.
  • Editorial inference: if the agreement-based confidence generalizes across models, this NW-voting procedure could serve as a model-agnostic uncertainty layer for proprietary transcription APIs, replacing unreliable verbalized confidence.
  • Editorial inference: the paper's selection result implies that searching explicitly for low error-correlation augmentation pairs—rather than scoring single-configuration accuracy—could squeeze out most of the remaining gap to the oracle bound.
  • Editorial inference: because the method uses one model and no training, it is a natural substrate for active-learning loops: low-confidence fields from the vote can be sent to a human, and the corrected transcripts can be used to audit or specialize the transcription policy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a test-time augmentation ensemble for MLLM-based extraction from historical documents. Given an input image, it applies N transformations from one of several distortion families (blur, resize, Gaussian noise, pixel-shift padding, grid warp) or samples the model at different temperatures, obtains candidate field transcriptions from Gemini 2.0 Flash, and progressively aligns them with a Needleman-Wunsch-style dynamic program to produce a consensus transcription and a per-character agreement-based confidence score. The authors contribute a new dataset of 622 Pennsylvania death records with 3,684 non-blank name fields, and report that the best validation-selected consensus ensemble improves field accuracy from 71.2% (single-pass baseline) to 75.2% at 10 samples, with grid-warp augmentations giving the best confidence-accuracy correlation (0.642). The paper also reports calibration metrics, error-correlation analyses, and qualitative failure cases.

Significance. If the reported gain is real and generalizes, the method is practically valuable: it is simple, black-box, model-agnostic, and requires no fine-tuning or access to internal logits, while also providing a confidence signal that is useful for triage. The new death-record dataset is a useful resource, and the paper draws a clean, testable distinction between augmentations that improve raw accuracy (padding, blur) and those that improve confidence separation (grid warp). The qualitative analysis of remaining failure modes is honest and informative. However, the headline result is currently vulnerable to a validation/testing ambiguity, and the confidence-calibration claims are stronger than the raw calibration numbers support.

major comments (3)
  1. [§3.5 and §4, Table 2] The central 4-point accuracy claim is not yet supported by the evaluation protocol as described. §3.5 says the top-10 augmentation configurations are selected by 5-fold cross-validation on a 'dedicated validation set', but §4 reports Table 2's 'Best Validation CER after Consensus' as the final result without introducing a separate held-out test partition or stating that the numbers are aggregated out-of-fold. If the reported 75.2% field accuracy is computed on the same validation data used to select configurations, the improvement over the 71.2% baseline is inflated by selection over a large pool of configurations, and the small gap to the Oracle method (75.2 vs 76.2) is consistent with overfitting to the validation set. Please state the exact split/protocol, or report results on a held-out test split with confidence intervals.
  2. [§3.6 and Table 3] The abstract and conclusion call the confidence score 'well-calibrated', but Table 3 shows raw ECE values of 0.1677 for Grid Warp and roughly 0.17–0.28 for the other transforms. Only after isotonic regression does the ECE drop below 0.05. Because the confidence score is defined as the fraction of ensemble votes for the consensus character, it also inherits any systematic model bias; unanimous agreement is not independent evidence of correctness. Please qualify the 'well-calibrated' claim as applying after the recalibration step, or include the isotonic regression as an explicit required part of the pipeline, and discuss the moderate raw calibration as a limitation.
  3. [§4, Tables 1–2] The paper reports all accuracy and calibration results as point estimates with no uncertainty quantification. The comparisons are made on 3,684 fields drawn from 622 documents, so fields are not independent and several differences between methods are only 0.5–1.0 percentage point (e.g., 7.5 vs 7.0 CER comparing Best Validation CER and Oracle at 5 samples). Please provide bootstrap or permutation confidence intervals, or a paired statistical test, and account for document-level clustering. This is necessary to support the ordinal claims about which augmentation and selection strategies are most effective.
minor comments (6)
  1. [§3.4, §3.5, and Table 3] The text says 20 samples are generated for each temperature setting (0.5, 1.0, 2.0), but Table 3 lists 12 experiments for those temperature rows. Clarify the discrepancy.
  2. [§3.2] The theoretical analysis is presented as a 'perspective', but the variance expression assumes exchangeable/correlated Bernoulli variables and the 'Hoeffding-style' bound is not a formal concentration result for dependent variables. Please label this subsection explicitly as heuristic and avoid language that implies a proof.
  3. [§3.7] The error-correlation measure is defined as the Pearson correlation of binary per-field error vectors. The text should state clearly that this is field-level, not character-level, and describe how pairs of transcriptions are pooled across the dataset.
  4. [§1 and Abstract] The abstract says '4 percentage points' while the introduction says 'over 4% relative'. These are different quantities; use one consistent expression.
  5. [§3.3 and §3.4] No dataset URL or code release is provided. For reproducibility, please release the alignment code and either the full dataset or a representative sample, with appropriate licensing information.
  6. [Figure 5] The caption introduces numbers that are not in the main text: '870 had consistent predictions using the full pool of 100 augmentation configurations' and '3 of the 7 times'. Please move this context into the main text.

Circularity Check

1 steps flagged

The 4-point accuracy gain is the validation-CER objective used to select augmentations, reported without a separate test partition.

specific steps
  1. fitted input called prediction [Sec. 3.5 (augmentation selection) and Sec. 4 / Table 2 (headline results)]
    "For the image-distortion strategies, we conduct a grid search over 20 configurations and create an ensemble from the top 10. This selection is based on a 5-fold cross-validation performance on a dedicated validation set ... Best Validation CER after Consensus ... With 10 samples, this method achieves a 7.2% CER and a 75.2% Field Accuracy, a 4 percentage point improvement in accuracy over the 71.2% baseline."

    The top-10 augmentation configuration set is the fitted input: it is explicitly chosen to minimize validation CER. The headline result is then reported as 'Best Validation CER after Consensus' (75.2% field accuracy / 7.2% CER) on the same validation criterion. No held-out test partition is introduced in Section 4, so the claimed 4-point improvement is the value of the selection objective on the data used for selection, not an independent out-of-sample prediction. The small gap to the Oracle (76.2%) is consistent with selection overfitting rather than demonstrated generalization.

full rationale

The central ensembling/voting mechanism is not circular: it is a standard NW/ROVER-style consensus with a confidence score defined as the vote fraction of the consensus character, and its calibration (correlation 0.642, ECE, Brier) is an empirical result, not a tautology. There are no load-bearing self-citations and no ansatz smuggled in via cited prior work by the present authors; the paper explicitly builds on ROVER and NW. The one substantive circularity is evaluative: the augmentation pool is selected by validation CER, and the reported 4-point gain appears to be measured on that same validation criterion/set, with no separate test partition described. This makes the headline improvement a partially fitted score rather than a true prediction, so the circularity score is 6 rather than higher.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The method is empirical and depends on a hand-tuned augmentation grid, ensemble size, alignment scoring, and validation-based augmentation selection. It introduces no new theoretical entities or physical constructs. The central empirical claims rest on the quality of the private Ancestry dataset, the validity of the label-preserving assumption, and the integrity of the evaluation split, none of which are independently verifiable from the paper.

free parameters (5)
  • Augmentation configuration grids = Blur kernels 5-17px, resize scales 100/75/50%, noise patch sizes {2,4,8,16} and sigma {4,6,8,10,15}, padding offsets {8,
    These ranges are hand-chosen and the top-10 configurations per transform are selected by validation performance, so the final ensemble depends on multiple data-fitted choices.
  • Baseline resize factor = 0.5
    The authors swept resizing factors and chose 50% as the smallest scale that performed no worse than original, a data-driven design choice.
  • Ensemble size N = 5 and 10 (with 20 candidates generated)
    The reported headline improvement uses 10 samples; results vary with ensemble size, so N is a hand-selected parameter.
  • Needleman-Wunsch scoring = match=+1, mismatch=-1, gap=-1
    The alignment scoring is hand-set and described as robust, but it directly affects the consensus and confidence values.
  • Sampling temperature and top-p = temperatures 0.5, 1.0, 2.0 with top-p=0.95
    These values are chosen for the temperature-based ensemble experiments and represent additional design choices.
axioms (5)
  • domain assumption Augmentations are label-preserving: A_theta(I) maps to the same ground truth y
    Section 3.1 defines the augmentation operators as label-preserving. If heavy warp, padding, or noise changes the legible content, the consensus can be systematically biased.
  • domain assumption MLLM errors per character are correlated Bernoulli trials with average error probability below 0.5 and pairwise correlation rho
    Section 3.2 uses this model to motivate the ensemble and the effective sample size argument, but the values of epsilon_k and rho are not measured or validated.
  • domain assumption Ground-truth transcriptions of the 622 records are correct and non-blank fields are correctly identified
    Section 3.3 states the dataset is from Ancestry.com, but no inter-annotator agreement or quality audit is reported, and errors in ground truth would directly inflate measured CER.
  • domain assumption Character-wise majority vote over aligned transcriptions recovers the latent correct string
    Section 3.6 assumes the consensus operator is a valid estimator. Systematic model errors shared across augmentations can make the majority vote confidently wrong.
  • standard math Hoeffding concentration inequality applies with the effective sample size
    Section 3.2 invokes a Hoeffding-style bound using N_eff; this is standard probability but the applicability depends on the correlation model being accurate.

pith-pipeline@v1.3.0-alltime-deepseek · 11127 in / 12596 out tokens · 126891 ms · 2026-08-04T20:55:43.301895+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Improving MLLM Historical Record Extraction with Test-Time Image." pith.science (2026). https://pith.science/paper/RWNVR5J2

@misc{pith2026250909722,
  author       = {Pith},
  title        = {Pith review of: Improving MLLM Historical Record Extraction with Test-Time Image},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RWNVR5J2}},
  note         = {Machine review of arXiv:2509.09722}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We present a novel ensemble framework that stabilizes LLM based text extraction from noisy historical documents. We transcribe multiple augmented variants of each image with Gemini 2.0 Flash and fuse these outputs with a custom Needleman Wunsch style aligner that yields both a consensus transcription and a confidence score. We present a new dataset of 622 Pennsylvania death records, and demonstrate our method improves transcription accuracy by 4 percentage points relative to a single shot baseline. We find that padding and blurring are the most useful for improving accuracy, while grid warp perturbations are best for separating high and low confidence cases. The approach is simple, scalable, and immediately deployable to other document collections and transcription models.

Figures

Figures reproduced from arXiv: 2509.09722 by Taylor Archibald, Tony Martinez.

Figure 1
Figure 1. Figure 1: A death record segment under different distortions can [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Consensus CER vs. Number of Samples by Experiment [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of Ensemble Selection Methods. This graph [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Precision-Recall for Unanimous Predictions. This chart [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Across 3,684 predicted name fields, 870 had consistent [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 23 canonical work pages · 1 internal anchor

  1. [1]

    Advancements and Challenges in Handwritten Text Recognition: A Comprehensive Survey,

    W. AlKendi et al., “Advancements and Challenges in Handwritten Text Recognition: A Comprehensive Survey,”Journal of Imaging, vol. 10, no. 1, p. 18, Jan. 2024,ISSN: 2313-433X.DOI:10 . 3390 / jimaging10010018Accessed: May 31, 2025 (cit. on p. 1). [2]GPT-4V(ision) system card, https://openai.com/index/gpt-4v-system-card/, Feb. 2024. Accessed: Jun. 1, 2025 (c...

    work page 2024

  2. [3]

    Yang et al.,The Dawn of LMMs: Preliminary Ex- plorations with GPT-4V(ision), Oct

    Z. Yang et al.,The Dawn of LMMs: Preliminary Ex- plorations with GPT-4V(ision), Oct. 2023.DOI:10. 48550 / arXiv . 2309 . 17421arXiv:2309 . 17421 [cs]. Accessed: May 31, 2025 (cit. on pp. 1, 2)

    work page 2023

  3. [4]

    Team et al.,Gemini: A Family of Highly Capable Multimodal Models, May 2025.DOI:10.48550/ arXiv.2312.11805arXiv:2312.11805

    G. Team et al.,Gemini: A Family of Highly Capable Multimodal Models, May 2025.DOI:10.48550/ arXiv.2312.11805arXiv:2312.11805. Ac- cessed: Jun. 1, 2025 (cit. on pp. 1, 2)

    Pith/arXiv arXiv doi:10.48550/arxiv.2312.11805 2025

  4. [5]

    Kim et al.,Early evidence of how LLMs out- perform traditional systems on OCR/HTR tasks for historical records, Jan

    S. Kim et al.,Early evidence of how LLMs out- perform traditional systems on OCR/HTR tasks for historical records, Jan. 2025.DOI:10 . 48550 / arXiv . 2501 . 11623arXiv:2501 . 11623 [cs]. Accessed: Jun. 1, 2025 (cit. on pp. 1, 2)

    work page 2025

  5. [7]

    Confidence in the Rea- soning of Large Language Models,

    Y . Pawitan and C. Holmes, “Confidence in the Rea- soning of Large Language Models,”Harvard Data Science Review, vol. 7, no. 1, Jan. 2025,ISSN: 2644- 2353,DOI:10 . 1162 / 99608f92 . b033a087 Accessed: Jun. 1, 2025 (cit. on p. 1)

    work page 2025

  6. [8]

    A general method applicable to the search for similarities in the amino acid sequence of two proteins,

    S. B. Needleman and C. D. Wunsch, “A general method applicable to the search for similarities in the amino acid sequence of two proteins,”Journal of Molecular Biology, vol. 48, no. 3, pp. 443–453, Mar. 1970,ISSN: 0022-2836.DOI:10 . 1016 / 0022 - 2836(70)90057- 4Accessed: Jun. 1, 2025 (cit. on pp. 1, 4)

    work page 1970

  7. [9]

    Large language models for generative information extraction: A survey,

    D. Xu et al., “Large language models for generative information extraction: A survey,”Frontiers of Com- puter Science, vol. 18, no. 6, p. 186 357, Nov. 2024, ISSN: 2095-2236.DOI:10.1007/s11704-024- 40555-yAccessed: Jul. 7, 2025 (cit. on p. 2)

    doi:10.1007/s11704-024- 2024

  8. [10]

    LayoutLM: Pre-training of Text and Layout for Document Image Understanding,

    Y . Xu et al., “LayoutLM: Pre-training of Text and Layout for Document Image Understanding,” inPro- ceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Min- ing, Aug. 2020, pp. 1192–1200.DOI:10 . 1145 / 3394486.3403172arXiv:1912.13318 [cs]. Accessed: May 31, 2025 (cit. on p. 2)

    arXiv 2020

  9. [11]

    OCR-Free Document Understand- ing Transformer,

    G. Kim et al., “OCR-Free Document Understand- ing Transformer,” inComputer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVIII, Berlin, Hei- delberg: Springer-Verlag, Oct. 2022, pp. 498–517, ISBN: 978-3-031-19814-4.DOI:10 . 1007 / 978 - 3 - 031 - 19815 - 1 _ 29Accessed: May 31, 2025 (cit. on p. 2)

    work page 2022

  10. [12]

    End-to-End Document Recognition and Understanding with Dessurt,

    B. Davis et al., “End-to-End Document Recognition and Understanding with Dessurt,” inComputer Vi- sion – ECCV 2022 Workshops: Tel Aviv, Israel, Octo- ber 23–27, 2022, Proceedings, Part IV, Berlin, Hei- delberg: Springer-Verlag, Feb. 2023, pp. 280–296, ISBN: 978-3-031-25068-2.DOI:10 . 1007 / 978 - 3- 031- 25069- 9_19Accessed: Aug. 29, 2023 (cit. on p. 2). ...

    work page 2022

  11. [14]

    Robustness of Struc- tured Data Extraction from In-plane Rotated Docu- ments using Multi-Modal Large Language Models (LLM),

    A. Biswas and W. Talukdar, “Robustness of Struc- tured Data Extraction from In-plane Rotated Docu- ments using Multi-Modal Large Language Models (LLM),” Mar. 2024. Accessed: Jul. 7, 2025 (cit. on p. 2)

    work page 2024

  12. [15]

    Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Lan- guage Models Fine-Tuned with Human Feedback,

    K. Tian et al., “Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Lan- guage Models Fine-Tuned with Human Feedback,” inProceedings of the 2023 Conference on Empir- ical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds., Singapore: As- sociation for Computational Linguistics, Dec. 2023, pp. 5...

    work page 2023

  13. [16]

    Can LLMs Express Their Uncer- tainty? An Empirical Evaluation of Confidence Elic- itation in LLMs,

    M. Xiong et al., “Can LLMs Express Their Uncer- tainty? An Empirical Evaluation of Confidence Elic- itation in LLMs,” inThe Twelfth International Con- ference on Learning Representations, Oct. 2023. Ac- cessed: May 31, 2025 (cit. on p. 2)

    work page 2023

  14. [17]

    Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models

    M. Borszukovszki, I. P. de Jong, and M. Valdenegro- Toro,Know What You do Not Know: Verbalized Un- certainty Estimation Robustness on Corrupted Im- ages in Vision-Language Models, Apr. 2025.DOI: 10.48550/arXiv.2504.03440arXiv:2504. 03440 [cs]. Accessed: Jun. 4, 2025 (cit. on p. 2)

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2504.03440 2025

  15. [18]

    A post-processing system to yield reduced word error rates: Recognizer Output V oting Error Re- duction (ROVER),

    J. Fiscus, “A post-processing system to yield reduced word error rates: Recognizer Output V oting Error Re- duction (ROVER),” in1997 IEEE Workshop on Au- tomatic Speech Recognition and Understanding Pro- ceedings, Dec. 1997, pp. 347–354.DOI:10.1109/ ASRU.1997.659110Accessed: Jun. 1, 2025 (cit. on pp. 2, 4)

    arXiv 1997

  16. [19]

    Using Consensus Sequence V oting to Correct OCR Errors,

    D. Lopresti and J. Zhou, “Using Consensus Sequence V oting to Correct OCR Errors,”Computer Vision and Image Understanding, vol. 67, no. 1, pp. 39–47, Jul. 1997,ISSN: 1077-3142.DOI:10 . 1006 / cviu . 1996.0502Accessed: Jun. 1, 2025 (cit. on p. 2)

    work page arXiv 1997

  17. [20]

    Hidden Markov model- based ensemble methods for offline handwritten text line recognition,

    R. Bertolami and H. Bunke, “Hidden Markov model- based ensemble methods for offline handwritten text line recognition,”Pattern Recognition, vol. 41, no. 11, pp. 3452–3460, Nov. 2008,ISSN: 0031-3203. DOI:10.1016/j.patcog.2008.04.003Ac- cessed: Jun. 1, 2025 (cit. on p. 2)

    work page doi:10.1016/j.patcog.2008.04.003ac- 2008

  18. [21]

    Self-Consistency Improves Chain of Thought Reasoning in Language Models,

    X. Wang et al., “Self-Consistency Improves Chain of Thought Reasoning in Language Models,” inThe Eleventh International Conference on Learning Rep- resentations, Sep. 2022. Accessed: Jun. 3, 2025 (cit. on p. 2)

    work page 2022

  19. [22]

    Benchmarking Multiple Large Language Models for Automated Clinical Trial Data Extraction in Ag- ing Research,

    R. J. Young, A. M. Matthews, and B. Poston, “Benchmarking Multiple Large Language Models for Automated Clinical Trial Data Extraction in Ag- ing Research,”Algorithms, vol. 18, no. 5, p. 296, May 2025,ISSN: 1999-4893.DOI:10 . 3390 / a18050296Accessed: Jun. 3, 2025 (cit. on p. 2)

    work page 2025

  20. [23]

    Confidence-Aware Document OCR Error Detection,

    A. Hemmer et al., “Confidence-Aware Document OCR Error Detection,” inDocument Analysis Sys- tems: 16th IAPR International Workshop, DAS 2024, Athens, Greece, August 30–31, 2024, Proceed- ings, Berlin, Heidelberg: Springer-Verlag, Sep. 2024, pp. 213–228,ISBN: 978-3-031-70441-3.DOI:10 . 1007 / 978 - 3 - 031 - 70442 - 0 _ 13Accessed: Jul. 7, 2025 (cit. on p. 2)

    work page 2024

  21. [24]

    Zhang et al.,Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self- Improving OCR, Apr

    Y . Zhang et al.,Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self- Improving OCR, Apr. 2025.DOI:10 . 48550 / arXiv . 2504 . 11101arXiv:2504 . 11101 [cs]. Accessed: May 26, 2025 (cit. on p. 2)

    work page 2025

  22. [25]

    Ma et al.,Test-Time Generative Augmentation for Medical Image Segmentation, Jun

    X. Ma et al.,Test-Time Generative Augmentation for Medical Image Segmentation, Jun. 2024.DOI:10. 48550 / arXiv . 2406 . 17608arXiv:2406 . 17608 [cs]. Accessed: Jun. 3, 2025 (cit. on p. 2)

    work page 2024

  23. [26]

    Zhang et al.,A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?May 2025.DOI:10.48550/arXiv

    Q. Zhang et al.,A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?May 2025.DOI:10.48550/arXiv. 2503 . 24235arXiv:2503 . 24235 [cs]. Ac- cessed: Jun. 3, 2025 (cit. on p. 2)

    doi:10.48550/arxiv 2025

  24. [27]

    Kish,Survey Sampling(Survey Sampling)

    L. Kish,Survey Sampling(Survey Sampling). Ox- ford, England: Wiley, 1965 (cit. on p. 3)

    work page 1965

  25. [28]

    Probability Inequalities for sums of Bounded Random Variables,

    W. Hoeffding, “Probability Inequalities for sums of Bounded Random Variables,” inThe Collected Works of Wassily Hoeffding, N. I. Fisher and P. K. Sen, Eds., New York, NY: Springer, 1994, pp. 409– 426,ISBN: 978-1-4612-0865-5.DOI:10 . 1007 / 978 - 1 - 4612 - 0865 - 5 _ 26Accessed: Jun. 5, 2025 (cit. on p. 3)

    work page 1994

  26. [29]

    Data Augmentation for Recog- nition of Handwritten Words and Lines Using a CNN-LSTM Network,

    C. Wigington et al., “Data Augmentation for Recog- nition of Handwritten Words and Lines Using a CNN-LSTM Network,” inProceedings of the In- ternational Conference on Document Analysis and Recognition, ICDAR, vol. 1, IEEE Computer Society, Jan. 2018, pp. 639–645,ISBN: 978-1-5386-3586-5. DOI:10 . 1109 / ICDAR . 2017 . 110Accessed: Aug. 28, 2019 (cit. on p. 4)

    work page 2018

  27. [30]

    Wang et al.,Scaling Laws in Patchification: An Im- age Is Worth 50,176 Tokens And More, Feb

    F. Wang et al.,Scaling Laws in Patchification: An Im- age Is Worth 50,176 Tokens And More, Feb. 2025. DOI:10 . 48550 / arXiv . 2502 . 03738arXiv: 2502.03738 [cs]. Accessed: May 5, 2025 (cit. on p. 8)

    arXiv 2025

  28. [31]

    Three Things Everyone Should Know About Vision Transformers,

    H. Touvron et al., “Three Things Everyone Should Know About Vision Transformers,” inComputer Vi- sion – ECCV 2022, S. Avidan et al., Eds., Cham: Springer Nature Switzerland, 2022, pp. 497–515, ISBN: 978-3-031-20053-3.DOI:10 . 1007 / 978 - 3-031-20053-3_29(cit. on p. 8)

    work page 2022

  29. [32]

    Qiao et al.,UniViTAR: Unified Vision Trans- former with Native Resolution, May 2025.DOI:10

    L. Qiao et al.,UniViTAR: Unified Vision Trans- former with Native Resolution, May 2025.DOI:10. 48550 / arXiv . 2504 . 01792arXiv:2504 . 01792 [cs]. Accessed: Jun. 5, 2025 (cit. on p. 8)

    work page 2025