REVIEW 3 major objections 5 minor 31 references
Towards the Influence of Text Quantity on Writer Retrieval
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Writer retrieval from handwriting can match full-page accuracy with as few as four lines of text when deep-learning features are used.
desk verdict A useful low-text writer retrieval benchmark, but the four-line threshold is likely inflated by same-page lines in the gallery and needs a clean re-run. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the local-feature-plus-encoding pipeline: 32×32 patches are sampled at handwriting contours, described by RootSIFT or by a ResNet20 embedding trained with triplet loss, then pooled into a global descriptor through VLAD or NetVLAD with 100 clusters, followed by sum pooling, power normalization, and PCA whitening. The threshold claim comes from a line-merging protocol that gradually stacks consecutive lines of each document, normalizes the resulting mAP by the page-level mAP, and identifies the smallest number of lines for which normalized performance stays above 90%.
What would settle it
Re-run the line-merging experiment with all lines from the query's own document removed from the gallery, and check whether four stacked lines still keep mAP above 90% of the page-level value; if the normalized mAP drops below 90% once same-page lines are excluded, the reported threshold is an artifact of same-page leakage rather than a genuine text-quantity requirement.
Extended reading notes
Core claim
The paper claims to establish a text-quantity threshold for writer retrieval: for deep-learning feature extractors (ResNet20 with VLAD or NetVLAD), merging about four lines of text per sample recovers more than 90% of the full-page retrieval performance, whereas the handcrafted SIFT+VLAD baseline needs seven or eight lines to exceed 80% of page-level mAP. It also reports that reducing both query and gallery to a single line drops mAP by roughly 20–30 points relative to full pages, that a single-line query against a full-page gallery causes almost no drop, and that word-level retrieval reaches only about 9.3% mAP on CVL and 13.6% on IAM with the best method, though retrieving only identical word instances gives much higher scores (e.g., 71.4% mAP for the word 'Dann' on CVL). Across these settings, NetVLAD consistently outperforms classical VLAD encoding, especially when text is scarce.
Load-bearing premise
The line-level and merged-line evaluations rank other lines from the same document page in the gallery, so same-page lines are trivially easy positive matches; if those were excluded, the four-line threshold could shift.
Editorial extensions
If this is right
- Forensic or historical retrieval can use a one-line query against a full-page reference gallery with almost no loss in ranking accuracy.
- Deep-learning writer retrieval becomes computationally efficient at low text amounts because performance saturates at about 1,000 sampled features per line, while SIFT keeps improving up to 5,000.
- Any system that must match short fragments to short fragments should expect a 20–30 percentage point drop when only one line is available, so the four-line threshold is a practical lower bound for reliable snippet-to-snippet matching.
- Word-level writer retrieval is not reliable as a general ranking task, but word-specific retrieval (gallery restricted to the same word) is a viable fallback for low-text scenarios.
- Handcrafted features are the wrong choice for scarce-text writer retrieval; learned features with NetVLAD encoding degrade much more gracefully.
Reading between the lines
- The reported four-line threshold is probably optimistic for real cross-document retrieval, because the line-level gallery includes other lines from the query's own page, which share ink and writing conditions and are trivially easy matches; excluding same-page lines could push the threshold higher.
- The word-specific retrieval results suggest a practical hybrid: use text-independent global descriptors for an initial ranking, then re-rank using matches of identical or visually similar words when only a short snippet is available.
- Because deep features saturate at 1,000 samples per line, the bottleneck at four lines is feature aggregation rather than feature density; learned aggregation over lines could plausibly lower the threshold further.
- On historical or degraded handwriting, where page-level scores are already low (e.g., 21–36% mAP on Norhandv2), the relative gain from adding lines is smaller, so the four-line rule should be revalidated before being applied to historical collections.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies writer retrieval at line and word granularity, evaluating three VLAD-based systems (SIFT+VLAD, ResNet20+VLAD, ResNet20+NetVLAD) on CVL, IAM, and Norhandv2. It reports a 20-30% mAP drop when only one line is used as query and gallery, and claims that deep-learning methods recover above 90% of full-page performance with about four lines. The authors further analyze the effect of feature sampling density on retrieval performance and introduce word-specific retrieval as a task.
Significance. If the threshold result survives re-evaluation under a cross-document protocol, this is a practically relevant benchmark finding for forensic and historical writer retrieval, where documents are often fragmented. The systematic comparison of feature sampling density, the inclusion of a historical dataset, and the word-specific retrieval analysis are useful contributions. However, the central quantitative claim is currently entangled with a same-page evaluation protocol, so its significance will be determined by whether the four-line threshold persists when the gallery excludes samples from the query's own page.
major comments (3)
- [Section 4.3, Table 3 and Fig. 6] Line-level and merged-line retrieval include other samples from the query's own document page. For line-level retrieval, Section 4.3 states that each line serves as a query and 'the remaining lines are ranked' with no exclusion of lines from the same page. Since relevance is defined by the writer label (Eq. 5), same-page lines are positive matches and share page-specific ink, skew, and writing conditions, making them artificially easy. The same issue affects the Short Query - Short Gallery experiment: when lines of a document are merged into n-line chunks, other chunks from the same page remain in the gallery. The normalized mAP in Fig. 6 is therefore a same-document/cross-document mixture, and the conclusion that deep-learning methods 'reach more than 90% of the page level performance with four lines' is not a valid threshold for the forensic/historical scenarios the paper motivates, where the gallery contains different documents. Please re-run the line-level and merged-line experiments excluding all samples from the query's source page, and provide the resulting normalized mAP values in a table (currently the merged-line numbers appear only in a plot).
- [Section 4.3, Table 4] The Short Query - Long Gallery experiment reports no significant drop when using one line or half a page as query, but the protocol does not state whether the full page containing the query line or half-page was excluded from the gallery. If it was not excluded, the query is a sub-image of a gallery document, so the top-ranked match is trivially correct; this would explain the near-identical mAP values in Table 4. The authors must specify the exclusion criterion and, if the source page is currently included, re-evaluate after removing it. The sentence 'retrieving relevant documents with just a line of text as the query does not negatively impact the retrieval process' should be conditioned on the gallery composition.
- [Evaluation protocol and all result tables and figures] All results are reported as point estimates from a single training run per method and dataset. The ResNet20-based models are trained with stochastic triplet mining, and the headline 'about four lines' threshold is read off Fig. 6 from curves that have no error bars. Without reporting variance over multiple seeds (at least three) or, failing that, the fixed seed and a stability check, the numeric threshold and the 20-30% drop cannot be assessed for robustness. Please add mean and standard deviation over seeds, or state the seed and verify that the 90%-of-page-level threshold is stable across runs; also indicate the exact normalized mAP values at each line count (e.g., in a companion table).
minor comments (5)
- [Section 3.2] Typo: 'the two feature descriptors used in our word' should read 'used in our work'.
- [Abstract and Section 5] The abstract states a '20-30%' drop with one line, while the conclusion says performance 'drops by approximately a third'; reconcile these numbers, since 30% and 33% are not the same.
- [Section 4.3, Fig. 6] The y-axis label 'mAPmAPPage' appears to be a typo; it should be 'mAP / mAPPage' or 'normalized mAP'.
- [Table 6] The 'Hard Top-x' metric definition is ambiguous: 'indicating if the first x documents are written by the same writer' could mean at least one of the top x or all of the top x. Based on the decreasing values with increasing x, it appears to be the strict 'all of the first x' criterion; please state this explicitly.
- [Section 4.4, Qualitative Results] The observation that query words frequently retrieve words from the same line or adjacent lines (Fig. 8) is presented as evidence of style consistency, but it also illustrates the same-page confound affecting the quantitative results; consider discussing this as a limitation in the evaluation protocol.
Circularity Check
No significant circularity: the four-line threshold is an empirical measurement summary, not a fitted or self-referential prediction.
full rationale
This paper is an empirical benchmark study, not a derivation chain. The central claims—a 20–30% mAP drop for single-line retrieval and retention of more than 90% of page-level mAP with four lines—are post-hoc summaries of measurements taken under a fixed protocol defined in Section 4.1 (mAP via Eq. 4, relevance via Eq. 5). No target quantity is used to define the inputs: the vocabulary size (100), descriptor dimension (256), triplet margin (0.1), and feature counts are fixed hand-set hyperparameters, not fitted to reproduce the four-line threshold. The threshold is read off the measured curves in Fig. 6, so it is not a prediction forced by construction; it is an observed crossing point. Self-citations (e.g., NetMVLAD, SAGHOG, Kairacters) are used only as related-work context and baseline references, not as load-bearing mathematical premises that forbid alternatives or justify the result. The protocol does describe line-level retrieval as ranking 'the remaining lines' (Section 4.3), and the short-query/short-gallery merging keeps consecutive lines from the same document; this raises a genuine experimental-design validity question about whether same-page lines inflate the reported scores in forensic/cross-document scenarios. However, that is a potential empirical confound, not circularity: it concerns whether the measured quantity generalizes, not whether the measured quantity is definitionally equal to its inputs. Thus the finding is an honest non-finding: score 0.
Assumptions & free parameters
free parameters (5)
- VLAD vocabulary size =
100
- Global descriptor dimension after PCA =
256
- Triplet loss margin =
0.1
- Features per line (page/line level) =
5000
- Features per word (word level) =
500
assumptions (5)
- domain assumption Line and word annotations of CVL and IAM are accurate and usable as ground truth segments.
- ad hoc to paper Same-page lines are treated as valid retrieval targets in line-level and merged-line evaluation.
- domain assumption The self-defined IAM training/test split (writers with 2-3 pages for training) is representative.
- domain assumption Otsu binarization preserves writer-discriminative information.
- domain assumption Writer labels in CVL, IAM, and Norhandv2 are correct ground truth.
Cite this review
Pith. "Pith review of Towards the Influence of Text Quantity on Writer Retrieval." pith.science (2026). https://pith.science/paper/IA5ZRBVQ
@misc{pith2026250607566,
author = {Pith},
title = {Pith review of: Towards the Influence of Text Quantity on Writer Retrieval},
year = {2026},
howpublished = {\url{https://pith.science/paper/IA5ZRBVQ}},
note = {Machine review of arXiv:2506.07566}
}
read the original abstract
This paper investigates the task of writer retrieval, which identifies documents authored by the same individual within a dataset based on handwriting similarities. While existing datasets and methodologies primarily focus on page level retrieval, we explore the impact of text quantity on writer retrieval performance by evaluating line- and word level retrieval. We examine three state-of-the-art writer retrieval systems, including both handcrafted and deep learning-based approaches, and analyze their performance using varying amounts of text. Our experiments on the CVL and IAM dataset demonstrate that while performance decreases by 20-30% when only one line of text is used as query and gallery, retrieval accuracy remains above 90% of full-page performance when at least four lines are included. We further show that text-dependent retrieval can maintain strong performance in low-text scenarios. Our findings also highlight the limitations of handcrafted features in low-text scenarios, with deep learning-based methods like NetVLAD outperforming traditional VLAD encoding.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Arandjelovic, R., Gronát, P., Torii, A., Pajdla, T., Sivic, J.: Netvlad: CNN ar- chitecture for weakly supervised place recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. pp. 5297–5307 (2016)
work page 2016
-
[2]
In: 2012 IEEE Conference on Computer Vision and Pattern Recog- nition
Arandjelovic, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: 2012 IEEE Conference on Computer Vision and Pattern Recog- nition. pp. 2911–2918. Providence, RI (2012)
work page 2012
-
[3]
In: 2013 IEEE Conference on Computer Vision and Pattern Recognition
Arandjelovic, R., Zisserman, A.: All About VLAD. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition. pp. 1578–1585. Portland, OR, USA (2013)
work page 2013
-
[4]
PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors
Balntas, V., Johns, E., Tang, L., Mikolajczyk, K.: PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors. arXiv:1601.05030 [cs] (Jan 2016), arXiv: 1601.05030
work page Pith review arXiv 2016
-
[5]
Christlein, V.: Handwriting Analysis with Focus on Writer Identification and Writer Retrieval. Ph.D. thesis (2018)
work page 2018
-
[6]
Christlein, V., Bernecker, D., Angelopoulou, E.: Writer identification using VLAD encoded contour-zernike moments. In: 13th International Conference on Document Analysis and Recognition, ICDAR 2015, Nancy, France, August 23-26, 2015. pp. 906–910 (2015)
work page 2015
-
[7]
In: IEEE Winter Conference on Applications of Computer Vision
Christlein,V.,Bernecker,D.,Honig,F.,Angelopoulou,E.:Writeridentificationand verification using GMM supervectors. In: IEEE Winter Conference on Applications of Computer Vision. pp. 998–1005. Steamboat Springs, CO, USA (2014)
work page 2014
-
[8]
Christlein, V., Bernecker, D., Maier, A., Angelopoulou, E.: Offline Writer Iden- tification Using Convolutional Neural Network Activation Features. In: Pattern Recognition. vol. 9358, pp. 540–552 (2015)
work page 2015
Show all 31 references
-
[9]
In: 14th IAPR International Conference on Document Analysis and Recognition, ICDAR 2017, Kyoto, Japan, November 9-15, 2017
Christlein, V., Gropp, M., Fiel, S., Maier, A.K.: Unsupervised feature learning for writer identification and writer retrieval. In: 14th IAPR International Conference on Document Analysis and Recognition, ICDAR 2017, Kyoto, Japan, November 9-15, 2017. pp. 991–997 (2017)
2017
-
[10]
In: 13th IAPR International Workshop on Document Analysis Systems, DAS 2018, Vienna, Austria, April 24-27, 2018
Christlein, V., Maier, A.K.: Encoding CNN activations for writer recognition. In: 13th IAPR International Workshop on Document Analysis Systems, DAS 2018, Vienna, Austria, April 24-27, 2018. pp. 169–174 (2018)
2018
-
[11]
In: 2019 International Conference on Document Analysis and Recognition (ICDAR)
Christlein, V., Nicolaou, A., Seuret, M., Stutzmann, D., Maier, A.: ICDAR 2019 Competition on Image Retrieval for Historical Handwritten Documents. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 1505–1509. Sydney, Australia (2019)
2019
-
[12]
In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)
Fiel, S., Kleber, F., Diem, M., Christlein, V., Louloudis, G., Nikos, S., Gatos, B.: ICDAR2017CompetitiononHistoricalDocumentWriterIdentification(Historical- WI). In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). pp. 1377–1382. Kyoto (2017)
2017
-
[13]
In: Computer Analysis of Images and Patterns
Fiel, S., Sablatnig, R.: Writer Identification and Retrieval Using a Convolutional Neural Network. In: Computer Analysis of Images and Patterns. vol. 9257, pp. 26–37. Springer International Publishing (2015)
2015
-
[14]
In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778. IEEE, Las Vegas, NV, USA (2016)
2016
-
[15]
In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA
Jégou, H., Douze, M., Schmid, C.: On the burstiness of visual elements. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA. pp. 1169–1176 (2009) Towards the Influence of Text Quantity on Writer...
2009
-
[16]
In: 2018 16th International Conference on Fron- tiers in Handwriting Recognition (ICFHR)
Keglevic, M., Fiel, S., Sablatnig, R.: Learning Features for Writer Retrieval and Identification using Triplet CNNs. In: 2018 16th International Conference on Fron- tiers in Handwriting Recognition (ICFHR). pp. 211–216. Niagara Falls, NY, USA (2018)
2018
-
[17]
In: 12th International Conference on Document Analysis and Recognition, ICDAR 2013, Washington, DC, USA, August 25-28, 2013
Kleber, F., Fiel, S., Diem, M., Sablatnig, R.: CVL-DataBase: An off-line database for writer retrieval, writer identification and word spotting. In: 12th International Conference on Document Analysis and Recognition, ICDAR 2013, Washington, DC, USA, August 25-28, 2013. pp. 560...
2013
-
[18]
In: Artificial Intelligence and Security, vol
Liang,D.,Wu,M.,Hu,Y.:OfflineWriterIdentificationUsingConvolutionalNeural Network and VLAD Descriptors. In: Artificial Intelligence and Security, vol. 12736, pp. 253–264 (2021)
2021
-
[19]
In: 2013 12th International Conference on Doc- ument Analysis and Recognition
Louloudis, G., Gatos, B., Stamatopoulos, N., Papandreou, A.: ICDAR 2013 Com- petition on Writer Identification. In: 2013 12th International Conference on Doc- ument Analysis and Recognition. pp. 1397–1401. Washington, DC, USA (2013)
2013
-
[20]
Interna- tional Journal of Computer Vision60(2), 91–110 (2004)
Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. Interna- tional Journal of Computer Vision60(2), 91–110 (2004)
2004
-
[21]
International Journal on Document Analysis and Recognition5(1), 39–46 (2002)
Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for of- fline handwriting recognition. International Journal on Document Analysis and Recognition5(1), 39–46 (2002)
2002
-
[22]
International Journal on Document Analysis and Recognition (IJDAR)27(3), 265–274 (Jun 2024)
Mattick, A., Mayr, M., Seuret, M., Kordon, F., Wu, F., Christlein, V.: Evaluating learned feature aggregators for writer retrieval. International Journal on Document Analysis and Recognition (IJDAR)27(3), 265–274 (Jun 2024)
2024
-
[23]
In: 16th Interna- tionalConferenceonDocumentAnalysisandRecognition,ICDAR2021,Lausanne, Switzerland, September 5-10, 2021, Proceedings, Part II
Ngo, T.T., Nguyen, H.T., Nakagawa, M.: A-VLAD: an end-to-end attention-based neural network for writer identification in historical documents. In: 16th Interna- tionalConferenceonDocumentAnalysisandRecognition,ICDAR2021,Lausanne, Switzerland, September 5-10, 2021, Proceedings,...
2021
-
[24]
Otsu,N.:AThresholdSelectionMethodfromGray-LevelHistograms.IEEETrans- actions on Systems, Man, and Cybernetics9(1), 62–66 (1979)
1979
-
[25]
In: 26th International Conference on Pattern Recog- nition, ICPR 2022, Montreal, QC, Canada, August 21-25, 2022
Peer, M., Kleber, F., Sablatnig, R.: Writer retrieval using compact convolutional transformers and netmvlad. In: 26th International Conference on Pattern Recog- nition, ICPR 2022, Montreal, QC, Canada, August 21-25, 2022. pp. 1571–1578 (2022)
2022
-
[26]
In: Document Analysis and Recog- nition - ICDAR 2024
Peer, M., Kleber, F., Sablatnig, R.: SAGHOG: Self-Supervised Autoencoder for Generating HOG Features for Writer Retrieval. In: Document Analysis and Recog- nition - ICDAR 2024. pp. 121–138. Athens, Greece (2024)
2024
-
[27]
In: Pattern Recognition
Peer, M., Sablatnig, R., Serbaeva, O., Marthot-Santaniello, I.: Kairacters: Character-level-based writer retrieval for greek papyri. In: Pattern Recognition. pp. 73–88 (2025)
2025
-
[28]
IET Biom.11(1), 10–22 (2022)
Rasoulzadeh, S., BabaAli, B.: Writer identification and writer retrieval based on netvlad with re-ranking. IET Biom.11(1), 10–22 (2022)
2022
-
[29]
Raven, T., Matei, A., Fink, G.A.: Self-supervised Vision Transformers for Writer Retrieval, p. 380–396. Springer Nature Switzerland, Athens, Greece (2024)
2024
-
[30]
In: 17th In- ternational Conference on Frontiers in Handwriting Recognition, ICFHR 2020, Dortmund, Germany, September 8-10, 2020
Seuret, M., Nicolaou, A., Maier, A., Christlein, V., Stutzmann, D.: ICFHR 2020 competition on image retrieval for historical handwritten fragments. In: 17th In- ternational Conference on Frontiers in Handwriting Recognition, ICFHR 2020, Dortmund, Germany, September 8-10, 2020....
2020
-
[31]
In: Document Analysis and Recognition - ICDAR 2024
Tarride, S., Schneider, Y., Generali-Lince, M., Boillet, M., Abadie, B., Kermorvant, C.: Improving Automatic Text Recognition with Language Models in the PyLaia Open-Source Library. In: Document Analysis and Recognition - ICDAR 2024. p. 387–404 (2024)
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.